Data Scientist, GenAI Studio

About the Team  
The GenAI Studio is an internal incubator at College Board that launched in January 2024. Through the Studio, College Board is investing time and resources for Generative AI ideation. The Studio is responsibly experimenting with AI internally with staff, in the development of our products, and even in AP classrooms. Our rapid prototyping and experimentation help us make informed decisions about AI in our broader portfolio of offerings. The Studio is comprised of a diverse, cross-functional team of strategists, technologists, and product managers at College Board committed to solving the most impactful and interesting problems arising with AI. Across the team we divide the work around strategic enablement within our organization, transformative AI innovations that are both internally and externally facing and advancing AI Literacy in the education space. 

About the Opportunity   
As a Data Scientist for our GenAI Studio, you are passionate about applying new ideas and technologies to creatively solve challenging, real-world problems. You are an early-adopter who is enthusiastic about finding high-value applications for emerging tools and new technology. You are highly capable at evaluating large data sets and telling a story with data. You are versatile, adaptable, and comfortable with uncertainty and ambiguity. You are resourceful and able to forge ahead even when faced with incomplete information. While representing our Data Science functional team, you will work day-to-day within our Strategy team where you will experiment with different generative AI techniques and tools to find the best solution for the students and teachers who use our services. You will work on cross-functional projects that involve data analysis, machine learning, natural language processing, large language models, model evaluation, and more. You will also help us define and build infrastructure to support rapid prototyping and testing of AI solutions. You will have the opportunity to learn from and collaborate with other data scientists, strategists, analysts, and project managers who share your passion for innovation and impact.  We recognize that GenAI is an evolving field. If you're excited about this space and meet most (but not all) of these qualifications, we encourage you to apply. 

In this role, you will:  

Prototype new tools leveraging emerging technologies and models (50%) 

• Collaborate with users, product and strategy teams to scope and define problem statements 
• Guide product iteration using model results, user feedback, and experimentation insights 
• Use orchestration and prompt engineering techniques to build exploratory solutions (e.g., LangChain, LlamaIndex) 
• Independently manage and complete all the work needed to deliver in a rapid-prototyping environment 
• Rapidly prototype user-facing AI tools to test and validate hypotheses 
• Create customized or multi-staged models to address nuanced use cases 
• Analyze structured and unstructured data to uncover patterns and inform model usage 

Evaluate generative AI models and tools (30%) 

• Utilize state-of-the-art techniques to evaluate generative model outputs—including off-the-shelf, open-source, and custom models—using quantitative and qualitative methods 
• Judiciously evaluate the feasibility, fairness, and effectiveness of GenAI solutions and articulate considerations and concerns around implementing models in the context of specific business applications 
• Translate complex findings into clear insights for technical and non-technical audiences 

Learning, ideating, and sharing (20%) 

• Keep abreast of rapid technological developments and relevant product offerings, especially within the Generative AI space 
• Workshop new ideas with the immediate team and program stakeholders 
• Share knowledge of new developments with the team via chats, recurring meetings, and presentations when appropriate 
• Communicate clearly with non-technical stakeholders to enhance their understanding of what the technology is and is not capable of 

About you, you have: 

• 5+ years' experience in data science, product analytics, or adjacent fields with a focus on applied machine learning 
• Strong analytical and product thinking skills with a track record of delivering real-world impact 
• Experience working with large language models, generative AI APIs, or orchestration frameworks 
• Hands-on expertise with Python (e.g., Pandas, LangChain, Scikit-learn) and SQL, as well as familiarity with LLM evaluation tools 
• Experience in data and statistical analysis including experience working with and evaluating text data 
• Knowledge in developing applications on a Serverless or Microservices cloud platform (AWS/Azure/GCP, etc.) 
• Excellent communication and listening skills with comfort presenting to both technical and business audiences, including senior leadership  
• Eagerness to develop deep understanding of the problems students, educators, and College Board employees face in order to best address them 
• Bachelor’s degree in a quantitative discipline or computer science preferred, Master’s degree a strong plus 

For all roles at College Board: 
We are seeking individuals who are passionate about expanding educational and career opportunities and committed to mission-driven work. Candidates must be authorized to work in the United States for any employer and should possess clear and concise communication skills, both written and verbal. Proficiency in Microsoft Suite tools is preferred, though a willingness to learn is equally valued. We look for those with curiosity and enthusiasm for emerging technologies, particularly AI-driven solutions, and a proactive approach to independently learning and applying new digital tools. Most importantly, applicants should demonstrate the skills and mindsets aligned with College Board’s Operating Principles, reflecting a commitment to continuous growth, collaboration, and impact, notably:  

• A commitment to candid, timely, respectful feedback   
• A learner orientation and an openness to ideas and diverse perspectives  
• The ability to push for excellence through data-informed decision-making, iterative learning, external benchmarking and user-inputs  
• Strong problem-solving skills, including the ability to break down complex issues and identify clear paths forward  
• A track record of prioritizing high-impact work, simplifying complexity, taking initiative, and making decisions quickly with clarity of purpose  
• A habit of collaborating across differences, practicing empathy, and contributing to a culture of trust and shared success 

About Our Process

• Application review will begin immediately and will continue until the position is filled. This role is expected to accept applications for a minimum of 5 business days. 
• While the hiring process may vary, it generally includes: resume and application submission, recruiter phone/video screen, hiring manager interview, performance exercise such as live coding, a panel interview, a conversation with leadership and reference checks.    

What We Offer 
At College Board, we offer more than just a paycheck—we provide a meaningful career, a supportive team, and a comprehensive package designed to help you thrive. We’re a self-sustaining nonprofit that believes in fair and competitive compensation, grounded in your qualifications, experience, impact, and the market. 

A Thoughtful Approach to Compensation 

• The hiring range for this role is $150,000-$163,000. 
• Your exact salary will depend on your location, experience, and how your background compares to others in similar roles at the College Board. 
• We aim to make our best offer upfront, rooted in fairness, transparency, and market data. 
• We adjust salaries by location to ensure fairness, no matter where you live. 
• You’ll have open, transparent conversations about compensation, benefits, and what it’s like to work at College Board throughout your hiring process. Check out our careers page for more. 

#LI-TheAcorn 
#LI-Remote