Sr. Manager, Experimentation (Technical) - job 1 of 3

NBCUniversal is one of the world's leading media and entertainment companies. We create world-class content, which we distribute across our portfolio of film, television, and streaming, and bring to life through our theme parks and consumer experiences. We own and operate leading entertainment and news brands, including NBC, NBC News, MSNBC, CNBC, NBC Sports, Telemundo, NBC Local Stations, Bravo, USA Network, and Peacock, our premium ad-supported streaming service. We produce and distribute premier filmed entertainment and programming through Universal Filmed Entertainment Group and Universal Studio Group, and have world-renowned theme parks and attractions through Universal Destinations & Experiences. NBCUniversal is a subsidiary of Comcast Corporation.

Our impact is rooted in improving the communities where our employees, customers, and audiences live and work. We have a rich tradition of giving back and ensuring our employees have the opportunity to serve their communities. We champion an inclusive culture and strive to attract and develop a talented workforce to create and deliver a wide range of content reflecting our world.

Comcast NBCUniversal has announced its intent to create a new publicly traded company ('Versant') comprised of most of NBCUniversal's cable television networks, including USA Network, CNBC, MSNBC, Oxygen, E!, SYFY and Golf Channel along with complementary digital assets Fandango, Rotten Tomatoes, GolfNow, GolfPass, and SportsEngine. The well-capitalized company will have significant scale as a pure-play set of assets anchored by leading news, sports and entertainment content. The spin-off is expected to be completed during 2025.

Data is part of the DNA at NBCUniversal, and the Experimentation team plays a large role in driving our data driven culture. We own the technology that enables experimentation and act as subject matter experts to help teams measure the performance of the features and campaigns that customers experience every day. Our work is high visibility as it directly informs product, advertisement, marketing, and merchandising decisions.

The Sr. Manager, Experimentation (Technical) will own the statistical frameworks and processes that power our analytics and operational solutions to support self-service insights and decisioning within the organization. The team’s output enables automated dashboards and exploratory analysis that will be used by Product, Marketing, Engineering and Merchandising stakeholders to monitor performance and generate insights from experiments. This role will manage a team and work closely with other Data and Decision Science partners to define best practices and continuously improve statistical methods of how we analyze and share experiment results.

Responsibilities include, but are not limited to:

• Develop statistical frameworks and processes designed to analyze experiments to enable fast, reliable and accurate decision-making for product, marketing, ads and marketing teams
• Enhance experiment analysis methodologies, including Bayesian and frequentist approaches, variance reduction techniques, and sequential testing
• Collaborate with engineering to integrate statistical algorithms and new metrics into our experimentation platform
• Develop automated solutions for experiment diagnostics, anomaly detection, and guardrail monitoring
• Stay ahead of industry advancements, improving our statistical engine with the latest methodologies and best practices
• Communicate statistical concepts to stakeholders, including product managers, analysts, and engineering teams
• Lead Center of Excellence for Experimentation – Define best practices and frameworks for trustworthy online controlled experiments; drive cultural adoption of data-driven decision making across product and engineering teams.

• 6+ years of applied experience in experimentation at an ecommerce, social network, direct to consumer, media entertainment or similar tech company
• 3+ years of experience leading a technical team and developing talent
• BA/BS in technical or quantitative disciple or significant relevant work experience
• Direct experience working with 1st and/or 3rd party experimentation platforms (i.e. in-house, Optimizely, Eppo, Stat-sig, or Similar)
• Expert knowledge of experimental design, statistics and other analytical concepts (i.e., RCTs, Frequentist and Bayesian analysis)
• Advanced coding skills to implement and automate computational methods and algorithms, related but not limited to causal inference and experimentation (Python, R, SQL, etc.)

Desired Characteristics

• Interest in inferring experimentation led changes in human behavior and business impact of these changes with the help of big data
• A great storyteller and communicator that can use data to help influence product decisions and change opinions
• Combination of technical skill sets and business savvy to interface with all levels of the organization
• Past relevant experience in computational programming in either industry or academia
• Ability to work independently, multitask, and drive your own projects
• A passion for TV and movies and defining the future of streaming entertainment

Additional Requirements:

Hybrid: This position has been designated as hybrid, generally contributing from the office a minimum of three days per week.

This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks. Learn more about the benefits offered by NBCUniversal by visiting the Benefits page of the Careers website.

Salary range: $130,000 - $180,000 (bonus and long-term incentive eligible)

As part of our selection process, external candidates may be required to attend an in-person interview with an NBCUniversal employee at one of our locations prior to a hiring decision. NBCUniversal's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law.

If you are a qualified individual with a disability or a disabled veteran and require support throughout the application and/or recruitment process as a result of your disability, you have the right to request a reasonable accommodation. You can submit your request to [email protected].