RL Environments Engineer (Remote, Contractor) - Preference Model

About the company

Preference Model is building the next generation of training data to power the future of AI. Today's models are powerful but fail to reach their potential across diverse use cases because so many of the tasks that we want to use these models are out of distribution. Preference Model creates RL environments where models encounter research and engineering problems, iterate, and learn from realistic feedback loops.

Our founding team has previous experience on Anthropic’s data team building data infrastructure, tokenizers, and datasets behind the Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential. We are backed by Tier 1 Silicon Valley VC.

Brief Description of the Role

We’re hiring RL Environments Engineers to design and build MLE environments. The goal is to teach LLMs better reasoning / advanced concepts from modern ML.
This is a remote contractor role with ≥4 hours overlap to PST and advanced English (C1/C2) required.

Minimum Qualifications:

• Strong Python (engineering-quality, not notebook-only)

• Docker + production mindset (debugging, reliability, iteration speed)

• Clear understanding of LLMs, their current limitations

• Ability to meet throughput expectations and respond quickly to feedback.

You may be a good fit if one of the following applies

• Strong expertise in CUDA or Pallas kernel development, optimizing non-trivial neural modules to specific hardware

• Expert knowledge in an active DL/ML research area, with publications or public code to show for it. We're especially interested in areas that are math-heavy and don't require massive compute. Examples include but aren't limited to:

• Architectures: SSMs, KANs, tensor networks, Hypernetworks, etc

• Generative modeling: diffusion, flow matching, probabilistic programming

• Geometry and Topology: geometric DL, topological DL, optimal transport

• Reasoning: neuro-symbolic methods, algorithmic reasoning

• Mechanistic Interpretability: circuit analysis, causal discovery, grokking

• Foundations: learning theory, control and constraint optimization

• ML for science: physics-informed neural nets, computational neuroscience, quantum chemistry, structural bioinformatics, chemoinformatics, genomics

• Numerical & simulation methods: stochastic time series, fluid dynamics, numerical relativity, Bayesian inference, Monte Carlo methods

• You have strong fundamentals and broad research interests, you read many papers, understand them deeply and have creativity to translate them into RLVR problems

• You have built complex interactive RL environments and have strong insights into open-ended RL-based learning systems