Research Engineer - Evaluations
About the Role
Luma is pushing the boundaries of generative AI, building tools that redefine how visual content is created. We're seeking a Research Engineer to design and scale the infrastructure that powers our model evaluation efforts. This role is about building the pipelines, metrics, and automated systems that close the loop between model output, evaluation, and improvement. You'll work across research, engineering, and product teams to ensure our models are measured rigorously, consistently, and in ways that directly inform development.
Responsibilities
Design and implement scalable pipelines for automated evaluation of generative models, with a focus on visual and multimodal outputs (image, video, text, audio).
Develop novel metrics and evaluation models that capture qualities like fidelity, coherence, temporal consistency, and alignment with human intent.
Integrate evaluation signals into training loops (including reinforcement learning and reward modeling) to continuously improve model performance.
Build infrastructure for large-scale regression testing, benchmarking, and monitoring of multimodal generative models.
Collaborate with researchers running human studies to translate human evaluation frameworks into automated or semi-automated systems.
Partner with model researchers to identify failure cases and build targeted evaluation harnesses.
Maintain dashboards, reporting tools, and alerting systems to surface evaluation results to stakeholders.
Stay current with emerging evaluation techniques in generative AI, multimodal LLMs, and perceptual quality assessment.
Qualifications
Master's or PhD in Computer Science, Machine Learning, or a related technical field (or equivalent industry experience).
3+ years of experience building ML evaluation systems, model pipelines, or large-scale infrastructure.
Hands-on experience working with visual data (images and/or video), including evaluation, modeling, or data preparation.
Proficiency in Python and ML frameworks (PyTorch, JAX, or TensorFlow).
Familiarity with human-in-the-loop evaluation workflows and how to scale them with automation.
Strong background in machine learning, with experience in generative models (diffusion, LLMs, multimodal architectures).
Strong software engineering skills (CI/CD, testing, data pipelines, distributed systems).
Nice to Have
Experience with reinforcement learning or reward modeling.
Prior work on perceptual metrics, multimodal evaluation benchmarks, or retrieval-based evaluation.
Background in large-scale model training or evaluation infrastructure.
Experience designing metrics for perceptual quality
Familiarity with creative media workflows (film, VFX, animation, digital art).
Contributions to open-source evaluation libraries or benchmarks.
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
Lead the creation of Luma's test automation and QA function, building end-to-end frameworks and CI-integrated pipelines to ensure high-quality multimodal AI products.
Luma AI is hiring a product-minded backend engineer to build and own the enterprise platform powering secure, scalable APIs, authentication, and analytics for enterprise customers.
Novartis is hiring a Senior Scientist to drive discovery and validation of inflammatory disease targets using genetics, -omics and advanced cellular models in Cambridge, MA.
Experienced, hands-on research associates are needed to support and optimize plant delivery and transformation workflows in a fast-moving agricultural biotechnology startup.
Novartis is hiring a Senior Scientist, Metabolomics to lead LC‑MS bioanalytical studies and method development within its Translational Profiling team in Cambridge, MA.
AbbVie seeks an experienced Medical Director in Clinical Development (Neuroscience) to lead complex clinical programs, shape development strategy, and ensure high-quality clinical data and regulatory readiness.
Lead systems engineering for Philips’ next-generation interventional platform in San Diego, guiding cross-functional teams and external partners from requirements through launch while ensuring regulatory compliance.
Kitware is hiring an R&D Engineer to build scalable visualization and data-analysis software (VTK/ParaView) for HPC environments while contributing to open-source projects and mentoring junior engineers.
Field-based Medical Science Liaison (Dermatology) to engage healthcare experts in Southern California, deliver scientific education, and support AbbVie’s medical and research objectives across the product lifecycle.
Experienced analytical scientist needed to develop and validate LNP-focused analytical assays and support platform and drug-product characterization at an early-stage RNA therapeutics venture backed by Fapon Biotech.
AbbVie seeks specialist physicians (MD/DO) for a two-year Physician Development Program based in North Chicago to develop clinical research and medical affairs expertise through rotational assignments across R&D functions.
AbbVie seeks early-career physician specialists to participate in a two-year onsite Physician Development Program in North Chicago, rotating through core R&D functions as Associate Medical Directors.
A two-year onsite rotational program at AbbVie’s North Chicago HQ that develops specialist physicians into Associate Medical Directors through cross-functional R&D rotations in oncology and related therapeutic areas.
Join Paragonix's R&D team as a Product Development Engineer to lead electro-mechanical product design, V&V, and manufacturing transfer for medical device preservation systems.
Lead formulation and drug delivery technology development at Foghorn Therapeutics to advance complex injectable and oral products from preclinical to clinical stages.
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
