Senior Backend Engineer, Inference Platform
Skills
- Proven experience building large-scale, fault-tolerant distributed systems and API microservices
- Deep understanding of OS concepts: multithreading, memory management, networking, and storage performance
- Expert-level programming in one or more: Rust, Go, Python, or TypeScript
- Experience with Kubernetes or container orchestration
- Familiarity with GPU software stacks (CUDA, Triton, NCCL) and HPC technologies (InfiniBand, NVLink, MPI) is valuable
- Knowledge of modern LLMs and generative model serving patterns is a plus
- Experience with profiling, performance analysis, and system optimization
Responsibilities
- Build and optimize global and local request routing and low-latency load balancing across data centers and model engine pods
- Design and implement auto-scaling systems to dynamically allocate resources while meeting strict SLOs
- Develop multi-tenant traffic shaping, rate limiting, and resource regulation to ensure fairness and consistent UX
- Engineer trade-offs between latency and throughput to serve diverse workloads efficiently
- Optimize prefix caching and other techniques to reduce model compute and speed up responses
- Collaborate with ML researchers to bring new model architectures into production
- Continuously profile system-level performance, identify bottlenecks, and implement optimizations
Education
- Bachelor's or Master's degree in Computer Science, Computer Engineering, or related field
- Or equivalent practical experience in systems engineering and production deployments
Benefits
- Competitive base salary, equity, and benefits
- Health insurance
- Opportunity to work with state-of-the-art accelerators (H100, H200, GB200) at scale
- Direct collaboration with world-class researchers and open source communities
- High-impact work and deep technical ownership
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
Standard Fleet is hiring an early-stage DevOps engineer to own GCP infrastructure, CI/CD, and reliability for a fast-growing EV-focused fleet management platform.
Lead customer-facing integrations and end-to-end engineering projects to embed Foxglove in robotics and autonomy deployments across cloud and on‑prem environments.
Cognite seeks an AI Solutions Engineer to rapidly prototype and deploy GenAI-driven, full-stack solutions that showcase ATLAS AI capabilities for industrial customers.
Contribute to a cross-disciplinary team in Palo Alto building full-stack web applications and backend infrastructure to serve generative AI models for next-generation biological and medical research.
Lead development of scalable, photorealistic simulation environments and pipelines to power robotics research and training at OpenAI's San Francisco site.
Experienced Java/J2EE developer needed to modernize and migrate monolithic applications to AWS using containers, serverless services, and IaC for a contract engagement in Rockville.
Experienced C++ engineer needed to develop and optimize high-throughput remote-sensor data exploitation software for SciTec's next-generation missile warning programs in Boulder, CO.
Experienced AI Developer needed to develop and deploy secure AI/ML solutions for federal healthcare programs in a fully remote capacity.
Visa is hiring a Software Engineer in Highlands Ranch to design and build scalable, secure payment access systems using Java, OOP practices, and modern development tools.
Lead architecture and delivery of cloud-native, automated network solutions as a Senior Technical Architect at Mavenir, designing scalable systems for global communications service providers.
Ro is looking for a Senior Site Reliability Engineer to lead reliability, observability, and infrastructure automation for its production systems at scale.
Experienced full‑stack engineer needed to lead technical design and delivery of scalable, secure payment services at Visa.
Preql seeks a Senior Product Engineer to architect and deliver agentic AI workflows that convert natural-language instructions into auditable, production-ready financial data transformations.
together ai is a research-driven artificial intelligence company. we contribute leading open-source research, models, and datasets to advance the frontier of ai. our decentralized cloud services empower developers and researchers at organizations ...
3 jobs
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
