Software Architect, NIM Factory

NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Software Architect to define and own the technical vision for the NVIDIA Inference Microservices (NIM) Factory. You will set the architectural direction for how we build, deploy, and scale enterprise-grade AI services to delight customers, while staying hands-on to guide our most critical implementations. The scope spans day-0 launches and the follow-through to harden them into enterprise-grade software, ensuring reliability, performance, and security across thousands of GPUs. You will shape our strategy for emerging challenges like disaggregated LLM inference and safeguard the long-term technical health of the platform.

What you'll be doing:

• Define the end-to-end technical architecture for the NIM Factory, from container build systems and CI/CD to Kubernetes deployment patterns and runtime optimization.

• Drive technical strategy and roadmap, making high-impact decisions on frameworks, technologies, and standards that empower dozens of engineering teams.

• Architect and influence the design of workflow orchestration systems that underpin the NIM factory.

• Guide and support senior engineers throughout the organization in building a culture centered on technical excellence and innovation.

• Advocate for guidelines in software development, encompassing API composition, automation, observability, and secure supply chain management.

• Collaborate with leadership across research, backend, SRE, and product to align technical vision with product goals and influence technical roadmaps.

What we need to see:

• 15+ years of experience building large-scale, production distributed systems.

• Consistent track record in a technical leadership or architect role, setting technical direction, and implementing.

• Deep architectural expertise in cloud-native technologies, including Kubernetes, containers, and microservices.

• Exceptional ability to mentor, and grow senior engineers with a passion for raising the technical bar of the entire organization.

• Proficiency in languages like Python for building tooling and services.

• Experience architecting solutions for GPU-accelerated or other high-performance computing workloads.

• Excellent communication and collaboration skills, with the ability to articulate complex technical concepts to diverse audiences and drive consensus.

• A degree in Computer Science, Computer Engineering, or a related field (BS or MS) or equivalent experience.

Ways to stand out from the crowd:

• Hands-on with LLM inference stacks (Triton Inference Server, TensorRT-LLM, vLLM).

• Experience optimizing large-model serving (KV cache sharding/paging, tensor/sequence parallelism, speculative decoding, dynamic batching).

• Experience architecting next-generation container build systems or CI/CD platforms at scale.

• Background with workflow orchestration engines (e.g., Temporal, Airflow) for complex, distributed processes.

• Expertise in designing multi-tenant, multi-cluster, or edge/air-gapped deployment architectures.

With competitive salaries and a generous benefits package, NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and. due to unprecedented growth, our exclusive engineering teams are rapidly growing. If you're a creative and autonomous engineer with a real passion for technology, we want to hear from you!

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 272,000 USD - 425,500 USD.Applications for this job will be accepted at least until September 18, 2025.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.