Distinguished Engineer, Scaled Out Inferencing

Bazeta; United States; A distancia; USD 320.000 - 489.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Equity

Benefits

Descripción de la vacante

NVIDIA is seeking a Distinguished Engineer to lead the architecture of distributed, high-performance AI inferencing pipelines for large-scale model serving in enterprise and cloud environments. You will drive deployment strategies, scaling, and co-optimization for peak performance across NVIDIA’s AI stack.

You will work with product managers, hardware engineers, and research teams, shaping industry standards and contributing to open-source initiatives.

Formación

• MS or PhD in Computer Science, Electrical Engineering, or a related field (or equivalent experience).
• 15+ years of experience in distributed systems, cloud infrastructure, or AI/ML engineering.
• Deep expertise in AI inferencing frameworks, including TensorRT-LLM, vLLM, and other model serving technologies.
• Strong understanding of GPU architecture, CUDA programming, and performance optimization techniques.
• Proven track record of leading cross-functional teams and driving technical strategy at an organizational level.

Responsabilidades

• Lead the architecture and development of distributed, high-performance inferencing pipelines for large-scale AI model serving across enterprise and cloud environments.
• Define the technical strategy for model deployment, versioning, and scaling to support hundreds of thousands of requests per second with low latency and high reliability.
• Drive hardware-software co-optimization to deliver peak performance at scale.
• Guide open-source efforts and community collaboration around Dynamo, TensorRT-LLM, and inference-related projects.
• Develop and maintain relationships with product managers, hardware engineers, and research teams to align inferencing strategy with business goals.

Conocimientos

Distributed systems

Cloud infrastructure

AI Inferencing

Kubernetes

Docker

CUDA

TensorRT-LLM

vLLM

Team leadership

Educación

MS or PhD in Computer Science/EE or related

Herramientas

Kubernetes

Docker

TensorRT-LLM

CUDA

Cloud-native tech

Descripción del empleo

# Distinguished Engineer, Scaled Out InferencingUnited States46 minutes agoID 5136756Price on request## DetailsEmployment type: Full-timeRemote: YesCompany: NVIDIA## DescriptionNVIDIA is leading the way in groundbreaking developments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. As a technology leader at NVIDIA, you will lead the development of our global strategy for scaled-out AI inferencing. You will architect the high-throughput, low-latency distributed pipelines and model serving strategies required for massive scale and production reliability. You will define and drive the technical roadmap for full-lifecycle, from deployment and versioning to automated scaling, across enterprise and cloud environments. Working with NVIDIA leadership, you will establish the systems and orchestration layers that enable the world’s most advanced AI models to run with peak performance and efficiency. This is a high-impact role where your technical decisions will influence the trajectory of AI systems globally. What You’ll Be Doing • Lead the architecture and development of distributed, high-performance inferencing pipelines for large-scale AI model serving across enterprise and cloud environments. • Define the technical strategy for model deployment, versioning, and scaling to support hundreds of thousands of requests per second with low latency and high reliability. • Drive hardware-software co-optimization, ensuring that NVIDIA’s AI stack—including Dynamo and TensorRT-LLM—delivers peak performance at scale. • Guide open-source efforts and community collaboration around Dynamo, TensorRT-LLM, and inference-related projects, shaping industry standards. • Develop and maintain relationships with key stakeholders, including product managers, hardware engineers, and research teams, to align inferencing strategy with broader business goals. • Stay at the forefront of inferencing technologies, evaluating emerging techniques and integrating them into NVIDIA’s offerings. What We Need to See • MS or PhD in Computer Science, Electrical Engineering, or a related field (or equivalent experience). • 15+ years of experience in distributed systems, cloud infrastructure, or AI/ML engineering, with a focus on large-scale model serving and inference. • Deep expertise in AI inferencing frameworks, including TensorRT-LLM, vLLM, and other model serving technologies. • Strong understanding of GPU architecture, CUDA programming, and performance optimization techniques. • Proven track record of leading cross-functional teams and driving technical strategy at an organizational level. • Experience with Kubernetes, Docker, and cloud-native technologies for deploying and scaling AI workloads. • Excellent communication skills and the ability to influence technical direction across the company. Ways to Stand Out From the Crowd • Contributions to open-source inference projects, such as Dynamo, TensorRT-LLM, or vLLM. • Experience with large-scale AI deployments in production environments, including multi-region and multi-cloud setups. • Deep knowledge of model optimization techniques, including quantization, pruning, and distillation. • Published research or patents in AI inference, distributed systems, or related fields. NVIDIA is leading the way in groundbreaking developments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. Our work opens up new universes to explore, enables amazing creativity and discovery, and powers what were once science fiction inventions from artificial intelligence to autonomous cars. NVIDIA is looking for great people like you to help us accelerate the next wave of artificial intelligence. The base salary range for this role is 320,000 USD – 488,750 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis. Learn more about NVIDIA benefits at www.nvidiabenefits.com . We are committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. This posting is for an existing vacancy. The use of AI tools in our application and hiring processes is limited to the following specific purposes: – Drafting, summarizing, and/or reviewing candidate resumes and cover letters – Translating candidate information – Drafting summaries of candidate interview notes or assessments We do not use AI tools to make, or assist with, employment-related decisions. Source: Jobspresso (https://jobspresso.co/job/distinguished-engineer-scaled-inferencing/)This listing comes from a partner feed. Apply on the source website.