Senior ML Infrastructure Engineer - Large-Scale GPU Serving

Selby Jennings; New York (NY); On-site; USD 180,000 - 250,000; Full time

Job summary

Selby Jennings is building a new machine learning infrastructure initiative and seeks an experienced engineer to own the design and deployment of large-scale model serving platforms. You will influence architecture decisions and work closely with researchers and engineers to push state-of-the-art models into production on GPU farms.

The role emphasizes ownership from day one, with opportunities to shape the platform roadmap and optimize hardware utilization across growing GPU clusters in a

Qualifications

• 8+ years in software, systems or ML infra.
• Deep experience with model serving and optimization at scale.
• Strong knowledge of CUDA, kernel optimization and GPU acceleration.
• Expert programming in C++ and Python.

Responsibilities

• Design and build large-scale serving and inference infra for state-of-the-art ML models.
• Optimize performance, latency and GPU utilization in production.
• Deploy and scale models across thousands of GPUs; improve efficiency and reliability.
• Develop compression, quantization, distillation techniques to maximize hardware performance.
• Move cutting-edge models from research to production with hardware integration.
• Collaborate with researchers and engineers to speed experimentation and deployment.
• Shape architecture and roadmap for a rapidly growing ML platform.

Skills

Large-scale infra

CUDA GPU

C++

Python

PyTorch

JAX

TensorFlow

Model serving

Inference optimization

GPU acceleration

Education

BS/MS/PhD in CS/Engineering

Tools

CUDA

Linux

Job description

Selby Jennings is building a new machine learning infrastructure initiative and seeks an experienced engineer to own the design and deployment of large-scale model serving platforms. You will influence architecture decisions and work closely with researchers and engineers to push state-of-the-art models into production on GPU farms.

The role emphasizes ownership from day one, with opportunities to shape the platform roadmap and optimize hardware utilization across growing GPU clusters in a