Ml engineer in real-time AI inference

Cavendish Professionals; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Компания Cavendish Professionals ищет исключительного Staff/Principal ML Engineer для ведения полной технической собственности над инференсом и оптимизацией; проектирует и разворачивает решения на GPU-инфраструктуре.

Работа 100% удаленно, международная коммуникация; требуется опыт работы с Ray, Kubernetes, Docker, vLLM и TensorRT-LLM; владение C++, Python, Rust; оканчивается на международном уровне.

Qualifications

• Опыт разработки и оптимизации ML-систем на продакшн-платформе.
• Опыт работы с Kubernetes, Docker и Ray.
• Глубокие знания CUDA и GPU-архитектур.

Responsibilities

• Полная техническая ответственность за lifecycle сервинга и инференса.
• Контейнеризация моделей для продакшн.
• Оптимизация latency и высокой пропускной способности.
• Работа с распределенными инференс-платформами на GPU кластерах.
• Наставничество и межфункциональное сотрудничество.

Skills

C++

Python

Rust

GPU optimization

Distributed systems

CUDA

Kubernetes

Ray

vLLM

TensorRT-LLM

Tools

Docker

NVIDIA CUDA

TensorRT

Job description

Описание: We are looking for an exceptional Staff/Principal ML Engineer to join our client, a cutting-edge real-time AI research lab and inference provider backed by top-tier venture funds. Powering hundreds of millions of daily active users and serving over 10 trillion LLM tokens monthly, the platform operates at massive scale behind low-latency voice and conversational AI APIs. In this role, you will take true end-to-end technical ownership of inference optimization, containerizing foundational models and scaling sub-second, high-concurrency workloads on high-performance GPU infrastructure.

Задачи:

Taking complete end-to-end technical ownership of the model serving and inference optimization lifecycle. Taking cutting-edge speech, voice, and LLM architectures directly from the AI research team and containerizing them for production deployment. Optimizing serving runtimes and kernels for sub-second latency and ultra-high-concurrency workloads on distributed NVIDIA GPU clusters. Architecting and scaling distributed inference platforms using Ray, Kubernetes, and Docker to ensure 99.99% operational reliability under heavy global traffic. Diagnosing low-level system bottlenecks, memory allocations, and GPU compute performance across custom model serving pipelines. Driving architectural standards, mentoring senior team members, and collaborating cross-functionally between research and core platform engineering.

Требования:

Staff/Principal-level software engineering and ML systems background (equivalent to Google L5/L6 or Meta E5/E6). Deep systems-level experience profiling, benchmarking, and optimizing high-throughput, low-latency machine learning pipelines in production. Proven track record scaling large distributed systems handling mission-critical, enterprise-grade traffic. Consultant mindset: exceptional technical leadership, high ownership, ability to make sound architectural trade-offs, and clear communication across engineering squads. Strong problem-solving abilities within low-level hardware-software interfaces and GPU architectures. Strong programming mastery in C++, Python, and/or Rust. Hands-on expertise with high-performance model serving frameworks: vLLM and TensorRT-LLM (TRT-LLM). Deep knowledge of CUDA programming, GPU kernel optimization, and NVIDIA GPU architecture. Proven production experience with containerization and distributed orchestration using Kubernetes, Ray, and Docker. Demonstrable track record deploying and running large-scale LLM/GenAI inference systems in production. Fluent English for daily international technical collaboration.

Условия:

100% Remote working model (Serbia, Germany, UK, Switzerland).