Staff/Principal ML Engineer

Cavendish Professionals; Deutschland; On-site; EUR 110,000 - 190,000; Full time

Job summary

Cavendish Professionals partners with a cutting-edge real-time AI research lab to hire a Staff/Principal ML Engineer. You will own the end-to-end model serving and inference optimization lifecycle, containerizing architectures for production and scaling sub-second, high-concurrency workloads on NVIDIA GPU clusters.

You'll architect distributed inference platforms using Ray, Kubernetes, and Docker, diagnose low-level bottlenecks, and mentor senior engineers while collaborating across research and

Qualifications

• Strong programming mastery in C++, Python, and/or Rust.
• Deep systems-level experience profiling, benchmarking, and optimizing high-throughput pipelines.
• Proven production experience with containerization and distributed orchestration.
• Fluent English for international collaboration.

Responsibilities

• Take end-to-end ownership of model serving and inference optimization.
• Containerize cutting-edge speech, voice, and LLM architectures for production.
• Optimize serving runtimes and kernels for sub-second latency on NVIDIA GPU clusters.
• Architect and scale distributed inference platforms using Ray, Kubernetes, and Docker.
• Diagnose bottlenecks and drive performance improvements across pipelines.

Skills

C++

Python

Rust

Distributed systems

GPU optimization

Tools

Kubernetes

Docker

Ray

TensorRT

vLLM

Job description

We are looking for an exceptional Staff/Principal ML Engineer to join our client, a cutting-edge real-time AI research lab and inference provider backed by top-tier venture funds. Powering hundreds of millions of daily active users and serving over 10 trillion LLM tokens monthly, the platform operates at massive scale behind low-latency voice and conversational AI APIs. In this role, you will take true end-to-end technical ownership of inference optimization, containerizing foundational models and scaling sub-second, high-concurrency workloads on high-performance GPU infrastructure.

What You’ll Be Doing:

• Taking complete end-to-end technical ownership of the model serving and inference optimization lifecycle.
• Taking cutting-edge speech, voice, and LLM architectures directly from the AI research team and containerizing them for production deployment.
• Optimizing serving runtimes and kernels for sub-second latency and ultra-high-concurrency workloads on distributed NVIDIA GPU clusters.
• Architecting and scaling distributed inference platforms using Ray, Kubernetes, and Docker to ensure 99.99% operational reliability under heavy global traffic.
• Diagnosing low-level system bottlenecks, memory allocations, and GPU compute performance across custom model serving pipelines.
• Driving architectural standards, mentoring senior team members, and collaborating cross-functionally between research and core platform engineering.

What We’re Looking For:

• Staff/Principal-level software engineering and ML systems background (equivalent to Google L5/L6 or Meta E5/E6).
• Deep systems-level experience profiling, benchmarking, and optimizing high-throughput, low-latency machine learning pipelines in production.
• Consultant mindset: exceptional technical leadership, high ownership, ability to make sound architectural trade-offs, and clear communication across engineering squads.
• Strong problem-solving abilities within low-level hardware-software interfaces and GPU architectures.

Must-Haves:

• Strong programming mastery in C++, Python, and/or Rust.
• Hands-on expertise with high-performance model serving frameworks: vLLM and TensorRT-LLM (TRT-LLM).
• Deep knowledge of CUDA programming, GPU kernel optimization, and NVIDIA GPU architecture.
• Proven production experience with containerization and distributed orchestration using Kubernetes, Ray, and Docker.
• Demonstrable track record deploying and running large-scale LLM/GenAI inference systems in production.
• Fluent English for daily international technical collaboration.