Production ML Engineer: Real-Time Inference & MLOps

Cantina Labs; United States; Remote; USD 120.000 - 180.000; Vollzeit

Benefits dieser Stelle

Equity

Medical, dental, and vision insurance

Paid time off 42+ days

Zusammenfassung

Cantina Labs is seeking an ML Engineer to own the production inference stack for text-to-speech and speech recognition models. You will build low-latency, streaming services that run cost-efficiently on GPUs, and manage the inference engines, Kubernetes infrastructure, and performance benchmarking.

You will bridge research and production, owning latency, throughput, and accuracy as you scale. Collaboration with research and a strong focus on reproducibility are essential.

Qualifikationen

• Hands-on experience deploying ML models in production.
• Experience with end-to-end inference stacks from CUDA kernels to serving frameworks.
• Proficiency in GPU performance profiling and memory optimization.

Aufgaben

• Build and optimize streaming inference engines for TTS and ASR models.
• Optimize latency and throughput using batching, CUDA Graphs, torch.compile and mixed precision.
• Maintain numerical parity with the reference implementation via automated tests.
• Deploy and operate GPU inference services on Kubernetes: autoscaling, containerization, and IaC.
• Build CI/CD for model artifacts with versioning and reproducible releases.
• Own observability and benchmarking: latency at p50/p99, RT factor, GPU utilization, cost per request.
• Partner with research to productionize new models.

Kenntnisse

ML deployment

Python

PyTorch

GPU optimization

Kubernetes

CI/CD

Performance benchmarking

Tools

CUDA

TensorRT/RT

vLLM/Triton

Nsight Systems

Jobbeschreibung

Cantina Labs is seeking an ML Engineer to own the production inference stack for text-to-speech and speech recognition models. You will build low-latency, streaming services that run cost-efficiently on GPUs, and manage the inference engines, Kubernetes infrastructure, and performance benchmarking.

You will bridge research and production, owning latency, throughput, and accuracy as you scale. Collaboration with research and a strong focus on reproducibility are essential.