Real-Time Speech Translation ML Engineer

lilt-corporate; Washington (IN); On-site; USD 140,000 - 210,000; Full time

Job summary

LILT is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend from live audio to translated output. You will work on Ray Serve on GPU Kubernetes clusters and our in-house adaptive MT models, ensuring low-latency streaming in production.

This hands-on role requires deep Python expertise, async programming, and a full-stack view of the ML stack, from model integration to production infrastructure, with strong collaboration across

Qualifications

• BS or MS in Computer Science or related field, or equivalent practical experience.
• 3+ years building production backend or ML serving systems in Python, including async programming (asyncio).
• Experience with real-time streaming transport (WebSocket or gRPC) and latency budgets.
• Production ML serving on GPUs (Ray Serve, Triton, vLLM) with Docker and Kubernetes.
• Experience integrating speech/NLP models into production, preferably streaming ASR.
• Latency-focused mindset with profiling, instrumentation, and per-stage budgeting.
• US citizenship and residence in the United States.

Responsibilities

• Real-time pipeline architecture: Build and manage services for high-throughput, real-time audio and text streaming.
• ML model integration: Integrate and serve streaming ASR and MT models with latency budgets.
• Quality and confidence workflows: Develop model-based confidence scoring and routing for human intervention.
• Infrastructure and scale: Scale production ML infrastructure on GPU-accelerated Kubernetes clusters.
• Latency engineering: Instrument, identify bottlenecks, and reduce end-to-end latency.
• Interface and API definition: Define contracts for audio ingestion and downstream integrations.
• Collaboration and technical leadership: Align teams with clear APIs and contracts.

Skills

Python

Asyncio

WebSocket

gRPC

Ray Serve

Latency engineering

AI coding agents

Education

BSc/MS in Computer Science or related field

Tools

Docker

Kubernetes

Ray Serve

Job description

LILT is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend from live audio to translated output. You will work on Ray Serve on GPU Kubernetes clusters and our in-house adaptive MT models, ensuring low-latency streaming in production.

This hands-on role requires deep Python expertise, async programming, and a full-stack view of the ML stack, from model integration to production infrastructure, with strong collaboration across