Real-Time Speech Translation ML Engineer

Lilt---Lega-Italiana-Per-La-Lotta-Contro-I-Tumori-1; Boston (MA); On-site; USD 140,000 - 210,000; Full time

Job summary

LILT in the United States is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend.

You will manage end-to-end streaming pipelines, integrate ASR and MT models, and optimize latency on GPU Kubernetes platforms, working with researchers and platform engineers. Strong Python, async skills, and hands-on problem solving are essential.

Qualifications

• 3+ years building production backend or ML serving systems in Python with async programming.
• Hands-on experience with real-time streaming transport: WebSocket or gRPC bidirectional streaming.
• Experience serving ML models in production on GPUs (Ray Serve, Triton, vLLM), with Docker and Kubernetes.
• Experience integrating speech or NLP models into production systems, ideally streaming ASR.
• A latency-engineering mindset: profiling, instrumenting, and optimizing real-time systems.
• Effective use of AI coding agents on top of fundamentals learned the hard way; can debug and review thoroughly.
• US citizenship and residence in the United States.

Responsibilities

• Real-time pipeline architecture: Build and manage services for high-throughput, real-time audio and text streaming.
• ML model integration: Integrate streaming ASR and MT models with latency budgets.
• Quality and confidence workflows: Develop routing and real-time updates with human-in-the-loop as needed.
• Infrastructure and scale: Architect ML infra on GPU-accelerated Kubernetes clusters with batching and autoscaling.
• Latency engineering: Instrument and optimize to meet production latency standards.
• Interface and API definition: Define contracts for audio ingestion and downstream integrations.
• Collaboration and technical leadership: Align teams with clear API interfaces and contracts.

Skills

Python backend development

Async programming (asyncio)

Real-time streaming

GPU/ML serving (Ray Serve)

Docker & Kubernetes

ASR/MT integration experience

Latency optimization

AI coding agents usage

US citizenship

Education

BS or MS in Computer Science or related field

Tools

Ray Serve

Triton

vLLM

Docker

Kubernetes

WebSocket

gRPC

Job description

LILT in the United States is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend.

You will manage end-to-end streaming pipelines, integrate ASR and MT models, and optimize latency on GPU Kubernetes platforms, working with researchers and platform engineers. Strong Python, async skills, and hands-on problem solving are essential.