Real-Time Speech Translation Engineer

LILT; Washington (District of Columbia); On-site; USD 140,000 - 200,000; Full time

Job summary

LILT is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend end-to-end. You will integrate ASR and MT models on Ray Serve within GPU Kubernetes, delivering low-latency streaming translations.

You will work with researchers and engineers, specializing in production systems, AI tooling, and scalable infrastructure to ensure robust performance in production environments in the United States.

Qualifications

• BS or MS in Computer Science or related field, or equivalent practical experience.
• 3+ years building production backend or ML serving systems in Python, including async programming.
• Hands-on experience with real-time streaming transport (WebSocket or gRPC).
• Experience serving ML models in production on GPUs (Ray Serve, Triton, vLLM) with Docker and Kubernetes.
• Experience integrating speech/NLP models into production systems, preferably streaming ASR.
• Latency-engineering mindset with instrumented, low-latency systems.
• Familiarity with AI coding agents (Claude Code, Codex) in development workflows.
• US citizenship and residence in the United States.

Responsibilities

• Architect and maintain real-time audio/text streaming pipelines for low latency translation.
• Integrate and serve streaming speech recognition and machine translation models.
• Implement confidence scoring and routing of segments to human review when needed.
• Scale production ML infrastructure on GPU-accelerated Kubernetes clusters; manage batching and autoscaling.
• Measure and reduce end-to-end latency across the pipeline.
• Define interfaces for audio ingestion and downstream integrations; collaborate with frontend and platform teams.
• Lead cross-team alignment on API contracts and system integration.

Skills

Python (asyncio)

Real-time streaming

WebSocket/gRPC

ML model serving

Latency optimization

Agentic AI tooling

US citizenship

Education

BS or MS in Computer Science

Tools

Ray Serve

Triton

Docker

Kubernetes

Job description

LILT is building a new live translation product and seeks an ML Engineer to own the real-time speech translation backend end-to-end. You will integrate ASR and MT models on Ray Serve within GPU Kubernetes, delivering low-latency streaming translations.

You will work with researchers and engineers, specializing in production systems, AI tooling, and scalable infrastructure to ensure robust performance in production environments in the United States.