Senior Real-Time ML Engineer — Low-Latency Inference

AItoolnavio; Mountain View (CA); On-site; USD 250,000 - 300,000; Full time

Benefits offered by this job

Relocation assistance

Job summary

Inworld is seeking engineers who can advance agentic systems and sub-second multimodal inference at scale. We value fast learners from diverse backgrounds who can build, break, and understand complex systems.

You will own full lifecycle from research to production, optimize latency and throughput, and work with Kubernetes, Ray, and CUDA to scale across GPUs.

We offer relocation assistance and a Mountain View office with a base salary range of $250,000–$300,000 plus bonus and equity.

Qualifications

• Inference optimization experience with vLLM or TRT-LLM.
• Model acceleration techniques: quantization, distillation, caching, batching, attention methods.
• Proficient in backend/ML systems with performance profiling and optimization.
• Experience with distributed systems and multi-GPU/multi-node inference.

Responsibilities

• Take a model from research to containerized production.
• Own end-to-end deployment and reliability in production environments.
• Contribute to open-source inference engines and document architectural decisions.
• Focus on latency, throughput, and scalability to support real-time workloads.

Skills

Inference optimization

Model acceleration

High-performance systems

Distributed systems

Open-source

Full-cycle ownership

Background in CS/Physics/Math

Education

PhD in CS/Physics/Math or equivalent

Tools

Kubernetes

Ray

CUDA

Python

Job description

Inworld is seeking engineers who can advance agentic systems and sub-second multimodal inference at scale. We value fast learners from diverse backgrounds who can build, break, and understand complex systems.

You will own full lifecycle from research to production, optimize latency and throughput, and work with Kubernetes, Ray, and CUDA to scale across GPUs.

We offer relocation assistance and a Mountain View office with a base salary range of $250,000–$300,000 plus bonus and equity.