Senior ML Engineer — Real-Time Inference at Scale

Inworld; Mountain View (CA); On-site; USD 250,000 - 300,000; Full time

Benefits offered by this job

Relocation assistance

Bonus and equity

Benefits package

Job summary

Inworld, a research-focused AI company in Mountain View, seeks engineers to optimize inference, accelerate models, and scale high-performance systems. You’ll own end-to-end delivery from research to production, with exposure to Kubernetes-based deployments and large-scale AI workloads.

You should have deep expertise in C++, CUDA, Rust, and Python, with a PhD or equivalent practical experience and a track record of contributing to open-source projects.

Qualifications

• PhD or equivalent practical experience in backend or ML systems.
• Strong knowledge of modern serving frameworks and techniques (e.g., vLLM, TRT-LLM).
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
• Proficiency in profiling and optimizing code for NVIDIA GPUs.
• Experience with Kubernetes, Ray, multi-GPU/multi-node inference and scalable systems.
• Contributions to open-source inference engines or deep-dive technical write-ups.
• Ability to take a model from research to containerized production serving.

Responsibilities

• Take a model from research to production: containerize and optimize serving.
• Design benchmarks and prototypes to measure latency and throughput.
• Contribute to open-source inference engines and document insights.
• Collaborate in a fast-paced, ambiguous environment and ship features.

Skills

Inference optimization

Model acceleration

High-performance systems

Distributed systems & scaling

Open-source contributions

Full-cycle ownership

Education

PhD in CS, Physics, Math, or equivalent practical experience

Tools

C++

CUDA

Rust

Python (optimized)

Job description

Inworld, a research-focused AI company in Mountain View, seeks engineers to optimize inference, accelerate models, and scale high-performance systems. You’ll own end-to-end delivery from research to production, with exposure to Kubernetes-based deployments and large-scale AI workloads.

You should have deep expertise in C++, CUDA, Rust, and Python, with a PhD or equivalent practical experience and a track record of contributing to open-source projects.