Senior ML Inference Engineer — High-Performance, Scalable

Inworld, Inc.; Mountain View (CA); On-site; USD 250,000 - 300,000; Full time

Benefits offered by this job

Relocation assistance

Office in Mountain View

Job summary

Inworld, Inc. in Mountain View seeks a Senior Machine Learning Engineer for inference. You will drive end-to-end model serving for high-volume consumer AI apps, tackling real-time challenges with optimized frameworks and scalable infrastructure.

PhD or equivalent experience is valued, and relocation assistance is offered. Join a team building state-of-the-art speech models, LLM serving, and high-throughput APIs to power production deployments.

Qualifications

• Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
• Proficiency in C++, CUDA, Rust, or highly optimized Python; you know how to profile code and squeeze performance on NVIDIA GPUs.

Responsibilities

• Own end-to-end model serving from research to production.
• Collaborate with research and platform teams to optimize latency and throughput.

Skills

Inference Optimization

Model Acceleration

High-Performance Systems

Distributed Systems

Public work

Education

PhD in CS, Physics, Math, or equivalent

Tools

Kubernetes

Ray

CUDA

C++

Rust

Python

Job description

Inworld, Inc. in Mountain View seeks a Senior Machine Learning Engineer for inference. You will drive end-to-end model serving for high-volume consumer AI apps, tackling real-time challenges with optimized frameworks and scalable infrastructure.

PhD or equivalent experience is valued, and relocation assistance is offered. Join a team building state-of-the-art speech models, LLM serving, and high-throughput APIs to power production deployments.