Senior ML Systems Engineer - Real-Time Inference & Scale

BITKRAFT Ventures; Mountain View (CA); On-site; USD 270,000 - 430,000; Full time

Benefits offered by this job

Relocation assistance

Equity

Benefits

Job summary

Inworld seeks a seasoned systems engineer to advance real-time AI inference at scale from Mountain View. You will push high-performance backends, optimize model serving, and drive production-ready systems from research to deployment.

You should have deep C++, CUDA, Rust or optimized Python experience, plus distributed systems with Kubernetes and multi-GPU setups. Relocation support is provided for the Mountain View office.

Qualifications

• Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, and speculative decoding.
• Proficiency in C++, CUDA, Rust, or highly optimized Python; strong profiling to maximize NVIDIA GPUs.
• Experience with Kubernetes, Ray, load balancing, multi-GPU/multi-node inference.

Responsibilities

• Own a model from research to containerization, optimization, and reliable production deployment.
• Improve latency and throughput for real-time multimodal inference at scale.
• Contribute to open-source projects or publish deep-dive technical work.
• Collaborate across teams to ensure robust distributed serving infrastructure.

Skills

Inference optimization

Model acceleration

High-performance systems

Distributed systems

Public work

Full-cycle ownership

PhD or equivalent

C++

CUDA

Rust

Python (optimized)

Education

PhD in CS, Physics, Math, or equivalent practical experience

Tools

C++

CUDA

Rust

Python

Kubernetes

Ray

Job description

Inworld seeks a seasoned systems engineer to advance real-time AI inference at scale from Mountain View. You will push high-performance backends, optimize model serving, and drive production-ready systems from research to deployment.

You should have deep C++, CUDA, Rust or optimized Python experience, plus distributed systems with Kubernetes and multi-GPU setups. Relocation support is provided for the Mountain View office.