ML Model Serving Engineer — High-Throughput Inference Expert

Sesame HR; San Francisco (CA); Presencial; USD 200.000 - 320.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

401(k) match

Health, vision, and dental benefits

Unlimited PTO

Flexible spending account (FSA)

EAP

Stock options

Descripción de la vacante

Sesame is building a next-generation voice-enabled computer and seeks an ML engineering leader to turbocharge our serving layer for LLM, speech, and vision models. You will optimize performance and reliability.

Collaborate with ML infra and training teams to deploy fast, scalable inference, minimize latency, reduce startup times, and implement in-flight batching, caching, and custom kernels. Join a team with deep roots in Oculus, Meta, and beyond.

Formación

• Expert in differentiable array computing frameworks, preferably PyTorch.
• Expert in optimizing ML models for serving reliably at high throughput, with low latency.
• Significant systems programming experience; comfortable with internals of VLLM and PyTorch codebases.
• Significant performance engineering experience; bottleneck analysis in high-scale server systems.
• Up-to-date on latest techniques for model serving optimization.

Responsabilidades

• Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
• Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
• Modify and extend LLM serving frameworks like VLLM and SGlang to take advantage of the latest techniques in high-performance model serving.
• Work with the training team to identify opportunities to produce faster models without sacrificing quality.
• Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
• Find ways to reduce model initialization times without sacrificing quality.

Conocimientos

PyTorch

High-throughput serving

Low latency optimization

Performance engineering

Systems programming

Herramientas

VLLM

SGlang

Kubernetes

Descripción del empleo

Sesame is building a next-generation voice-enabled computer and seeks an ML engineering leader to turbocharge our serving layer for LLM, speech, and vision models. You will optimize performance and reliability.

Collaborate with ML infra and training teams to deploy fast, scalable inference, minimize latency, reduce startup times, and implement in-flight batching, caching, and custom kernels. Join a team with deep roots in Oculus, Meta, and beyond.