ML Model Serving Engineer - High-Performance Inference

Sesame; New York (NY); On-site; USD 175,000 - 280,000; Full time

Benefits offered by this job

401(k) max employer match: 3.5%

100% employer‑paid health, vision, and dental benefits

Unlimited PTO and sick time

Flexible spending account with employer matching

Guardian Employee Assistance Program

Competitive stock options

Job summary

Sesame in New York is seeking an expert in optimizing machine learning models to turbocharge their serving layer, integrating LLM, speech, and vision models. The ideal candidate has significant experience in systems programming and performance engineering, aiming to improve high-throughput, low-latency serving.

Join a team dedicated to pioneering advancements in voice agents and enjoy competitive benefits, including health coverage and stock options, with a salary range of $175K to $280K.

Qualifications

• Expert in some differentiable array computing framework, preferably PyTorch.
• Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
• Significant systems programming experience on high‑performance server systems.
• Significant performance engineering experience with bottleneck analysis.
• Up to date on the latest techniques for model serving optimization.

Responsibilities

• Turbocharge the serving layer with LLM, speech, and vision models.
• Collaborate with ML engineers to build a fast and reliable serving layer.
• Modify LLM serving frameworks like VLLM for performance.
• Identify opportunities with the training team to produce faster models.
• Reduce model initialization times without sacrificing quality.

Skills

Expert in PyTorch

Optimizing machine learning models

Systems programming experience

Performance engineering experience

Model serving optimization techniques

Tools

Kubernetes

Ray

GCP

AWS

Azure

Job description

Sesame in New York is seeking an expert in optimizing machine learning models to turbocharge their serving layer, integrating LLM, speech, and vision models. The ideal candidate has significant experience in systems programming and performance engineering, aiming to improve high-throughput, low-latency serving.

Join a team dedicated to pioneering advancements in voice agents and enjoy competitive benefits, including health coverage and stock options, with a salary range of $175K to $280K.