Staff Research Engineer - LLM Inference & Serving

Modal Labs; New York (NY); On-site; USD 180,000 - 240,000; Full time

Job summary

Modal Labs is building a platform that covers the entire life of an LLM—from training to deployment and observation. We seek a research‑leaning engineer with a systems background to own inference bets, optimize autoscaling, and ship ideas back into production.

Work in person in NYC or San Francisco, collaborate with customers and labs, and help turn frontier serving techniques into usable primitives and products with measurable impact on latency and quality.

Qualifications

• A research‑leaning or systems background in LLM inference with work you can point to.
• Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling.
• A record of shipping research or systems that others build on, whether in a lab or industry.
• The drive to independently take a research bet from idea to result, working in the open with the team.
• Ability to work in‑person, in our NYC or San Francisco office.

Responsibilities

• Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and more.
• Train custom speculators against real production traffic and feed learnings back into target models.
• Collaborate with customers alongside Forward Deployed Engineers to deploy and tune models and feed results back to research.
• Collaborate with external labs on projects like DFlash, specdec, and Flash Attention kernels.
• Turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh, observability for production latency/quality.
• Help shape the research agenda; your work guides the future.

Skills

LLM inference

Research experience

Systems background

Autoscaling

In-person collaboration

Tools

Quantization tooling

KV-cache techniques

Job description

Modal Labs is building a platform that covers the entire life of an LLM—from training to deployment and observation. We seek a research‑leaning engineer with a systems background to own inference bets, optimize autoscaling, and ship ideas back into production.

Work in person in NYC or San Francisco, collaborate with customers and labs, and help turn frontier serving techniques into usable primitives and products with measurable impact on latency and quality.