Staff Research Engineer, LLM Inference & Systems

Fdetimes; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

Modal is building an infrastructure layer for AI, enabling end-to-end LLM serving from training to deployment. As a research-focused engineer, you will own high-impact inference bets and work with the research lead to push them from concept to production.

You will collaborate with customers and Forward Deployed Engineers to tune models, investigate speculative decoding, quantization, and autoscaling, and help shape the research agenda for scalable, low-latency AI at scale.

Qualifications

• You have a research-leaning or systems background in LLM inference, with work you can point to.
• You’re fluent in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling.
• You have a record of shipping research or systems that other people build on, whether in a lab or in industry.
• You have the drive to independently take a research bet from idea to result, working in the open with the rest of the team.
• You’re able to work in person in our NYC or San Francisco office

Responsibilities

• Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization, KV-cache and memory management, autoscaling for spiky serverless traffic.
• Train custom speculators against real production traffic and feed what you learn back into target models; acceptance length is the metric that decides the win
• Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research
• Carry and expand collaborations with outside research labs, for example: our work with ZLab on DFlash, a speculator design built on KV injection and blockwise parallel drafting
• Work with engineering to turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh for models that keep training after deployment, observability for quality and latency in production, or even a next-generation inference engine

Skills

LLM inference

LLM serving stack

Quantization

Kernels

Tools

DFlash KV injection

Specdec

Flash Attention 4 kernels

Job description

Modal is building an infrastructure layer for AI, enabling end-to-end LLM serving from training to deployment. As a research-focused engineer, you will own high-impact inference bets and work with the research lead to push them from concept to production.

You will collaborate with customers and Forward Deployed Engineers to tune models, investigate speculative decoding, quantization, and autoscaling, and help shape the research agenda for scalable, low-latency AI at scale.