Staff LLM Inference Research Engineer

Mixpeek; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

Modal is seeking a highly skilled researcher to own end-to-end inference bets for our LLM serving platform. You will collaborate with the research lead to prioritize initiatives that reduce cost per token and tail latency, while working directly with customers and deployment engineers.

You will drive collaborations with external labs and shape the research agenda, turning frontier serving techniques into practical products and scalable solutions for production workloads.

Qualifications

• A research-leaning or systems background in LLM inference.
• Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling.
• A record of shipping research or systems that others build on, whether in a lab or industry.
• The drive to independently take a research bet from idea to result, working in the open with the rest of the team.
• Ability to work in-person, in NYC or San Francisco offices.

Responsibilities

• Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization, KV-cache and memory management, autoscaling for spiky serverless traffic, and related research tasks.
• Train and evaluate speculators against production traffic and feed insights back into models.
• Collaborate with Forward Deployed Engineers to deploy and tune models with customers.
• Expand collaborations with external labs (e.g., ZLab, SGLang, Flash Attention) and integrate learnings into research.
• Partner with engineering to translate frontier serving techniques into products and define future research directions.

Skills

LLM inference

Research mindset

Production systems

Open collaboration

In-person collaboration

Tools

Kernels

Quantization

Autoscaling

Job description

Modal is seeking a highly skilled researcher to own end-to-end inference bets for our LLM serving platform. You will collaborate with the research lead to prioritize initiatives that reduce cost per token and tail latency, while working directly with customers and deployment engineers.

You will drive collaborations with external labs and shape the research agenda, turning frontier serving techniques into practical products and scalable solutions for production workloads.