LLM Inference Performance Engineer: Speed & Efficiency

Baseten; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Baseten powers mission-critical inference for AI companies and enables models to run faster at scale. We are seeking an Inference Performance Engineer to work across the stack from the inference engine to scheduling, serving, and routing.

You will apply techniques like prefill/decode disaggregation, speculative decoding, and KV-cache management, and you will shape model performance, latency, and cost tradeoffs for our customers.

Qualifications

• Experience with inference optimization techniques and end-to-end profiling.

Responsibilities

• Implement and productionize inference techniques in runtime internals.

Skills

Inference optimization

Runtime internals

Performance profiling

Kernel optimization

Education

Bachelor's / Master's / PhD in CS, Engineering or Math

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Baseten powers mission-critical inference for AI companies and enables models to run faster at scale. We are seeking an Inference Performance Engineer to work across the stack from the inference engine to scheduling, serving, and routing.

You will apply techniques like prefill/decode disaggregation, speculative decoding, and KV-cache management, and you will shape model performance, latency, and cost tradeoffs for our customers.