Production AI Inference Engineer for Fast LLMs

Crusoe; San Francisco (CA); On-site; USD 215,000 - 260,000; Full time

Benefits offered by this job

Competitive compensation and equity

Restricted Stock Units

Paid time off & holidays

Health, dental & vision insurance

401(k) with company match

Professional development

Mental health & wellness

Commuter benefits

Cell phone stipend

Job summary

Crusoe is hiring to accelerate AI workloads by making large language models run faster and cheaper in production. You will own the end-to-end inference stack, profile time and cost, and apply modern optimization techniques inside real deployments.

You will work with customer engineering teams to tailor deployments and ensure gains translate to production workloads, using Python and low-level optimizations in a hands-on role.

Qualifications

• A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
• Hands-on production experience shipping code in Python or C++.
• Familiarity with optimizing LLMs for high throughput/low latency.
• Comfort with LLM serving frameworks like vLLM or SGLang; profiling to kernel level.
• Knowledge of how GPUs are built and behave.

Responsibilities

• Own inference stack end-to-end from profiling to production deployment.
• Design and optimize serving architectures for latency, throughput, and cost.
• Tune deployments across various ML models, emphasizing large language models.

Skills

Python

C++

ML perf optimization

GPU knowledge

Strong communication

Education

Bachelor’s/Master’s/PhD in CS/Engineering/Math

Tools

Docker

Kubernetes

CUDA

vLLM

SGLang

Job description

Crusoe is hiring to accelerate AI workloads by making large language models run faster and cheaper in production. You will own the end-to-end inference stack, profile time and cost, and apply modern optimization techniques inside real deployments.

You will work with customer engineering teams to tailor deployments and ensure gains translate to production workloads, using Python and low-level optimizations in a hands-on role.