Senior AI Inference Engineer — Production-Scale LLMs

AI Chopping Block, Inc.; San Francisco, Northern (CA, KY); Hybrid; USD 250,000 - 300,000; Full time

Benefits offered by this job

Competitive compensation and equity

RSUs

Paid time off

Health, dental & vision insurance

401(k) with company match

Parental leave

Professional development

Wellness support

Meal allowance

Job summary

Crusoe is hiring an AI inference engineer to optimize large language model deployments in production. You will own the inference stack end to end, profiling time and cost, and collaborating with customer teams to tailor deployments for latency and throughput targets.

The role emphasizes hands-on coding, low-level optimization, and direct impact on real customer workloads, using Python as a primary language and strong GPU awareness.

Qualifications

• Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
• Hands-on experience shipping code in production with Python or C++.
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference.
• Comfort with LLM serving frameworks such as vLLM or SGLang and profiling down to the kernel level.
• Understanding GPUs and how they behave.

Responsibilities

• Bring current inference techniques into production and refine them.
• Design and optimize serving architectures, including prefill, decode disaggregation and routing.
• Profile and tune deployments against latency, throughput, and cost targets.
• Adapt and scale optimization methods across many ML models, especially large language models.
• Own delivery end to end from experiment to production.

Skills

Python

C++

LLM knowledge

Strong communication

Education

Bachelor's/Master's/PhD in CS or related field

Tools

Docker

Kubernetes

CUDA

Job description

Crusoe is hiring an AI inference engineer to optimize large language model deployments in production. You will own the inference stack end to end, profiling time and cost, and collaborating with customer teams to tailor deployments for latency and throughput targets.

The role emphasizes hands-on coding, low-level optimization, and direct impact on real customer workloads, using Python as a primary language and strong GPU awareness.