GPU‑Focused AI Inference Platform Engineer

DRW; Chicago (IL); On-site; USD 200,000 - 250,000; Full time

Benefits offered by this job

Group medical

Dental and vision insurance

401k

Disability insurance

Life/AD&D insurance

Health Savings Account

Flexible Spending Accounts

Job summary

DRW is seeking an AI Inference Platform Engineer to build, operate, and optimize the systems that serve large language, vision, multimodal, and embedding models across DRW. You’ll own the end-to-end serving platform from onboarding models to performance, reliability, and cost optimization in a production environment.

You will optimize GPU-backed inference, design distributed architectures, and collaborate with SRE to automate deployment and observability.

Qualifications

• Hands-on experience serving LLMs on NVIDIA GPUs.
• Familiarity with modern inference runtimes and architectures.
• Experience with model serving, CI/CD, and observability.

Responsibilities

• Optimize LLM inference performance across NVIDIA GPUs and runtimes.
• Build end-to-end profiling and observability to locate bottlenecks.
• Design KV cache and distributed inference architectures.
• Onboard new models and configure runtimes, precision, sharding, and batching.
• Maintain performance profiles and run regression tests.
• Measure quality equivalence across serving configurations.
• Manage production serving lifecycle: versioning, staging, canarying, rollback.
• Collaborate with SRE to automate deployment and reliability.
• Optimize placement, scaling, and resource allocation for cost efficiency.
• Design multi-tenant scheduling and isolation across shared GPU capacity.

Skills

GPU inference

Performance profiling

Multi-tenant scheduling

Systems troubleshooting

Linux/Unix

Tools

TensorRT-LLM

vLLM

SGLang

Job description

DRW is seeking an AI Inference Platform Engineer to build, operate, and optimize the systems that serve large language, vision, multimodal, and embedding models across DRW. You’ll own the end-to-end serving platform from onboarding models to performance, reliability, and cost optimization in a production environment.

You will optimize GPU-backed inference, design distributed architectures, and collaborate with SRE to automate deployment and observability.