Senior AI Inference Platform Engineer

Tradermath; Chicago, Northern (IL, KY); Hybrid; USD 200,000 - 250,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) with discretionary match

Job summary

DRW, a diversified trading firm, is seeking an AI Inference Platform Engineer to build, operate, and optimize systems that serve large language and multimodal models across the firm. You will own the serving platform end-to-end, onboarding models, tuning runtimes and configurations, and improving latency, throughput, reliability, and cost.

You will work across inference runtimes, distributed systems, and production platform engineering with deep GPU literacy, ensuring the platform scales across

Qualifications

• You take a measurement-driven approach to performance optimization.
• You take ownership of performance problems across hardware, runtime, model, and infrastructure boundaries.
• You can move quickly and reprioritize as trading needs change while maintaining a high bar for production systems.

Responsibilities

• Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes.
• Build end-to-end performance profiling and observability to identify bottlenecks from GPU kernels through multi-node inference systems.
• Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies.
• Own day-0 model onboarding, determining the appropriate runtime, precision, sharding, memory, batching, cache policy, and serving configuration for new models.
• Maintain validated performance profiles for important model and hardware combinations, including performance and quality regression testing.
• Measure and monitor quality equivalence across serving configurations, including KV cache quantization, speculative decoding acceptance thresholds, precision choices, and model routing.
• Manage the production serving lifecycle of models, including versioning, compatibility, staging, canarying, promotion, rollback, and retirement.
• Partner with SRE and platform teams to automate model deployment, distribution, production readiness, observability, and reliable operation across environments.
• Optimize model placement, scaling, and resource allocation across the inference fleet to improve utilization and cost efficiency while meeting performance and reliability requirements.
• Design and operate multi-tenant scheduling and isolation across shared GPU capacity, balancing latency SLOs, throughput, and priority across concurrent workloads.

Skills

Measurement-driven

Ownership

Clear communication

Tools

Nsight

DCGM

OpenTelemetry

Prometheus

Grafana

Linux

CUDA Graphs

Job description

DRW, a diversified trading firm, is seeking an AI Inference Platform Engineer to build, operate, and optimize systems that serve large language and multimodal models across the firm. You will own the serving platform end-to-end, onboarding models, tuning runtimes and configurations, and improving latency, throughput, reliability, and cost.

You will work across inference runtimes, distributed systems, and production platform engineering with deep GPU literacy, ensuring the platform scales across