Senior Staff AI Inference Performance Engineer

d-Matrix; Santa Clara (CA); Remote; USD 180,000 - 320,000; Full time

Job summary

d-Matrix is seeking an experienced Senior Staff Inference Performance Engineer in Santa Clara, CA, with a remote option. You will own end-to-end performance for inference workloads on a heterogeneous platform, profiling across orchestration, scheduling, and GPU-accelerated components.

You will drive playbooks, mentor teams, and push for scalable, efficient deployment. The role emphasizes deep systems knowledge, LLM serving familiarity, and a drive to frame problems and deliver measurable

Qualifications

• 8+ years of hands-on performance or systems engineering.
• Profiling in unfamiliar, sparsely documented systems to find overhead.
• Deep systems fundamentals: Linux, NUMA, CPU pinning, cpusets, PCIe, memory and cache behavior, and network paths.

Responsibilities

• Diagnose performance problems across the full serving path from orchestration to inference internals.
• Isolate each problem to a specific layer and decide on fixes or handoff with evidence.
• Write minimal, reproducible problem statements for cross-stack findings so owning teams can act.
• Protect throughput, latency, and stability targets by closing gaps against bare metal.

Skills

Performance engineering

Linux internals

NUMA

Kubernetes

Python

C/C++

LLM inference

Profiling

Telemetry/benchmarks

GPU programming

Rust

Tools

vLLM

SGLang

TensorRT-LLM

Nsight

NCCL

CUDA

Kubernetes

Job description

d-Matrix is seeking an experienced Senior Staff Inference Performance Engineer in Santa Clara, CA, with a remote option. You will own end-to-end performance for inference workloads on a heterogeneous platform, profiling across orchestration, scheduling, and GPU-accelerated components.

You will drive playbooks, mentor teams, and push for scalable, efficient deployment. The role emphasizes deep systems knowledge, LLM serving familiarity, and a drive to frame problems and deliver measurable