Senior AI Inference Performance Engineer

D-Matrix Corp.; Santa Clara (CA); Remote; USD 180,000 - 260,000; Full time

Job summary

d-Matrix, a leader in generative AI software/hardware integration, seeks a Senior Staff Inference Performance Engineer to own end-to-end performance of deployed workloads. You will profile, hypothesis-test, and drive changes across orchestration, batching, and accelerator paths to meet strict latency and throughput targets.

In this role, you bring deep systems knowledge, 8+ years of experience, and strong coding skills in Python and C/C++, along with practical Kubernetes experience.

Qualifications

• 8+ years of hands-on performance or systems engineering
• Track record of finding performance overhead through profiling in unfamiliar systems
• Deep systems fundamentals including Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior
• Experience running performance-sensitive workloads on Kubernetes
• Working knowledge of LLM inference serving including batching, KV-cache, prefill/decode, quantization
• Proficiency in Python and C/C++ to instrument, debug, patch code
• Clear technical writing that hardware, firmware, or platform teams can act on
• Sound judgment about ownership and handing off problems
• Fluency with AI and agentic tools for investigation and instrumentation

Responsibilities

• Diagnose performance problems across the full serving path, from orchestration to inference internals
• Isolate problems to a specific layer and owner, decide to fix or hand off
• Write minimal, reproducible problem statements for cross-stack findings
• Protect throughput, latency, and stability targets by closing gaps against bare metal
• Assess scale from single deployments to fleet operation including routing and shared-resource contention
• Capture workload node requirements in orchestration layer configurations
• Build lightweight telemetry and benchmarks for prototype stacks
• Turn recurring investigations into playbooks and mentor engineers

Skills

Performance engineering

Linux internals

CPU pinning

Profiling

Python

C/C++

System debugging

Root cause analysis

Cross-team collaboration

Tools

Kubernetes

CUDA

Nsight

NCCL

Job description

d-Matrix, a leader in generative AI software/hardware integration, seeks a Senior Staff Inference Performance Engineer to own end-to-end performance of deployed workloads. You will profile, hypothesis-test, and drive changes across orchestration, batching, and accelerator paths to meet strict latency and throughput targets.

In this role, you bring deep systems knowledge, 8+ years of experience, and strong coding skills in Python and C/C++, along with practical Kubernetes experience.