Senior AI Inference Systems Engineer — Deployment

d-Matrix; United States; Hybrid; USD 180,000 - 320,000; Full time

Job summary

d-Matrix is seeking a Principal System Software Engineer, AI Inference Execution, to join our team at the Santa Clara headquarters. You will help productize the software stack for our AI compute engine, driving development, enhancements, and maintenance of deployment software across the full stack.

You'll collaborate with ML, compiler, and hardware experts to optimize software-hardware co-design, scale deliverables, and meet tight development windows in a dynamic startup environment.

Qualifications

• BS in CS/engineering/Math/Physics with 12+ years of software experience; MS preferred with 6+ years.
• Strong grasp of system software, data structures, computer architecture, and ML fundamentals.
• Proficient in C/C++/Python in Linux and using standard development tools.
• Experience with distributed, high-performance software design and implementation.
• Self-motivated team player with ownership and leadership.

Responsibilities

• Productize the SW stack for our AI compute engine.
• Develop, enhance, and maintain next-generation AI deployment software.
• Collaborate across full-stack toolchain with ML, compiler, and hardware teams.
• Build and scale deployment infrastructure with a tight development window.

Skills

C/C++/Python

Distributed systems

High-performance software

System software

Data structures

Computer architecture

ML fundamentals

Ownership & leadership

Linux development

Kubernetes / MLOps

Education

BS in Computer Science/Engineering/Math/Physics

MS in Computer Science/Engineering/Math/Physics (preferred)

MS or PhD in CS/EE (preferred)

Tools

TensorRT-LLM

vLLM

SGLang

PyTorch

TensorFlow

ONNX Runtime

NCCL

OpenMPI

Kubernetes

Ray

Job description

d-Matrix is seeking a Principal System Software Engineer, AI Inference Execution, to join our team at the Santa Clara headquarters. You will help productize the software stack for our AI compute engine, driving development, enhancements, and maintenance of deployment software across the full stack.

You'll collaborate with ML, compiler, and hardware experts to optimize software-hardware co-design, scale deliverables, and meet tight development windows in a dynamic startup environment.