Senior LLM Inference Architect — Heterogeneous Hardware

d-Matrix inc.; Santa Clara (CA); On-site; USD 130,000 - 170,000; Full time

Benefits offered by this job

Competitive compensation

Equity

Inclusive work environment

Job summary

d-Matrix inc. is hiring end-to-end inference engineers to innovate LLM solutions in Santa Clara, CA. You will work with novel hardware and software to optimize performance and deploy advanced inference systems.

Ideal candidates possess strong Python and C/C++ skills, along with a deep understanding of generative AI architectures and extensive experience in performance optimization of LLM frameworks. Join a small, senior team committed to cutting-edge AI development.

Qualifications

• 10+ years of relevant engineering experience or equivalent.
• Experience optimizing attention kernels, KV cache, batching strategies, and quantization (INT8/FP8/INT4).
• Experience contributing to a major inference framework.

Responsibilities

• Prototype emerging LLM inference use cases.
• Build proof-of-concept systems.
• Develop and tune custom kernels for optimal performance.

Skills

Python

C/C++

LLM inference optimization

CUDA/Triton

Performance profiling tools

Education

Bachelor's degree in Computer Science or Electrical Engineering

Master's or PhD preferred

Tools

vLLM

SGLang

TensorRT-LLM

ONNX Runtime

Job description

d-Matrix inc. is hiring end-to-end inference engineers to innovate LLM solutions in Santa Clara, CA. You will work with novel hardware and software to optimize performance and deploy advanced inference systems.

Ideal candidates possess strong Python and C/C++ skills, along with a deep understanding of generative AI architectures and extensive experience in performance optimization of LLM frameworks. Join a small, senior team committed to cutting-edge AI development.