Runtime Engineer — LLM Inference & Host Stack

MatX Inc.; Mountain View (CA); Hybrid; USD 160,000 - 475,000; Full time

Benefits offered by this job

Time off: 4 weeks PTO + 12 holidays +

Health: Company-subsidized Medical + D

Financial Wellbeing: 401K with company

Professional Development Budget

Team meals and commuting reimbursement

AI resources assistance

Mental wellbeing benefits

Parental leave

Job summary

MatX Inc. seeks a systems programmer to build the host-side interface library and manage the compiler→runtime contract. You will design the custom-kernel ABI, and implement Python bindings to move tensors from Python to accelerator hardware.

The role involves working with CUDA/ROCm-style accelerators, memory models, and high-performance computing stacks, delivering efficient runtime and serving throughput.

Qualifications

• Experience in a systems programming language with memory management and ABI work.
• Production Python interop layers (PyO3, ctypes, pybind11) experience.
• Experience designing and maintaining API/ABI contracts between teams.

Responsibilities

• Build the host-side interface library: memory management, DMA, streams and events, and sync primitives.
• Own and extend the executable format: compiler→runtime contract, versioning, weight/quantization layouts.
• Design the custom-kernel ABI and host-side marshaling for Python tensors to device.

Skills

Rust

C/C++

Go

FFI/ABI

Python interop

Job description

MatX Inc. seeks a systems programmer to build the host-side interface library and manage the compiler→runtime contract. You will design the custom-kernel ABI, and implement Python bindings to move tensors from Python to accelerator hardware.

The role involves working with CUDA/ROCm-style accelerators, memory models, and high-performance computing stacks, delivering efficient runtime and serving throughput.