Runtime Systems Engineer - Host Stack & AI Inference

MatX; Mountain View (CA); On-site; USD 160,000 - 475,000; Full time

Benefits offered by this job

Time off

Health benefits

Retirement plan

Professional development

Team meals

Commute on us

MatX extras

Tech stipend

Mental health

Job summary

MatX is building custom silicon for LLM inference and training. You will develop the host-side interface library, manage device memory, DMA, streams and events, and extend the executable format to enable safe evolution of compiler-runtime contracts.

You will design the kernel ABI and Python bindings (PyO3) to move tensors to the device, and contribute to the LLM inference serving stack with scheduler and memory management features.

Qualifications

• Strong systems programming experience with memory management and ABI/FFI work.
• Experience building Python interop layers (PyO3, ctypes, pybind11 or equivalent).
• Experience designing and maintaining API/ABI contracts between teams.

Responsibilities

• Build the host-side interface library for device memory management, DMA, streams and events.
• Own and extend the executable format and its versioning, including weight and quantization layouts.
• Design the custom-kernel ABI and host-side marshaling for Python tensors to the device.

Skills

Rust

C

C++

Go

memory management

FFI/ABI

Tools

PyO3

ctypes

pybind11

Job description

MatX is building custom silicon for LLM inference and training. You will develop the host-side interface library, manage device memory, DMA, streams and events, and extend the executable format to enable safe evolution of compiler-runtime contracts.

You will design the kernel ABI and Python bindings (PyO3) to move tensors to the device, and contribute to the LLM inference serving stack with scheduler and memory management features.