Runtime Engineer

MatX Inc.; Mountain View (CA); Hybrid; USD 160,000 - 475,000; Full time

Benefits offered by this job

Time off: 4 weeks PTO + 12 holidays +

Health: Company-subsidized Medical + D

Financial Wellbeing: 401K with company

Professional Development Budget

Team meals and commuting reimbursement

AI resources assistance

Mental wellbeing benefits

Parental leave

Job summary

MatX Inc. seeks a systems programmer to build the host-side interface library and manage the compiler→runtime contract. You will design the custom-kernel ABI, and implement Python bindings to move tensors from Python to accelerator hardware.

The role involves working with CUDA/ROCm-style accelerators, memory models, and high-performance computing stacks, delivering efficient runtime and serving throughput.

Qualifications

• Experience in a systems programming language with memory management and ABI work.
• Production Python interop layers (PyO3, ctypes, pybind11) experience.
• Experience designing and maintaining API/ABI contracts between teams.

Responsibilities

• Build the host-side interface library: memory management, DMA, streams and events, and sync primitives.
• Own and extend the executable format: compiler→runtime contract, versioning, weight/quantization layouts.
• Design the custom-kernel ABI and host-side marshaling for Python tensors to device.

Skills

Rust

C/C++

Go

FFI/ABI

Python interop

Job description

What MatX is Building

MatX is building custom silicon for large-language-model inference and training, with HW/SW co-design across ISA, RTL, simulator, compiler, and kernels so each layer benefits from the others. The runtime owns the host-side stack and the contracts that bind those teams together.

What You'll Do Here

• Build the host-side interface library - device memory management, DMA, streams and events, sync primitives - that every compiler-emitted program runs on top of

• Own and extend the executable format: the compiler→runtime contract, its versioning, the weight and quantization layouts that let compiler and runtime evolve independently

• Design the custom-kernel ABI - calling convention, sync semantics, lifecycle - and the host-side marshaling layer (DLPack, the buffer protocol, numpy) that gets Python tensors to the device

• Build Python bindings via PyO3, with a C-ABI shim as the alternative integration path for downstream consumers

• Build the LLM inference serving stack - paged KV cache, continuous batching, request scheduling, token streaming - and the cluster orchestration primitives underneath it

• Bring up interconnect topology from the host and own the failure-detection and clean-teardown path for stop-restructure-resume recovery across racks

• Design what the chip exposes to host-side profilers and debuggers - perf counters, traces, and the Python surfaces ML engineers actually use - and hit measurable performance targets on runtime overhead and serving throughput

Who You Are

• Strong experience in a systems programming language - Rust, C, C++, or Go - including memory management, allocator design, and FFI/ABI work

• Have built Python interop layers in production (PyO3, ctypes, pybind11, or equivalent C-ABI bridging)

• Have designed and maintained API or ABI contracts between teams - versioning, evolution, breaking-change discipline - not just consumed someone else's

• Hands-on with at least one accelerator programming model (CUDA, ROCm, oneAPI Level Zero, TPU, or comparable) - enough to reason about device memory, async execution, and kernel launch

• ML-systems literate - comfortable with the training and inference loop, what collectives do, what a tensor layout is. Research depth not required.

Bonus Points If You Have

• LLM inference internals - vLLM, TensorRT-LLM, or SGLang (paged attention, scheduler design)

• Rust at depth, including proc macros, unsafe with soundness reasoning, and complex lifetime/trait work

• Custom allocator design (slab, paged, arena) or other low-level memory work

• ML framework integration experience (PyTorch custom backends, JAX/XLA, ONNX runtime)

• Profiler or tracing infrastructure work (perfetto, Nsight, or a custom stack)

• Driver-adjacent or kernel-bypass work, or prior new-silicon bring-up

Compensation

The US base salary for this full-time position is determined based on a variety of factors including role, experience, location, job-related skills, and relevant education and training. Career length is only a guideline for compensation.

• Early Career - $160,000 - $250,000 + equity

• Mid Career - $175,000 - $362,500 + equity

• Senior Career - $250,000 - $475,000 + equity

What We Offer

• Time off: 4 weeks PTO (accrued) + 12 company Holidays + up to 3 weeks remote work

• Health: Company-subsidized Medical (Kaiser or Anthem) for employees & dependents, Guardian Dental and Vision insurances for employee & dependents, and life insurance (employee only), plus HSA and FSA offerings via Lively. See attached benefits 1-pager and full benefits guide for more info on benefits.

• Financial Wellbeing: Choose from Roth IRA/ 401K (or both) retirement plans with up to 5% company contribution to 401K (even if you don't contribute). Also, 100% company-paid life insurance (up to $300K) and long-term disability insurances.

• Professional Development: $1500 Professional Development Budget (per year)

• Team Meals: MatX provides onsite team lunch & dinner Monday - Friday, with your choice of ordering via WeBox, Specialty’s or via our reimbursement system

• Commute on Us: Commute on our company Uber account, or reimburse your train rides. Either way, we pay 100% for your daily commute.

• MatX E[x]tras: $50/mo to use on the perk you value most

• Cell & Internet Reimbursement: $35/mo for cellular and $40/mo for wifi

• Mental Wellbeing: 100% paid mental health benefit via SpringHealth and Guardian EAP.

• Support to Parents: Up to 12 weeks paid parental leave regardless of path to parenthood, 10 weeks pregnancy disability leave, flexible return-to-work hours, and Benepass reproductive health & parental benefit.

• AI Resources: Up to $20K/month plus a dedicated internal AI Tooling Team to support your productivity

As part of our dedication to the diversity of our team and our focus on creating an inviting and inclusive work experience, MatX is committed to a policy of Equal Employment Opportunity and will not discriminate against an applicant or employee on the basis of race, color, religion, creed, national origin or ancestry, sex, gender, gender identity, gender expression, sexual orientation, age, physical or mental disability, medical condition, marital/domestic partner status, military and veteran status, genetic information or any other legally recognized protected basis under federal, state or local laws, regulations or ordinances.

All candidates must be authorized to work in the United States and work from our offices in Mountain View Tuesdays-Thursdays.

This position requires access to information that is subject to U.S. export controls. This offer of employment is contingent upon the applicants capacity to perform job functions in compliance with U.S. export control laws without obtaining a license from U.S. export control authorities.

MatX does not accept unsolicited resumes from individual recruiters or third-party recruiting agencies in response to job postings. No fee will be paid to third parties who submit unsolicited candidates directly to our hiring managers or People team and any resumes submitted are deemed to be the property of MatX.