Senior Runtime Engineer for AI Hardware & Inference

Normal Computing Corp.; Mountain View (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Normal Computing in Mountain View is seeking a senior systems engineer to own the runtime that loads, schedules, and executes programs on our AI hardware. You will work against simulators and FPGA prototypes to shape the interface before silicon arrives, and collaborate with compiler and serving teams to ensure end-to-end efficiency.

You will lead architecture choices for device management, memory allocation, and host-device transfer, while keeping runtime overhead low for LLM inference and

Qualifications

• Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
• Strong systems programming in C++ or Rust: memory management, concurrency, and low-level performance.
• Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams.
• Understanding of LLM inference workloads and what they demand of a runtime.
• Comfort building software for hardware that is still evolving, from simulation through bring-up.

Responsibilities

• Own the runtime design: device management, memory allocation, host-device transfer, and executable loading.
• Hide differences across simulators, FPGA prototypes, silicon behind a stable interface.
• Manage hardware-resident state such as model weights for serving layers.
• Minimize runtime overhead on the critical path for LLM inference and maximize throughput.
• Define boundaries with the compiler and driver/firmware, coordinating with the hardware team.
• Set the runtime's technical direction and review designs with compiler and serving engineers.

Skills

Runtime development

Systems programming

End-to-end ownership

LLM workloads

Hardware simulation

Tools

IREE HAL

CUDA

ROCm

vLLM integration

Job description

Normal Computing in Mountain View is seeking a senior systems engineer to own the runtime that loads, schedules, and executes programs on our AI hardware. You will work against simulators and FPGA prototypes to shape the interface before silicon arrives, and collaborate with compiler and serving teams to ensure end-to-end efficiency.

You will lead architecture choices for device management, memory allocation, and host-device transfer, while keeping runtime overhead low for LLM inference and