Lead Runtime Engineer, AI Inference Hardware

Normal Computing Corp.; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

Normal Computing Corp. is seeking a senior runtime engineer to own the design and implementation of the runtime that powers our memory-centric AI hardware.

You will shape device management, memory allocation, executable loading, and command submission, with a focus on stability across simulators, FPGA prototypes, and eventual silicon. This hands-on role requires deep systems programming in C++ or Rust, experience building runtimes or hardware abstraction layers, and a track record of delivering

Qualifications

• Experience building runtimes, drivers, or hardware abstraction layers.
• Strong systems programming in C++ or Rust.
• Proven end-to-end ownership of a complex systems component.

Responsibilities

• Own runtime architecture, device management, memory allocation, and host-device transfer.
• Hide hardware differences behind a stable interface across simulators, FPGA prototypes, and silicon.
• Manage resident model weights and other state for reliable serving.
• Keep runtime overhead off the critical path for LLM inference, optimizing latency and throughput.
• Define boundaries with compiler above the runtime and with driver/firmware below.

Skills

C++

Rust

Systems programming

Concurrency

Tools

IREE HAL

CUDA

ROCm

Job description

Normal Computing Corp. is seeking a senior runtime engineer to own the design and implementation of the runtime that powers our memory-centric AI hardware.

You will shape device management, memory allocation, executable loading, and command submission, with a focus on stability across simulators, FPGA prototypes, and eventual silicon. This hands-on role requires deep systems programming in C++ or Rust, experience building runtimes or hardware abstraction layers, and a track record of delivering