Remote Principal Engineer, Inference Performance

Runware; Palo Alto (CA); Remote; USD 260,000 - 380,000; Full time

Benefits offered by this job

Stock options

Remote-first culture

Generous PTO

Job summary

Runware is seeking a Principal Engineer, Model Performance to own how fast and cheaply every model runs across thousands of GPUs. You will write kernels, optimize fusion paths, and guide the team to push the Pareto frontier of latency and cost on real traffic.

You will collaborate with the Sonic Inference Engine and hardware teams, hire and mentor engineers, and ship production-ready solutions that scale for open models and future pods.

Qualifications

• Advanced degree or equivalent experience in a relevant field.
• Proven track record shipping production-grade inference optimizations at scale.
• Deep expertise in CUDA, kernel design and GPU memory hierarchies.
• Hands-on experience with major inference engines and multi-GPU deployment.

Responsibilities

• Own inference performance end to end for the fleet across GPUs and models.
• Design and implement fast kernels, fused ops and quantization techniques.
• Lead and mentor the Model Performance team and align roadmaps with traffic and costs.
• Collaborate with infrastructure, hardware and open-source communities; contribute upstream.
• Build and maintain benchmarks and regression systems for latency, throughput and cost.

Skills

CUDA

C++

Python

Deep learning systems

GPU architecture

Inference optimization

TensorRT-LLM

Multi-GPU scaling

Education

PhD or MS in CS/EE

Tools

Nsight Systems

NVIDIA CUDA Toolkit

Torch/Triton

Job description

Runware is seeking a Principal Engineer, Model Performance to own how fast and cheaply every model runs across thousands of GPUs. You will write kernels, optimize fusion paths, and guide the team to push the Pareto frontier of latency and cost on real traffic.

You will collaborate with the Sonic Inference Engine and hardware teams, hire and mentor engineers, and ship production-ready solutions that scale for open models and future pods.