Remote Principal AI Inference Performance Engineer

Runware; United States; Remote; USD 230,000 - 350,000; Full time

Benefits offered by this job

Stock options

Remote-first culture

Generous PTO

Company retreats

Job summary

Runware is seeking a Principal Engineer, Model Performance to own how fast and cheaply all models run across our fleet. You will write and optimize CUDA kernels, shape the serving stack, and drive production deployment of fast open models across GPUs.

You will build and lead the Model Performance team, collaborating with infrastructure, hardware, and open-source communities to reduce latency and cost while expanding the model catalog.

Qualifications

• 10+ years in GPU, HPC or deep learning systems engineering with production inference work.
• Shipping performance from a reference model to latency/cost Pareto frontier on real hardware.
• Deep CUDA and GPU architecture expertise: kernels, memory, Tensor Cores, fused kernels.
• Production experience with major inference engines and modern serving techniques.
• Fluent in Python and C++, comfortable with PyTorch internals and Triton.
• Experience leading engineers and building a performance-focused team.
• Ability to work with AI-accelerated tooling for kernel search and benchmarking.
• Comfort owning outcomes in a fast-moving, ambiguous environment.

Responsibilities

• Own inference performance end to end for the fleet: kernels, serving engines, model-level optimizations and benchmarks.
• Set the model performance roadmap and prioritize models, hardware and techniques.
• Write and tune CUDA kernels, fuse attention/MLP paths, experiment with quantization formats.
• Make new models fast on day one and build production-grade deployment pipelines.
• Shape serving stack decisions and contribute upstream where beneficial.
• Own benchmarking and regression system with hardware-aware metrics.
• Collaborate with infrastructure and hardware teams on memory, storage and network bottlenecks.
• Build and lead a high-impact team of principal and senior engineers.
• Leverage AI for kernel search, profiling, benchmarking and documentation.

Skills

CUDA kernels

GPU architecture

Python

C++

PyTorch internals

TensorRT-LLM

vLLM

SGLang

Triton

NCCL

Tools

Nsight Systems

Nsight Compute

CUDA Graphs

Torch.compile

Triton

InfiniBand / RoCE

Job description

Runware is seeking a Principal Engineer, Model Performance to own how fast and cheaply all models run across our fleet. You will write and optimize CUDA kernels, shape the serving stack, and drive production deployment of fast open models across GPUs.

You will build and lead the Model Performance team, collaborating with infrastructure, hardware, and open-source communities to reduce latency and cost while expanding the model catalog.