Principal Engineer, Model Performance — Remote Inference at Scale

Runware

San Francisco (CA)

Remote

USD 180,000 - 280,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Stock options
Remote-first culture
Flexible schedule
Twice-yearly company retreats

Job summary

Runware seeks a Principal Engineer, Model Performance to own end-to-end inference performance across our fleet of GPUs. You will write high-performance CUDA kernels, optimize model execution, and lead a growing team to push latency and cost improvements across diffusion, video, speech, 3D and LLM workloads.

You will collaborate with the Sonic Inference Engine, infrastructure and hardware teams, and external communities to land production-ready improvements across open models and internal

Qualifications

  • 10+ years in GPU, HPC or deep learning systems engineering with production inference optimization.
  • Proven track record shipping performance from reference to Pareto frontier on real hardware.
  • Hands-on CUDA and GPU architecture expertise: kernel design, memory hierarchy, Tensor Cores, CUDA Graphs.

Responsibilities

  • Own inference performance end to end for the fleet: kernels, serving engines, model-level optimizations and benchmarks.
  • Set the model performance roadmap and prioritize models, hardware and techniques.
  • Write and tune CUDA kernels, fuse attention and MLP paths, bring up quantization formats, and achieve parallelism across multi-GPU nodes.
  • Make new models fast on day one with production-grade deployment pipelines.
  • Shape the serving stack and decide what to adopt or fork and contribute upstream.
  • Own benchmarking and regression system integrated with CI to prevent regressions.
  • Collaborate with infrastructure and hardware teams on memory, storage, and network bottlenecks.
  • Build and lead the team: hiring, mentoring, and hands-on leadership.
  • Utilize AI for kernel search, profiling, benchmarking, and documentation.

Skills

CUDA kernels
GPU architecture
Python
C++
PyTorch internals
Triton
TensorRT-LLM
vLLM
SGLang
NCCL

Tools

TensorRT-LLM
vLLM
SGLang
Torch.compile
Triton
Nsight Systems
NVIDIA GPUs

Job description

Runware seeks a Principal Engineer, Model Performance to own end-to-end inference performance across our fleet of GPUs. You will write high-performance CUDA kernels, optimize model execution, and lead a growing team to push latency and cost improvements across diffusion, video, speech, 3D and LLM workloads.

You will collaborate with the Sonic Inference Engine, infrastructure and hardware teams, and external communities to land production-ready improvements across open models and internal

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Engineer, Model Performance (Remote, Equity)
Principal Engineer, Model Performance (Remote, Equity)

Runware • San Jose (CA)

Remote
USD 230,000 - 420,000
Meaningful stock options
Remote-first with in-person retreats
Generous paid time off
Remote Principal Engineer, Inference Performance
Remote Principal Engineer, Inference Performance

Runware • Palo Alto (CA)

Remote
USD 260,000 - 380,000
Stock options
Remote-first culture
Generous PTO
Remote-First Principal Engineer, Inference Performance
Remote-First Principal Engineer, Inference Performance

Runware • United States

Remote
USD 250,000 - 420,000
Stock options
Remote-first
Generous PTO
+2
Remote Principal Engineer, Model Performance & Inference
Remote Principal Engineer, Model Performance & Inference

Speedrun Talent Network • United States

Remote
USD 250,000 - 350,000
Stock options
Remote-first culture
Generous PTO
+2
Principal Engineer, Model Performance
Principal Engineer, Model Performance

present of AI • United States

Remote
USD 140,000 - 210,000
Senior AI Inference Performance Engineer
Senior AI Inference Performance Engineer

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Speedrun Talent Network • United States

Remote
USD 250,000 - 350,000
Stock options
Remote-first culture
Generous PTO
+2
Staff Engineer, AI Inference & Performance
Staff Engineer, AI Inference & Performance

Reflection • United States

Remote
USD 150,000 - 230,000
Top-tier compensation
Stock options
Health & wellness
+5
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Runware • San Jose (CA)

Remote
USD 230,000 - 420,000
Meaningful stock options
Remote-first with in-person retreats
Generous paid time off
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Runware • San Francisco (CA)

Remote
USD 180,000 - 280,000
Stock options
Remote-first culture
Flexible schedule
+1