Remote-First Principal Engineer, Inference Performance

Runware

United States

Remote

USD 250,000 - 420,000

Full time

11 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Stock options
Remote-first
Generous PTO
Parental leave
Company retreats

Job summary

Runware is seeking a Principal Engineer, Model Performance to own the speed and cost of running 400,000+ models across tens of thousands of GPUs. You’ll write kernels, shape the roadmap and lead the Model Performance team in a fully remote, fast-moving environment.

You’ll work with the Sonic Inference Engine, TensorRT-LLM, vLLM, and Triton, crafting new quantization, parallelism, and batching strategies to push latency and cost to the Pareto frontier on real hardware.

Qualifications

  • 10+ years in GPU/HPC or deep learning systems engineering with production inference experience.
  • Shipping performance improvements that impact latency and cost on real hardware.
  • Hands-on CUDA and GPU architecture expertise: kernels, memory, Tensor Cores, fused paths.
  • Production experience with inference engines (TensorRT-LLM, vLLM, SGLang) and serving techniques.
  • Fluent in Python and C++, comfortable in PyTorch internals and Triton.
  • Experience leading engineers and building a performance team.
  • Ability to own outcomes in fast-moving, ambiguous environments.

Responsibilities

  • Own inference performance end to end across the fleet: kernels, serving, model-level optimizations, benchmarks.
  • Set the roadmap for models, hardware, and techniques based on real traffic and cost.
  • Do the hardest work yourself: write and tune CUDA kernels, quantization formats, and parallelism strategies.
  • Make new models fast on day one with production-grade deployment pipelines.
  • Shape the serving stack (TensorRT-LLM, vLLM, SGLang) and contribute upstream where it pays.
  • Own benchmarking and regression systems integrated into CI to prevent regressions.
  • Collaborate with infrastructure and hardware teams on GPU memory, storage, and network bottlenecks.
  • Build and lead the team, hiring and mentoring principal/senior engineers.
  • Leverage AI for kernel search, profiling, benchmarking, and documentation.

Skills

CUDA kernels
GPU architecture
Python
C++
PyTorch internals
Triton
TensorRT-LLM
vLLM
SGLang
Quantization

Tools

Nsight Systems/Compute
NVIDIA GPUs
CUDA Graphs
NCCL

Job description

Runware is seeking a Principal Engineer, Model Performance to own the speed and cost of running 400,000+ models across tens of thousands of GPUs. You’ll write kernels, shape the roadmap and lead the Model Performance team in a fully remote, fast-moving environment.

You’ll work with the Sonic Inference Engine, TensorRT-LLM, vLLM, and Triton, crafting new quantization, parallelism, and batching strategies to push latency and cost to the Pareto frontier on real hardware.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Principal Engineer, Inference Performance
Remote Principal Engineer, Inference Performance

Runware • Palo Alto (CA)

Remote
USD 260,000 - 380,000
Stock options
Remote-first culture
Generous PTO
Principal Engineer, Model Performance — Remote Inference at Scale
Principal Engineer, Model Performance — Remote Inference at Scale

Runware • San Francisco (CA)

Remote
USD 180,000 - 280,000
Stock options
Remote-first culture
Flexible schedule
+1
Principal Engineer, Model Performance (Remote, Equity)
Principal Engineer, Model Performance (Remote, Equity)

Runware • San Jose (CA)

Remote
USD 230,000 - 420,000
Meaningful stock options
Remote-first with in-person retreats
Generous paid time off
Remote Principal Engineer, Model Performance & Inference
Remote Principal Engineer, Model Performance & Inference

Speedrun Talent Network • United States

Remote
USD 250,000 - 350,000
Stock options
Remote-first culture
Generous PTO
+2
Principal Engineer, Model Performance
Principal Engineer, Model Performance

present of AI • United States

Remote
USD 140,000 - 210,000
Senior Software Engineer: Performance ML Inference (Remote)
Senior Software Engineer: Performance ML Inference (Remote)

positron.ai • United States

Remote
USD 150,000 - 230,000
Remote-first policy
Comprehensive benefits (medical, etc.)
Unlimited paid time off
+3
Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)
Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)

positron.ai • United States

Remote
USD 120,000 - 180,000
Remote-first
Company-provided computer
Medical, dental, and vision coverage
+4
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Speedrun Talent Network • United States

Remote
USD 250,000 - 350,000
Stock options
Remote-first culture
Generous PTO
+2
Principal GPU Inference Optimization Engineer
Principal GPU Inference Optimization Engineer

digitalocean98 • United States

Hybrid
USD 250,000 - 312,000
Hybrid work model
Equity compensation
Conference & education reimbursement
+1
Senior Principal: LLM Inference & GPU Optimization
Senior Principal: LLM Inference & GPU Optimization

People in AI • United States

Remote
USD 230,000 - 280,000
Bonus
Equity
Remote work within United States