Remote Principal Engineer, Model Performance

JoinStartup; United States; Remote; USD 180,000 - 320,000; Full time

Job summary

Runware is seeking a Principal Engineer, Model Performance to own end-to-end inference speed and cost across a distributed GPU fleet handling image, video, audio, and LLM workloads. You will guide the performance roadmap, write optimized CUDA kernels, and advance quantization and parallelism strategies on tens of thousands of GPUs.

You will lead a team and coordinate with infrastructure and hardware groups to push Pareto-frontier latency and cost.

Qualifications

• 10+ years in GPU, HPC or DL systems engineering.
• Experience shipping production-quality inference performance improvements at scale.
• Hands-on CUDA and GPU architecture expertise with kernel design and memory management.

Responsibilities

• Own end-to-end inference performance for the fleet across workloads.
• Set the model performance roadmap based on traffic and hardware.
• Write and tune CUDA kernels and quantization formats.
• Build production-ready deployment pipelines for newly released models.
• Shape and decide on serving stack components and integration points.
• Own benchmarking and regression systems with CI integration.
• Lead and grow the Model Performance team and set technical standards.
• Collaborate with infrastructure, hardware, and open-source communities.

Skills

CUDA

GPU architecture

TensorRT-LLM

PyTorch

C++

Python

Triton

NCCL

Quantization

Speculative decoding

Tools

Nsight Systems

Nsight Compute

CUDA Graphs

SGLang

TensorRT-LLM

Job description

Runware is seeking a Principal Engineer, Model Performance to own end-to-end inference speed and cost across a distributed GPU fleet handling image, video, audio, and LLM workloads. You will guide the performance roadmap, write optimized CUDA kernels, and advance quantization and parallelism strategies on tens of thousands of GPUs.

You will lead a team and coordinate with infrastructure and hardware groups to push Pareto-frontier latency and cost.