Lead Architect, AI Inference Performance

Runware; San Jose (CA); Remoto; USD 180.000 - 260.000; Tempo pieno

Vantaggi offerti da questo lavoro

Meaningful stock options

Remote-first culture

Generous paid time off

Company retreats

Descrizione del lavoro

Runware is seeking a Principal Engineer, Model Performance to own how fast and how cheaply every model runs across the fleet. You will set performance roadmaps, write kernel code, and lead the Model Performance team to scale with thousands of GPUs and open AI workflows.

You will collaborate with the Sonic Inference Engine, infrastructure and hardware teams, and external communities to push for lowest latency and cost per token while maintaining quality across 400k+ models.

Competenze

• 10+ years in GPU or deep learning systems engineering.

Mansioni

• Own inference performance end to end for the fleet: kernels, serving engines, model-level optimizations and benchmarks across workloads and GPUs.
• Set the model performance roadmap and prioritize models, hardware and techniques based on real traffic and cost.
• Write and tune CUDA kernels, fuse attention/MLP paths, explore quantization formats, and implement parallelism strategies on multi-GPU nodes.
• Build production-grade deployment pipelines for newly released open models, ensuring fast time-to-market.
• Shape the serving stack: TensorRT-LLM, vLLM, SGLang, and diffusion/video pipelines, with upstream contributions where valuable.

Conoscenze

CUDA kernels

GPU architecture

Python

C++

PyTorch

TensorRT-LLM

vLLM

SGLang

Triton

Profiling Nsight

Strumenti

NCCL

Nsight Systems

Nsight Compute

CUDA Graphs

Kubernetes

CI/CD

Descrizione del lavoro

Runware is seeking a Principal Engineer, Model Performance to own how fast and how cheaply every model runs across the fleet. You will set performance roadmaps, write kernel code, and lead the Model Performance team to scale with thousands of GPUs and open AI workflows.

You will collaborate with the Sonic Inference Engine, infrastructure and hardware teams, and external communities to push for lowest latency and cost per token while maintaining quality across 400k+ models.