Real-Time AI Distillation Engineer

RB Labs; San Francisco (CA); On-site; USD 130,000 - 180,000; Full time

Job summary

RB Labs is building a real-time generative AI system where latency is the core challenge. We’re seeking a model distillation specialist to shrink large models into fast, deployment-ready versions without sacrificing quality.

This is a hands-on role: you’ll design and run distillation experiments, evaluate quality/latency tradeoffs, and ship the results to production GPU infrastructure.

Qualifications

• 2 years hands-on PyTorch experience including training/fine-tuning from scratch.
• Proven experience with model distillation and latency optimization.
• Strong knowledge of model compression techniques: quantization, pruning, knowledge distillation.
• Ability to read and extend dense research code.

Responsibilities

• Design and run knowledge distillation pipelines – teacher/student setups for large models.
• Optimize for latency via distillation, quantization, pruning, and mixed precision.
• Evaluate rigorously with quality-vs-latency comparisons and A/B tests.
• Own training loops – write and debug full PyTorch training code, not just pre-existing scripts.
• Ship distilled models to production GPUs and measure real-world performance.

Skills

PyTorch

Model distillation

Quantization

Pruning

LoRA/PE methods

Tools

PyTorch

Job description

RB Labs is building a real-time generative AI system where latency is the core challenge. We’re seeking a model distillation specialist to shrink large models into fast, deployment-ready versions without sacrificing quality.

This is a hands-on role: you’ll design and run distillation experiments, evaluate quality/latency tradeoffs, and ship the results to production GPU infrastructure.