Ai Distilation Expert

RB Labs; San Francisco (CA); On-site; USD 130,000 - 180,000; Full time

Job summary

RB Labs is building a real-time generative AI system where latency is the core challenge. We’re seeking a model distillation specialist to shrink large models into fast, deployment-ready versions without sacrificing quality.

This is a hands-on role: you’ll design and run distillation experiments, evaluate quality/latency tradeoffs, and ship the results to production GPU infrastructure.

Qualifications

• 2 years hands-on PyTorch experience including training/fine-tuning from scratch.
• Proven experience with model distillation and latency optimization.
• Strong knowledge of model compression techniques: quantization, pruning, knowledge distillation.
• Ability to read and extend dense research code.

Responsibilities

• Design and run knowledge distillation pipelines – teacher/student setups for large models.
• Optimize for latency via distillation, quantization, pruning, and mixed precision.
• Evaluate rigorously with quality-vs-latency comparisons and A/B tests.
• Own training loops – write and debug full PyTorch training code, not just pre-existing scripts.
• Ship distilled models to production GPUs and measure real-world performance.

Skills

PyTorch

Model distillation

Quantization

Pruning

LoRA/PE methods

Tools

PyTorch

Job description

About the Role

We’re building a real-time generative AI system where latency is the core challenge. We’re looking for a model distillation specialist to help us shrink large models into fast, deployment-ready versions without sacrificing quality.

This is a hands-on role: you’ll design and run distillation experiments, evaluate quality/latency tradeoffs, and ship the results to production GPU infrastructure.

What You’ll Do

• Design and run knowledge distillation pipelines – teacher/student setups for large neural models (LLMs and/or generative vision models)
• Optimize for latency – reduce inference cost via distillation, quantization, pruning, and mixed precision
• Evaluate rigorously – build empirical quality-vs-latency comparisons and A/B test variants
• Own training loops – write and debug full PyTorch training code, not just run existing scripts
• Ship to production – deploy distilled models on live GPU servers and measure real-world performance

Must Have

• 2 years hands-on PyTorch, including training/fine-tuning models from scratch – not just serving pretrained ones
• Proven experience with model distillation – you’ve actually distilled a model and measured the results
• Strong grasp of model compression techniques: quantization, pruning, knowledge distillation, LoRA/parameter-efficient methods
• Solid understanding of loss design for distillation (KL divergence, feature/logit matching, perceptual losses)
• Comfortable reading and extending dense, lightly-documented research code

Strong Plus

• Experience distilling autoregressive LLMs or speech models
• Experience with GANs, diffusion, or flow-matching models
• GPU inference optimization: torch.compile, CUDA memory management, multi-GPU setups
• Familiarity with streaming/real-time ML systems

Working Style

We operate like a research lab: empirical, fast iteration, incomplete docs, self-directed testing. You’ll need to be comfortable designing your own experiments and pushing for more engineering structure as you go.