Edge AI Inference Engineer: Model Optimization & Deployment

Zoox; Foster City (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Health insurance

Stock options

RSUs

Paid time off

Job summary

Zoox in Foster City is seeking a Model Optimization & Deployment Engineer to bring production-ready, large-scale models to our on-vehicle stack. You will design low-latency C++/CUDA code for real-time perception on edge devices and optimize models with quantization and mixed-precision inference.

You will build TensorRT pipelines for edge deployment, perform parity checks against PyTorch, and develop custom ML ops and TensorRT plugins to maximize memory bandwidth and minimize latency on AI

Qualifications

• Production-level C++ and Python for real-time edge inference.
• Deep expertise in model quantization and mixed-precision inference.
• Experience with TensorRT/edge deployment and latency benchmarks.

Responsibilities

• Design and develop low-latency C++/CUDA code for real-time perception.
• Optimize large-scale models using PTQ, QAT, and mixed precision.
• Architect model conversion pipelines with TensorRT for edge deployment.
• Benchmark latency and parity between PyTorch and edge binaries.
• Develop custom ML ops and TensorRT plugins for GPUs.

Skills

C++

Python

CUDA

Model compression

PTQ

QAT

TensorRT

Efficient Attention

KV-cache optimization

ONNX

Tools

TensorRT

ONNX

Torch.compile

Job description

Zoox in Foster City is seeking a Model Optimization & Deployment Engineer to bring production-ready, large-scale models to our on-vehicle stack. You will design low-latency C++/CUDA code for real-time perception on edge devices and optimize models with quantization and mixed-precision inference.

You will build TensorRT pipelines for edge deployment, perform parity checks against PyTorch, and develop custom ML ops and TensorRT plugins to maximize memory bandwidth and minimize latency on AI