Inference Optimization Engineer: Fast, Cost-Effective ML

Build AI; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Competitive pay

Medical, dental, and vision packages

Housing subsidy $2k/month near SF offi

Relocation support SF/Shenzhen

Wellness benefits

Daily lunch and dinner in office

Unlimited compute budget

Codex and Claude credits

Travel

Job summary

Build AI, an in-person team in San Francisco, seeks a skilled ML/systems engineer to optimize inference performance, reduce latency and cost, and enable scaling of the data engine. You will work closely with research and product to ensure affordable, accurate models and efficient serving.

Ideal candidates think in dollars and tokens per second, are proficient in Python and C++/Rust, and are comfortable profiling and tuning GPUs/accelerators in a small, cost–constrained team.

Qualifications

• Strong ML / systems engineer with real inference optimization experience (serving, compilers, CUDA/kernels, quantization, or similar).
• Comfortable in Python and in C++ or Rust for performance‑critical paths.
• You think in dollars and tokens/frames per second, not only in accuracy tables.
• Familiar with PyTorch (or JAX) and profiling tools.
• Able to work in a small research team shipping under cost pressure.

Responsibilities

• Own inference performance: latency, throughput, and cost per unit of work (tokens, frames, or jobs).
• Cut the 90% compute line: kernels, batching, quantization, compilation, serving, and hardware utilization.
• Profile pipelines (Nsight, PyTorch Profiler, or equivalent), find the bottleneck, and ship the fix.
• Collaborate with research and product so models are affordable to run at scale.
• Build serving and eval path so experiments don’t inflate the inference bill.
• Measure cost as a first‑class metric, not an afterthought when quality is done.

Skills

ML systems

Inference optimization

Python

C++/Rust

PyTorch

Profiling

Cost-aware thinking

Tools

Nsight

PyTorch Profiler

CUDA

Kernels

Job description

Build AI, an in-person team in San Francisco, seeks a skilled ML/systems engineer to optimize inference performance, reduce latency and cost, and enable scaling of the data engine. You will work closely with research and product to ensure affordable, accurate models and efficient serving.

Ideal candidates think in dollars and tokens per second, are proficient in Python and C++/Rust, and are comfortable profiling and tuning GPUs/accelerators in a small, cost–constrained team.