Inference Platform Engineer: Scale, Latency, Reliability

ARCHER; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Flexible Paid Time Off

Medical Insurance

Dental Insurance

Vision Insurance

Life Insurance

Disability Benefits

Retirement Plan

Parental Leave

Fertility And Family-Building Benefits

Lunch

Snacks And Coffee

Discretionary Benefit Stipend

Equity Plans

Alphorn Lessons

Job summary

Sierra is seeking an experienced infrastructure engineer to design and operate its inference architecture across self-hosted models and third-party providers. You will run and optimize production inference infrastructure focusing on latency, reliability, throughput, and cost, while collaborating with research, model, runtime, and provider teams.

Candidates should have strong distributed-systems fundamentals and experience building and operating large-scale production infrastructure; experience

Qualifications

• Strong distributed-systems fundamentals and experience in large-scale production infra.
• Experience with ML infra, LLM serving, GPU operations is a plus.
• Ability to make tradeoffs and own complex systems end-to-end.

Responsibilities

• Design and operate Sierra's inference architecture across self-hosted models and third-party providers.
• Run and optimize production inference infrastructure for latency, reliability, throughput and cost.
• Collaborate with research, model, runtime and provider teams.

Skills

Distributed Systems

Inference Architecture

Inference Routing

Large-Scale Production Systems

Reliability Engineering

Latency Optimization

Cost Optimization

GPU Infrastructure

Self-Hosted Inference

LLM Serving

Inference Frameworks

vLLM

SGLang

Post-Training Infrastructure

Job description

Sierra is seeking an experienced infrastructure engineer to design and operate its inference architecture across self-hosted models and third-party providers. You will run and optimize production inference infrastructure focusing on latency, reliability, throughput, and cost, while collaborating with research, model, runtime, and provider teams.

Candidates should have strong distributed-systems fundamentals and experience building and operating large-scale production infrastructure; experience