Software Engineer, Inference

ARCHER; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Flexible Paid Time Off

Medical Insurance

Dental Insurance

Vision Insurance

Life Insurance

Disability Benefits

Retirement Plan

Parental Leave

Fertility And Family-Building Benefits

Lunch

Snacks And Coffee

Discretionary Benefit Stipend

Equity Plans

Alphorn Lessons

Job summary

Sierra is seeking an experienced infrastructure engineer to design and operate its inference architecture across self-hosted models and third-party providers. You will run and optimize production inference infrastructure focusing on latency, reliability, throughput, and cost, while collaborating with research, model, runtime, and provider teams.

Candidates should have strong distributed-systems fundamentals and experience building and operating large-scale production infrastructure; experience

Qualifications

• Strong distributed-systems fundamentals and experience in large-scale production infra.
• Experience with ML infra, LLM serving, GPU operations is a plus.
• Ability to make tradeoffs and own complex systems end-to-end.

Responsibilities

• Design and operate Sierra's inference architecture across self-hosted models and third-party providers.
• Run and optimize production inference infrastructure for latency, reliability, throughput and cost.
• Collaborate with research, model, runtime and provider teams.

Skills

Distributed Systems

Inference Architecture

Inference Routing

Large-Scale Production Systems

Reliability Engineering

Latency Optimization

Cost Optimization

GPU Infrastructure

Self-Hosted Inference

LLM Serving

Inference Frameworks

vLLM

SGLang

Post-Training Infrastructure

Job description

Design and operate Sierra's inference architecture across self-hosted models and third-party providers, including serving, routing, failover, capacity, and quota systems. Run and optimize production inference infrastructure for latency, reliability, throughput, and cost, while collaborating with research, model, runtime, and provider teams.

Requirements

Candidates should have strong distributed-systems fundamentals and experience designing, building, and operating large-scale production infrastructure. The role also calls for sound tradeoff judgment and ownership of complex systems; experience with ML infrastructure, LLM serving, GPU operations, vLLM or SGLang, post-training, or inference optimization is advantageous.

Key Skills

• Distributed Systems
• Large-Scale Production Systems
• Inference Architecture
• Inference Routing
• Capacity Management
• Quota Management
• Reliability Engineering
• Latency Optimization
• Cost Optimization
• GPU Infrastructure
• Self-Hosted Inference
• LLM Serving
• Inference Frameworks
• vLLM
• SGLang
• Post-Training Infrastructure

Benefits

• Flexible Paid Time Off
• Medical Insurance
• Dental Insurance
• Vision Insurance
• Life Insurance
• Disability Benefits
• Retirement Plan
• Parental Leave
• Fertility And Family-Building Benefits
• Lunch
• Snacks And Coffee
• Discretionary Benefit Stipend
• Equity Plans
• Alphorn Lessons