Inference Systems Engineer: Scalable AI Infrastructure

Candidate; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible PTO

Medical, dental, vision benefits

Life insurance and disability benefits

401(k) / Retirement plan

Parental leave

Fertility and family building benefits

Snacks and coffee

Job summary

Sierra is the leading platform for customer-facing AI agents, based in San Francisco. We are seeking a Software Engineer on the Inference team to define architectures for self-hosted and provider-based inference, focusing on latency, reliability, and cost at scale.

You will work across distributed systems to build, optimize, and operate core inference infrastructure. This role emphasizes ownership of complex systems from design to production, and collaboration with frontier labs, providers, and

Qualifications

• Deep systems thinking and distributed systems fundamentals.
• Experience designing, building, and operating large-scale production systems.
• Strong judgment around latency, reliability, capacity, and cost.
• Experience taking ownership of complex infrastructure from architecture through production operation.
• Excitement about applying systems expertise to AI infrastructure and evolving tech.

Responsibilities

• Partner with frontier labs and providers to supply capacity and infrastructure.
• Shape Sierra’s inference architecture across models, infrastructure, and providers.
• Build for low latency and high reliability in routing, capacity management, and quotas.
• Build and operate self-hosted inference on GPU infrastructure and containers.
• Collaborate with Applied Research on latency and cost optimizations.
• Work across a hybrid inference stack with Sierra-managed and external platforms.
• Push the serving stack forward with inference providers to tune engines.
• Support the broader model lifecycle with infrastructure and cross-team collaboration.

Skills

Deep systems thinking

Distributed systems

Production systems

Latency and reliability tradeoffs

Ownership of infrastructure

AI infrastructure

Tools

vLLM

SGLang

Job description

Sierra is the leading platform for customer-facing AI agents, based in San Francisco. We are seeking a Software Engineer on the Inference team to define architectures for self-hosted and provider-based inference, focusing on latency, reliability, and cost at scale.

You will work across distributed systems to build, optimize, and operate core inference infrastructure. This role emphasizes ownership of complex systems from design to production, and collaboration with frontier labs, providers, and