Senior AI Inference Systems Engineer (Equity)

Emploive; San Francisco (CA); On-site; USD 230,000 - 390,000; Full time

Benefits offered by this job

Flexible PTO

Medical, dental, vision benefits

Retirement plan

Parental leave

Equity plans

Free snacks and beverages

Job summary

Sierra is the leading platform for customer-facing AI agents, based in San Francisco with offices across North America, Europe, and Asia. We are seeking a Software Engineer on the Inference team to shape architecture for self-hosted and third-party inference, focusing on low latency, reliability, and cost.

You'll partner with frontier labs and providers, build scalable serving layers, and operate GPU-based inference infrastructure in a systems-first role at the intersection of distributed infra

Qualifications

• Deep systems thinking and distributed systems fundamentals.
• Experience designing, building, and operating large-scale production systems.
• Strong judgment around latency, reliability, capacity, and cost tradeoffs.
• Experience owning complex infrastructure from architecture through production.

Responsibilities

• Partner with frontier labs and providers to supply capacity and infrastructure.
• Shape inference architecture and traffic flow across models and providers.
• Build for low latency, high reliability with routing and quota management.
• Operate self-hosted inference on GPU infrastructure and containers.
• Collaborate with Applied Research for latency and cost optimizations.
• Work across hybrid inference stack and Sierra-managed infra.
• Push serving stack forward with providers to tune engines.
• Support broader model lifecycle with Model and Agent Runtime teams.

Skills

Deep systems thinking

Distributed systems

Production systems ownership

Latency/reliability tradeoffs

Infra capacity management

Tools

vLLM

SGLang

Job description

Sierra is the leading platform for customer-facing AI agents, based in San Francisco with offices across North America, Europe, and Asia. We are seeking a Software Engineer on the Inference team to shape architecture for self-hosted and third-party inference, focusing on low latency, reliability, and cost.

You'll partner with frontier labs and providers, build scalable serving layers, and operate GPU-based inference infrastructure in a systems-first role at the intersection of distributed infra