Inference Systems Engineer: Low-Latency AI Infra

Sierra; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Sierra is building a leading platform for customer-facing AI agents, based in San Francisco with offices in North America, Europe, and Asia. We are seeking a Software Engineer on Inference to define Sierra’s inference architecture for self-hosted models and third-party providers, focusing on fast, reliable, and cost-efficient systems.

This role is a systems-first position intersecting distributed infrastructure and AI, requiring passion for complex systems and a willingness to work across hybrid

Qualifications

• Experience designing, building, and operating large-scale production systems.
• Strong judgment on tradeoffs for latency, reliability, cost.
• Ownership of complex infrastructure from architecture to production.
• Interest in applying systems expertise to AI infra and evolving tech.
• Experience with distributed systems fundamentals.

Responsibilities

• Partner with frontier labs and providers to supply capacity.
• Shape inference architecture across models, infra, and providers.
• Build low-latency, high-reliability systems for routing and quota.
• Operate self-hosted inference on GPU infrastructure.
• Optimize performance with decoding and serving-engine tweaks.
• Work across hybrid inference stack with Sierra and external platforms.
• Push the serving stack forward with providers.
• Support model lifecycle with post-training infrastructure.

Skills

Distributed systems

Production systems

Latency & reliability

Ownership

AI infrastructure

Tools

MLOps

GPU infrastructure

Containerization

Inference systems

Job description

Sierra is building a leading platform for customer-facing AI agents, based in San Francisco with offices in North America, Europe, and Asia. We are seeking a Software Engineer on Inference to define Sierra’s inference architecture for self-hosted models and third-party providers, focusing on fast, reliable, and cost-efficient systems.

This role is a systems-first position intersecting distributed infrastructure and AI, requiring passion for complex systems and a willingness to work across hybrid