Staff Software Engineer, Distributed Systems & AI Inference

Gimlet Labs, Inc.; San Francisco (CA); On-site; USD 150,000 - 390,000; Full time

Benefits offered by this job

Equity

Job summary

Gimlet Labs, Inc. in San Francisco, CA is seeking a Member of Technical Staff to build systems that schedule, route, and coordinate AI workloads across our neocloud infrastructure. You’ll work on scheduling, orchestration, control planes, and APIs to connect the full execution stack.

The role emphasizes reliability, fault tolerance, and scalability across heterogeneous hardware and data centers. Ideal candidates have production Distributed Systems experience and strong CS fundamentals.

Qualifications

• Experience building or operating distributed systems in production.
• Strong software-engineering and systems fundamentals.
• Ability to reason about concurrency, consistency, failure modes, and system tradeoffs.
• Experience with scheduling, resource management, RPC, or asynchronous messaging.

Responsibilities

• Build scheduling and orchestration systems for heterogeneous compute.
• Design systems that manage independently scalable stages of distributed inference pipelines.
• Improve the reliability and fault tolerance of production AI infrastructure.
• Develop control planes and APIs that simplify how workloads are deployed and managed.
• Scale platform across additional hardware, nodes, and data centers.

Skills

Distributed systems

Concurrency

RPC / Async messaging

Go / Python / C++

Education

Bachelor's degree or equivalent practical experience

Tools

Kubernetes

Job description

Gimlet Labs, Inc. in San Francisco, CA is seeking a Member of Technical Staff to build systems that schedule, route, and coordinate AI workloads across our neocloud infrastructure. You’ll work on scheduling, orchestration, control planes, and APIs to connect the full execution stack.

The role emphasizes reliability, fault tolerance, and scalability across heterogeneous hardware and data centers. Ideal candidates have production Distributed Systems experience and strong CS fundamentals.