Member of Technical Staff - Distributed Systems

Sail; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Sail is hiring for an engineering role in San Francisco to design and implement high-performance schedulers that optimize admission control, queuing, and fairness across a global fleet. You will also build LV routing systems to dispatch workloads with latency awareness and predictive autoscaling, and explore KV caching for memory/compute trade-offs in LLM inference stacks.

You will contribute to deep observability, tracing every millisecond and preemptively addressing failures before customers

Qualifications

• Strong distributed systems fundamentals including concurrency, networking, databases, and performance engineering.

Responsibilities

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).
• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies).
• LLM-specific routing optimizations, e.g. KV caching across GPU RAM, CPU RAM, NVMe flash.

Skills

Distributed systems

Testing & edge cases

Plan & docs

Job description

Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market.

Build the systems that make AI inference fast, reliable, and cost-efficient at global scale. You’ll design the control plane that schedules a huge queue of tokens over a diverse fleet of machines, spread all over the world.

What you’ll do

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).
• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies).
• LLM-specific routing optimizations, e.g. KV caching that lets us trade memory for compute, across the pyramid of GPU RAM, CPU RAM, and NVMe flash.
• Build deep observability: we want to trace every millisecond of our systems, and catch failures early enough that we can make things right before the customer even notices.

What we’re looking for

• Strong distributed systems fundamentals (concurrency, networking, databases, performance engineering).
• Eagerness to work with agents. Distributed systems are not easy to one-shot; you’ll always have to think carefully about testing correctness and edge cases. Writing extremely clear plans and tests is a must.
• Bonus: experience with ML inference stacks (vLLM/SGLang), GPUs/accelerators.

Interview process

1. Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately.
2. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear.
3. Come in to Sail's SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results.
4. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls.

Life at Sail

We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally.