Member of Technical Staff - Distributed Systems

Sail Research; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Free meals

Studio Display at desk

Friendly office environment with a cat

Job summary

Sail Research in San Francisco is seeking a talented engineer to design and implement robust systems that ensure fast and cost-efficient AI inference at global scale. You will be responsible for building high-performance schedulers and optimizing global routing while focusing on deep observability of our systems.

The ideal candidate has a strong background in distributed systems and is eager to engage in complex challenges. Enjoy a vibrant work environment with excellent meals and a collaborative team atmosphere.

Qualifications

• Strong distributed systems fundamentals in concurrency and networking.
• Eager to work with agents, focusing on correctness and edge cases.
• Bonus: experience with ML inference stacks and GPUs.

Responsibilities

• Design and implement high-performance schedulers.
• Build global routing and traffic management.
• Create deep observability systems for failure tracing.

Skills

Distributed systems fundamentals

Concurrency

Networking

Databases

Performance engineering

Experience with ML inference stacks

Job description

Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market.

Build the systems that make AI inference fast, reliable, and cost-efficient at global scale. You’ll design the control plane that schedules a huge queue of tokens over a diverse fleet of machines, spread all over the world.

What you’ll do

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).
• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies).
• LLM-specific routing optimizations, e.g. KV caching that lets us trade memory for compute, across the pyramid of GPU RAM, CPU RAM, and NVMe flash.
• Build deep observability: we want to trace every millisecond of our systems, and catch failures early enough that we can make things right before the customer even notices.

What we’re looking for

• Strong distributed systems fundamentals (concurrency, networking, databases, performance engineering).
• Eagerness to work with agents. Distributed systems are not easy to one-shot; you’ll always have to think carefully about testing correctness and edge cases. Writing extremely clear plans and tests is a must.
• Bonus: experience with ML inference stacks (vLLM/SGLang), GPUs/accelerators.

Interview process

1. Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately.
2. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear.
3. Come in to Sail’s SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results.
4. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls.

Life at Sail

We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally.