Staff Engineer — AI Inference & Distributed Systems

Sail; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Sail is hiring for an engineering role in San Francisco to design and implement high-performance schedulers that optimize admission control, queuing, and fairness across a global fleet. You will also build LV routing systems to dispatch workloads with latency awareness and predictive autoscaling, and explore KV caching for memory/compute trade-offs in LLM inference stacks.

You will contribute to deep observability, tracing every millisecond and preemptively addressing failures before customers

Qualifications

• Strong distributed systems fundamentals including concurrency, networking, databases, and performance engineering.

Responsibilities

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).
• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies).
• LLM-specific routing optimizations, e.g. KV caching across GPU RAM, CPU RAM, NVMe flash.

Skills

Distributed systems

Testing & edge cases

Plan & docs

Job description

Sail is hiring for an engineering role in San Francisco to design and implement high-performance schedulers that optimize admission control, queuing, and fairness across a global fleet. You will also build LV routing systems to dispatch workloads with latency awareness and predictive autoscaling, and explore KV caching for memory/compute trade-offs in LLM inference stacks.

You will contribute to deep observability, tracing every millisecond and preemptively addressing failures before customers