Staff Engineer

Salient Group; San Francisco (CA); Presencial; USD 180 000 - 260 000; Tempo integral

Resumo da oferta

Salient Group is building the infrastructure behind demanding model-serving workloads in San Francisco. You will own the production inference platform end-to-end, scaling Kubernetes, reducing latency, and ensuring reliability as real-time AI workloads evolve.

The role emphasizes diagnosing GPU/network bottlenecks, building robust failover, and enabling faster model launches with cost-efficient compute. A small, ownership-driven team environment awaits.

Qualificações

• Hands-on LLM inference serving experience.
• Kubernetes at production scale experience.
• Experience with high-performance networking (InfiniBand/RDMA) is a plus.
• Strong reliability and observability instincts.

Responsabilidades

• Own the production inference platform end-to-end.
• Scale Kubernetes to meet throughput and latency targets.
• Build failover and self-healing systems.
• Diagnose GPU/network bottlenecks impacting performance.
• Ensure model improvements translate into faster, more reliable production systems.

Conhecimentos

LLM inference serving

Kubernetes at production scale

High-performance networking

Reliability & observability

Ferramentas

Kubernetes

InfiniBand / RDMA

Descrição da oferta de emprego

I'm working with a fast-growing frontier AI company building the infrastructure behind some seriously demanding model-serving workloads. The problem is simple to explain, but hard to solve: How do you make cutting-edge model performance survive real production traffic?

That means handling huge inference workloads across GPU fleets, keeping latency low, scaling reliably across cloud environments, and debugging the ugly problems that appear somewhere between Kubernetes, CUDA, NCCL, networking and the serving layer.

You'll own the production inference platform end-to-end, scaling Kubernetes, improving throughput and latency, building failover and self-healing systems, diagnosing GPU/network bottlenecks, and making sure model improvements actually translate into faster, more reliable production systems.

The ideal background looks something like:

• Hands-on LLM inference serving
• Kubernetes at meaningful production scale
• InfiniBand / RDMA / high-performance networking
• Strong reliability and observability instincts

The impact is big: your work directly affects how quickly new models can launch, how efficiently compute is used, and how far the company can push real-time AI products.

Small technical team. High ownership. Serious systems problems.

If you've spent the last couple of years making inference faster, cheaper and more reliable I'd love to speak.