Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Together AI, the AI Native Cloud, is seeking a Staff AI Inference Reliability Engineer to own production reliability for GPU inference fleets and the Kubernetes-native control plane. You will triage failures, manage SLOs, and drive automated remediation across a distributed, multi-node environment.
You will work with ML researchers, engineers, and product managers to ship reliable tooling, improve latency and throughput, and reduce operational toil through automation.
We're looking for a software engineer, reliability-minded but still a builder, to own what happens in production to the Kubernetes-native control plane that provisions and runs our GPU inference fleet. The platform team builds the manifest-driven API and controllers that let the inference team declare what they need, a cluster, a model deployment, a capacity change, and get it reconciled underneath. Your job is to keep that promise true in the real world: triaging InfiniBand issues when prefill and decode nodes stall exchanging KV cache over RDMA, diagnosing hero-model and LoRA-serving faults that don't show up in generic health checks, and extending the automated self-healing system so each new failure mode becomes a permanent fix instead of a recurring page.
The core value we're after is keeping the decoupling real even when things break, and keeping the fleet efficient without that efficiency becoming a new source of instability. You'll own SLOs and error budgets for inference serving, lead incident response, and turn what postmortems teach you into actual changes in the controllers and automation, not just updated runbooks. You'll also work the reliability side of capacity efficiency, since a mistuned defragmentation or bin-packing pass is itself a production risk. Strong candidates have hands‑on production‑reliability experience on large-scale distributed or GPU-serving systems, are comfortable troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and think about GPU infrastructure, and the tools that keep it reliable, as software to be engineered.
A product mindset applies: the on‑call experience, the clarity of an alert, and the judgment built into an automated remediation are things you ship and iterate on.
Together AI, the AI Native Cloud, is purpose‑built for AI engineers. AI application developers get high‑performance inference that scales reliably, fine‑tuning and reinforcement learning for creating frontier‑level specialized models, and pre‑training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full‑time position is: $250,000 - $320,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job‑related knowledge.
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
As set forth in Together AI’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.
If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:
A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.
A "recently separated veteran" means any veteran during the three‑year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.
An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.
An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.