Remote AI Inference Support Engineer (SRE)

Together; United States; Remote; USD 160,000 - 230,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Remote work flexibility

Job summary

Together AI is seeking a Technical Support Engineer to be the first line of defense for customers building training, fine-tuning, and inference solutions. You will work with GPU clusters and Kubernetes-based endpoints, acting as a product expert and driving improvements within the Customer Experience team.

The role requires 6+ years in customer-facing tech, strong SRE/DevOps background, and deep knowledge of AI tools.

Qualifications

• 6+ years in a customer-facing technical role, SRE, DevOps, or infra engineering, with at least 1 year in an AI service.
• Experience as an SRE or DevOps engineer working with Kubernetes.
• Strong technical background with AI/ML and GPU HPC knowledge.
• Production-level infra and IaC experience (Kubernetes, SLURM, Ansible), and HPC storage familiarity.
• Familiarity with HPC storage like Vast and Weka.
• Diagnose complex network issues and read traces.
• Observability tooling experience (Prometheus, Grafana) at scale.
• REST API debugging and HTTP semantics.
• Experience with LLM inference frameworks and LoRA fine-tuning.
• Infra as Code and Git-based workflows.
• GPU cluster management background.
• Cloud experience (AWS, GCP, Azure).
• Compute cluster installation and admin skills.
• Strong problem solving and proactive approach.
• Cross-functional collaboration with Sales, Engineering, Support, Product, Research.
• Ownership mindset and willingness to learn.
• Excellent communication and interpersonal skills.
• Able to manage multiple projects with prioritization.

Responsibilities

• Engage with customers to resolve complex GPU and inference challenges.
• Maintain healthy, stable inference endpoints on Kubernetes.
• Become a product expert and escalate to Engineering/Product when needed.
• Assist with migrations and monitor dashboards for anomalies.
• Manage customer communications during incidents with clear updates.
• Contribute infra changes via pull requests for endpoint config and scaling.
• Flag engine-level bugs with logs and reproduction steps.
• Collaborate across teams to ensure customer satisfaction.
• Translate insights into roadmap inputs with GTM teams.
• Document configurations, procedures, and troubleshooting guides.
• Provide holiday/weekend coverage as required.

Skills

Problem solving

Communication

Ownership

Cross-functional

AI/ML knowledge

Tools

Kubernetes

Prometheus

Grafana

Ansible

SLURM

REST APIs

AWS

GCP

Azure

Terraform

Job description

Together AI is seeking a Technical Support Engineer to be the first line of defense for customers building training, fine-tuning, and inference solutions. You will work with GPU clusters and Kubernetes-based endpoints, acting as a product expert and driving improvements within the Customer Experience team.

The role requires 6+ years in customer-facing tech, strong SRE/DevOps background, and deep knowledge of AI tools.