AI Platform Support Engineer (US)

Lightning AI; San Francisco (CA); Hybrid; USD 115,000 - 140,000; Full time

Benefits offered by this job

Comprehensive medical, dental and视觉?

Job summary

Lightning AI in San Francisco/Bant? hybrid role seeks an ML Infrastructure Engineer to run ML workloads at scale, enabling training and inference across multi-node GPU systems.

You will work with Kubernetes, cloud infra, and observability tools, diagnosing failures and guiding customers through complex distributed systems challenges. Strong collaboration with ML teams is essential.

Qualifications

• Strong software engineering and systems troubleshooting background.
• Experience with Kubernetes and containerized environments.
• Linux systems knowledge including networking, storage, process management and performance tuning.
• Experience with cloud infrastructure and distributed systems.
• Experience with observability and debugging tools such as Prometheus, Grafana, or OpenTelemetry.
• Hands-on experience operating ML workloads in production or research environments.
• Experience with distributed ML systems and tooling such as PyTorch, CUDA, or NCCL.
• Familiarity with GPU infrastructure and orchestration.
• Experience troubleshooting performance, reliability, or scaling issues in ML infrastructure.
• Understanding of operational challenges running ML systems at scale.
• Strong communication skills and ability to work directly with highly technical customers and engineering teams.
• Comfortable operating in fast moving, highly ambiguous environments.
• Enjoys solving complex technical problems collaboratively.

Responsibilities

• Partner directly with customer engineering teams running training and inference workloads in production.
• Help customers diagnose and resolve complex distributed systems and ML infrastructure issues.
• Act as a technical advisor during high impact incidents and platform degradation events.
• Translate infrastructure level issues into actionable guidance for ML engineers.
• Build credibility with customers through strong technical reasoning and clear communication.
• Investigate failures involving distributed training, Kubernetes orchestration, GPU allocation, networking, and storage systems.
• Troubleshoot PyTorch, CUDA, NCCL, and inference serving related issues.
• Analyze logs, metrics, traces, and system behavior to isolate root causes.
• Debug containerized workloads running across Kubernetes and bare metal GPU environments.
• Support customers scaling workloads across multi node GPU systems.
• Diagnose performance bottlenecks involving compute, memory, networking, or storage.

Skills

Kubernetes

Containerized environments

Linux systems

Cloud infrastructure

Observability tooling

PyTorch

CUDA

NCCL

Distributed ML systems

GPU orchestration

Production ML workloads

Communication with customers

Problem solving

Python scripting

Tools

Prometheus

Grafana

OpenTelemetry

Job description

San Francisco, California, United States; Seattle, Washington, United States

Who We Are

Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with less friction.

Through our merger with Voltage Park, a neocloud and AI Factory, Lightning AI combines developer-first software with cost-efficient, large-scale compute. Teams get the tools they need for experimentation, training, and production inference, with security, observability, and control built in.

We serve solo researchers, startups, and large enterprises. Lightning AI operates globally with offices in New York City, San Francisco, Seattle, and London, and is backed by Coatue, Index Ventures, Bain Capital Ventures, and Firstminute.

What We’re Looking For

We’re looking for engineers who understand the realities of running machine learning workloads at scale.

This role sits at the intersection of ML systems, cloud infrastructure, Kubernetes, and customers. You’ll support engineers training models, deploying inference systems, and scaling GPU workloads in production.

You are not a ticket router or traditional support engineer. You are a technical partner to ML teams - helping diagnose failures, improve reliability, and guide customers through complex distributed systems problems.

The problems range from Kubernetes scheduling and GPU orchestration to distributed PyTorch failures, inference latency, networking bottlenecks, storage performance, and platform reliability.

You’ll gain exposure to a wide variety of real world AI workloads across industries and help shape the infrastructure powering the next generation of ML applications.

What You'll Do

Work Directly With ML Engineers

• Partner directly with customer engineering teams running training and inference workloads in production
• Help customers diagnose and resolve complex distributed systems and ML infrastructure issues
• Act as a technical advisor during high impact incidents and platform degradation events
• Translate infrastructure level issues into actionable guidance for ML engineers
• Build credibility with customers through strong technical reasoning and clear communication
• Investigate failures involving distributed training, Kubernetes orchestration, GPU allocation, networking, and storage systems
• Troubleshoot PyTorch, CUDA, NCCL, and inference serving related issues
• Analyze logs, metrics, traces, and system behavior to isolate root causes
• Debug containerized workloads running across Kubernetes and bare metal GPU environments
• Support customers scaling workloads across multi node GPU systems
• Diagnose performance bottlenecks involving compute, memory, networking, or storage

Improve Reliability & Platform Operations

• Identify recurring patterns across customer issues and drive long term reliability improvements
• Contribute to post incident reviews and operational improvements
• Build internal tooling, automation, documentation, and runbooks
• Partner closely with infrastructure, networking, and platform engineering teams
• Help improve observability, operational visibility, and troubleshooting workflowsImprove the customer experience through better processes and technical guidance

What This Role Is Not

• This is not a traditional help desk or ticket routing support role
• This is not purely customer success or account management
• This is not a backend engineering role
• This is not a passive escalation position

This role is for engineers who enjoy solving difficult technical problems while working closely with other engineers.

What You’ll Need

Required Qualifications

Infrastructure & Systems

• Strong software engineering and systems troubleshooting background
• Experience with Kubernetes and containerized environments
• Linux systems knowledge, including networking, storage, process management, and performance tuning
• Experience with cloud infrastructure and distributed systems
• Experience with observability and debugging tools such as Prometheus, Grafana, or OpenTelemetry

ML Infrastructure Experience

• Hands on experience operating machine learning workloads in production or research environments
• Experience with distributed ML systems and tooling such as PyTorch, CUDA, or NCCL
• Familiarity with GPU infrastructure and orchestration
• Experience troubleshooting performance, reliability, or scaling issues in ML infrastructure
• Understanding of the operational challenges involved in running ML systems at scale
• Strong communication skills and ability to work directly with highly technical customers and engineering teams
• Comfortable operating in fast moving, highly ambiguous environments
• Enjoys solving complex technical problems collaboratively

Nice-to-Haves

• Experience with large scale model training or distributed inference systems
• Familiarity with Ray, Kubeflow, Slurm, or similar distributed scheduling platforms
• Experience with InfiniBand, RDMA, or high-performance networking
• Experience operating bare metal infrastructure
• Familiarity with storage systems commonly used in ML environments
• Experience working at an AI infrastructure, cloud, MLOps, or developer tooling company
• Contributions to platform engineering, developer infrastructure, or operational tooling projects
• Experience writing automation, tooling, or scripts in Python or similar languages

This role is hybrid out of our Seattle or San Francisco offices, with an in-office requirement of at least 2 days per week and occasional team and company offsites. The role follows a Monday–Friday schedule, with working hours from 8:00 AM to 5:00 PM PST. We are not able to provide visa sponsorship for this role at this time.

We are committed to offering competitive compensation that reflects the value each team member brings to our mission. Final offers are based on factors such as experience, skills, geographic location, and role expectations. In addition to base salary, our total rewards package for eligible roles includes a discretionary bonus, a meaningful equity component, and comprehensive benefits.

The anticipated annual base salary range for this role is: $115,000 - $140,000 USD

Benefits and Perks

• Comprehensive medical, dental and vision coverage (U.S.); Private medical and dental insurance (U.K.)
• Retirement and financial wellness support (U.S.); Pension contribution (U.K.)
• Generous paid time off, plus holidays
• Paid parental leave
• Wellness and work-from-home stipends
• Flexible work environment

At Lightning AI, we are committed to fostering an inclusive and diverse workplace. We believe that diverse teams drive innovation and create better products. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected characteristic. We are dedicated to building a culture where everyone can thrive and contribute to their fullest potential.