Site Reliability Engineer, AI Cloud Infrastructure

Anyscale; San Francisco, Northern (CA, KY); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Anyscale in San Francisco is seeking a Site Reliability Engineer to join the Infrastructure team. You will help build the scalable, secure backbone powering distributed AI workloads in the cloud, spanning control plane and data plane.

The role emphasizes Kubernetes, container orchestration, and cloud-native infrastructure, with opportunities to work on Ray integrations and open‑source contributions. You will collaborate across teams to deliver high‑impact infrastructure features.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.
• 3+ years of experience writing high-quality production code.
• Hands-on experience in building and maintaining highly available, scalable distributed systems.
• Expertise in cloud-native technologies (AWS, Azure, GCP) and Kubernetes-based deployments.
• Deep understanding of networking, security, and authentication mechanisms in cloud environments.
• Familiarity with observability stacks (Prometheus, Grafana).
• Proficiency in Go and Python.
• Knowledge of low-level OS foundations (Linux kernel, filesystems, containers).

Responsibilities

• Design, build, and scale services that orchestrate Ray clusters across cloud and on‑prem environments.
• Optimize control plane components for large‑scale distributed AI/ML workloads.
• Build intelligent scheduling and resource management systems for heterogeneous compute clusters.
• Develop features to enhance reliability, performance, scalability, and observability of Ray workloads.
• Support and optimize accelerator integration (e.g., GPUs).
• Handle container image management and dependency resolution for distributed workloads.
• Participate in code reviews, design and architecture discussions.
• Provide on‑call support, collaborating with customer and field teams to troubleshoot infrastructure issues.

Skills

Go

Python

Distributed systems

Observability

Networking & security

On-call experience

Education

Bachelor's degree in CS/Engineering or equivalent

Tools

Kubernetes

AWS

Azure

GCP

Linux

Job description

Anyscale in San Francisco is seeking a Site Reliability Engineer to join the Infrastructure team. You will help build the scalable, secure backbone powering distributed AI workloads in the cloud, spanning control plane and data plane.

The role emphasizes Kubernetes, container orchestration, and cloud-native infrastructure, with opportunities to work on Ray integrations and open‑source contributions. You will collaborate across teams to deliver high‑impact infrastructure features.