Lead Cloud Infrastructure Engineer - GPU AI Kubernetes

FriendliAI; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Flexible working hours

Lunch and dinner provided

Health check-up support with top-tier硬

Top-tier equipment/hardware support

Job summary

FriendliAI is building the fastest inference cloud for agents, delivering low-latency, scalable GPU-accelerated AI workloads. We seek a Cloud Infrastructure Engineer to own cluster architecture, extend Kubernetes, and manage the network path for inference traffic.

The role demands hands-on experience with large multi-cluster deployments, designing topology, upgrades, and capacity across tenants. You will shape autoscaling and service mesh, collaborating across SRE, security, and engine teams to

Qualifications

• Extensive experience designing and operating large-scale Kubernetes infra.
• Strong programming skills in Go or Python for tooling.
• Proven track record debugging distributed systems and networks.

Responsibilities

• Own architecture of multi-cluster, multi-tenant Kubernetes fleet.
• Extend Kubernetes with custom controllers, operators, and CRDs.
• Design GPU scheduling, topology-aware placement, and quotas across tenants.
• Build autoscaling for inference traffic, including queue-driven pod scaling and scale-to-zero.
• Own the Kubernetes network data plane: CNI, IPAM, DNS, ingress, and L4/L7 load balancing.
• Design cross-AZ/region connectivity and operate service mesh (mTLS, routing).
• Define SLOs and lead post-incident hardening; deliver IaC with Terraform, Helm, GitOps.
• Collaborate with inference, platform, SRE, and security teams to turn requirements into platform capabilities.

Skills

Kubernetes

Go or Python

Distributed systems debugging

Communication

Education

Bachelors or Masters degree in CS/CE/EE

Tools

AWS

Terraform

Helm

Ansible

Job description

FriendliAI is building the fastest inference cloud for agents, delivering low-latency, scalable GPU-accelerated AI workloads. We seek a Cloud Infrastructure Engineer to own cluster architecture, extend Kubernetes, and manage the network path for inference traffic.

The role demands hands-on experience with large multi-cluster deployments, designing topology, upgrades, and capacity across tenants. You will shape autoscaling and service mesh, collaborating across SRE, security, and engine teams to