Software Engineer, Site Reliability

fal - Features & Labels; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

fal is building the generative media infrastructure and is seeking a seasoned SRE to keep production systems running at scale. You will own Kubernetes clusters, CI/CD pipelines, and the networking layer that ties it all together.

You will implement SLOs, runbooks, and incident response, automate fixes, and craft dashboards with Prometheus/Grafana. Expertise in Terraform/Ansible, GitOps, and metrics tooling will help you move fast while maintaining reliability.

Qualifications

• 5+ years experience in managing critical production systems and software development workflows.
• Strong production experience setting up and operating Kubernetes at scale, using infrastructure-as-code (Terraform, Ansible).
• Deep knowledge of Linux networking, container networking (CNI plugins, VXLAN, BGP), and DNS.
• Experience building CI/CD systems and GitOps workflows (FluxCD, ArgoCD).
• Proficiency in Python and either Go or Bash for tooling and automation.
• Strong experience with logging, monitoring and alerting (Prometheus, Grafana, Loki, Thanos, VictoriaMetrics, Datadog).
• Excellent communication and ability to drive technical decisions across teams.
• Self-starter who executes quickly, takes ownership, and constantly seeks improvement.
• Experience with managing GPU and AI/ML workloads
• Experience with kernel-based monitoring and routing (eBPF, XDP)
• Experience with security tooling (Falco, Coroot, SIEM)
• Experience with bare metal Kubernetes networking (Calico, Cilium, MetalLB)
• Experience with distributed storage systems (Ceph, Longhorn, etc.)

Responsibilities

• Own and operate our Kubernetes infrastructure: cluster lifecycle, upgrades, networking, and multi-tenant isolation for customer workloads
• Build and maintain CI/CD pipelines and deployment infrastructure
• Leverage AI to an extreme level to automate analysis and resolution of production issues, and improve software development speed, reliability and maintainability
• Build dashboards, alerting, and anomaly detection across our systems
• Define and enforce SLOs and build out incident response processes
• Manage and improve our networking, load balancing, and service mesh configurations
• Drive reliability improvements across the stack through automation, runbooks, and chaos engineering

Skills

Kubernetes at scale

SRE / Production systems

Automation and tooling

Python

Go or Bash

CI/CD / GitOps

Linux networking

Monitoring and logging

Communication

Ownership / initiative

Tools

Terraform

Ansible

FluxCD

ArgoCD

Prometheus

Grafana

Loki

Thanos

Datadog

eBPF

XDP

Calico

Cilium

MetalLB

Ceph

Longhorn

Job description

fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromise. For developers and enterprises, fal is the foundation that makes generative media not just possible, but practical: a unified platform where high-performance inference, orchestration, and observability come together to unlock new categories of AI-native products.

As generative media reshapes industries across a market projected to grow by hundreds of billions over the next decade, fal is becoming the ecosystem that ambitious teams build on.

You are a seasoned SRE who keeps production infrastructure running at scale. You own the reliability and availability of customer-facing systems — from Kubernetes clusters to deployment pipelines to the networking layer that connects it all. You think in SLOs, automate ruthlessly, and treat every incident as a chance to make the system better.

What you'll do:

• Own and operate our Kubernetes infrastructure: cluster lifecycle, upgrades, networking, and multi-tenant isolation for customer workloads

• Build and maintain CI/CD pipelines and deployment infrastructure

• Leverage AI to an extreme level to automate analysis and resolution of production issues, and improve software development speed, reliability and maintainability

• Build dashboards, alerting, and anomaly detection across our systems

• Define and enforce SLOs and build out incident response processes

• Manage and improve our networking, load balancing, and service mesh configurations

• Drive reliability improvements across the stack through automation, runbooks, and chaos engineering

Qualifications/Nice to have:

• 5+ years experience in managing critical production systems and software development workflows

• Strong production experience setting up and operating Kubernetes at scale, using infrastructure-as-code (Terraform, Ansible)

• Deep knowledge of Linux networking, container networking (CNI plugins, VXLAN, BGP), and DNS

• Experience building CI/CD systems and GitOps workflows (FluxCD, ArgoCD)

• Proficiency in Python and either Go or Bash for tooling and automation

• Strong experience with logging, monitoring and alerting (Prometheus, Grafana, Loki, Thanos, VictoriaMetrics, Datadog)

• Excellent communication and ability to drive technical decisions across teams

• Self-starter who executes quickly, takes ownership, and constantly seeks improvement

• Experience with managing GPU and AI/ML workloads

• Experience with kernel-based monitoring and routing (eBPF, XDP)

• Experience with security tooling (Falco, Coroot, SIEM)

• Experience with bare metal Kubernetes networking (Calico, Cilium, MetalLB)

• Experience with distributed storage systems (Ceph, Longhorn, etc.)

What we offer at fal

• Interesting and challenging work

• A lot of learning and growth opportunities

• Regular team events and offsites

U.S. EQUAL EMPLOYMENT OPPORTUNITY INFORMATION:

fal provides equal employment opportunities to applicants and employees without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability, or any other classification protected by applicable law.