Fleet SRE: Command & Control for Frontier Inference

etched; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

Etched is seeking a Site Reliability Engineer to help build the command-and-control layer for a fleet of frontier inference systems. You will define health signals, implement alerting standards, and automate recovery workflows, while coordinating with Supercomputing and Infrastructure for telemetry and rollout.

As one of the first SREs, you will influence incident management, on-call practices, and tooling to keep clusters serving with minimal downtime.

Qualifications

• 2+ years in site reliability, production engineering or similar.
• Experience designing alerting, SLOs and incident management.
• Familiarity with tracing and logs across distributed systems.

Responsibilities

• Define what a healthy inference system looks like with signals across racks, nodes and chips.
• Set a common alerting standard with team-specific SLOs.
• Cut alert noise by correlating alerts and retiring unused pages.
• Automate recovery with runbooks, automated restarts and traffic routing.
• Route events to the right engineer, with context and on-site support for hardware tasks.
• Shape fleet data with telemetry pipelines and OpenTelemetry standards.
• Run incident management with clear severity levels, channels, postmortems and handoffs.
• Improve service reliability through design reviews and canaries.

Skills

Site reliability

Production engineering

Incident management

OpenTelemetry

Troubleshooting

Tools

OpenTelemetry

Monitoring systems

Telemetry tooling

Job description

Etched is seeking a Site Reliability Engineer to help build the command-and-control layer for a fleet of frontier inference systems. You will define health signals, implement alerting standards, and automate recovery workflows, while coordinating with Supercomputing and Infrastructure for telemetry and rollout.

As one of the first SREs, you will influence incident management, on-call practices, and tooling to keep clusters serving with minimal downtime.