Infrastructure Engineer - Labs

Etched; San Jose (CA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

Housing subsidy of $2.5k/month

Relocation support to San Jose

Daily lunch in office

Wellness benefits

Unlimited compute budget

Job summary

Etched.ai is building hardware for frontier intelligence and is expanding its Infra Labs to manage rack-scale infrastructure across labs and data centers. You will own server infrastructure, provisioning, networking, and observability across multiple sites, ensuring systems are healthy, discoverable, and scalable.

You will work hands-on with bare-metal racks, collaborate with Lab Operations and Deployment teams, and drive automation using Ansible and Python to support next-generation hardware

Qualifications

• 4+ years of hands-on infrastructure, data center systems, or bare-metal operations experience in on-prem environments.
• Experience across servers, power, Linux, networking, and hardware; able to troubleshoot multiple layers.

Responsibilities

• Own racks and server infrastructure across labs and data center sites, from bring-up to testing.
• Provide full-stack infrastructure for new lab and data center sites: compute, networking, storage, DHCP/DNS, and provisioning services.
• Build and maintain bare-metal provisioning for servers, development kits, and switches using PXE/iPXE, DHCP, and Redfish-based imaging.
• Troubleshoot Linux hosts end-to-end on custom hardware: kernel, PCIe, NIC, storage, boot issues; root-cause analysis.
• Bring racks onto the network: top-of-rack configs, VLAN/IP planning, optics and cabling with network engineers.
• Diagnose hardware faults and coordinate repairs and RMAs with lab staff and vendors.
• Run shared lab compute: manage Slurm cluster for engineers; expand storage/compute as needed.
• Extend observability stack to all racks with dashboards and alerts for health, power, and link status.
• Write infrastructure automation in Ansible and Python; contribute via PRs and tests.
• Manage software releases and changes on lab systems; maintain runbooks and inventory.

Skills

Infrastructure

Linux systems

Networking

Automation (Ansible, Python)

Bare-metal provisioning

SRE/ops mindset

Observability (Prometheus, Grafana)

Cluster management (Slurm)

Tools

Prometheus

VictoriaMetrics

Grafana

Ansible

Python

PXE/iPXE

Redfish/OpenBMC

Job description

About Etched

Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on inference. Backed by hundreds of millions from top-tier investors and staffed by leading engineers, Etched is redefining the infrastructure layer for the fastest growing industry in history.

Job Summary

Every Etched system starts its life in a lab. Before a rack-scale inference product reaches a customer, it goes through bring-up, validation, and testing in our factories, labs, or data centers, and each of those environments needs networked, provisioned, monitored infrastructure that is ready before the hardware arrives. The Infra Labs team builds and runs that infrastructure.

As an Infrastructure Engineer on Infra Labs, you will own the racks and server infrastructure across Etched's labs and data center sites, from our office lab at Santana Row to our new hardware facility in Milpitas and our Bay Area colocation footprint. Once our forward‑deployed engineers physically install a system, you take it from there: getting it onto the network, provisioned, connected to the storage and compute services it depends on, observable, and healthy through bring-up and testing. You'll support our current rack‑scale systems, stand up labs for our next‑generation racks and next‑generation silicon, and help build the internal deployments that let us run our systems the way our customers do.

This is not a cloud role, and it is not a production SRE role. It is hands‑on, full‑stack infrastructure delivery in environments that change every week. One week you might rebuild a network‑boot provisioning flow for a new server type, chase a NIC firmware or PCIe issue on a liquid‑cooled development kit, and rack and cable a new lab row. You'll work alongside Lab Operations, Supercomputing, and Deployment teams to figure out what needs to be built, and then build it with scale in mind from day one: we are designing for gigawatt‑scale inference deployments, not a handful of racks.

Key Responsibilities

• Own the management and support of racks and server infrastructure across Etched's labs and data center sites, from handoff by our forward‑deployed engineers through bring‑up, validation, and testing.

• Deliver full‑stack infrastructure for new lab and data center sites: compute, networking, storage, and the core services (DHCP, DNS, zero‑touch provisioning, NFS) that need to be in place before hardware arrives.

• Build and maintain bare‑metal provisioning for servers, development kits, rack management controllers, and switches, using PXE/iPXE, DHCP, BMC/Redfish‑driven imaging, and golden images, so that a newly racked system comes up with no hands on the keyboard.

• Troubleshoot Linux hosts end‑to‑end on custom and early‑stage hardware: kernel, PCIe, NIC driver and firmware, bonding, storage, boot, and power issues. Drive each one to the root cause and write it up.

• Bring racks onto the network: top‑of‑rack and uplink configuration, VLAN and IP planning, optics and cabling, in partnership with our network engineers.

• Diagnose physical‑layer and hardware faults such as link flaps, CRC errors, dirty or failed optics, power faults, and coolant leaks, and coordinate repairs and RMAs with lab staff and vendors.

• Run shared lab compute: help our Infrastructure Team members manage the Slurm cluster that lets engineers reserve accelerator systems, including partitions, reservations, and maintenance drains, and expand storage and compute as programs grow.

• Extend our observability stack (Prometheus, VictoriaMetrics, Grafana) to every lab rack, with dashboards and alerts for health, power, and link status across sites.

• Write infrastructure automation in Ansible and Python through reviewed, tested pull requests, and build tooling that serves both tightly controlled environments and the varied needs of individual lab teams.

• Manage software releases and changes on lab systems under change control, respond to incidents, and keep runbooks, inventory, and design records current as you work.

Representative Projects

• Stand up the full infrastructure for our Milpitas hardware facility as it comes online for next‑generation rack validation: network, provisioning, storage, telemetry, and access, ready before the first engineering cabinets are built.

• Build a portable, containerized provisioning stack (DHCP, PXE, HTTP, ZTP) that can be dropped into a new lab or rack to bring up servers and switches straight off the network.

• Plan and build lab infrastructure for the bring‑up of our next‑generation silicon, so that systems are reachable and imaged the day engineering samples arrive.

• Take a new colocation cage from empty to live: coordinate cross‑connects and circuits, configure top‑of‑rack switches and uplinks, and hand racks over ready for internal workloads.

• Build a link‑and hardware‑health pipeline that spots rising bit error rates, CRC errors, and flapping links, and points to the right fix: clean an optic, swap a cable, or replace a NIC.

• Design an asset and inventory workflow that ties serial numbers, MAC and IP addresses, and rack locations together so every repair and RMA is traceable.

• Build high‑throughput NVMe and NFS storage for loading model weights across lab racks, with documented layouts and benchmarks.

• Move lab telemetry and firmware workflows onto standard Redfish and OpenBMC interfaces as our hardware platforms adopt them.

You may be a good fit if you

• Have 4+ years of hands‑on infrastructure, data center systems, or bare‑metal operations experience in on‑prem environments where you physically touched the hardware.
• Are comfortable across every layer of a rack: servers, power circuits and PDUs, Linux, networking, and the hardware itself, and can troubleshoot whichever one is broken.
• Have strong Linux systems skills on bare metal, including kernel and driver debugging, PCIe and NIC troubleshooting, storage, and boot issues.
• Have provisioned bare‑metal fleets over the network with PXE/iPXE, DHCP, and DNS, and used BMCs (Redfish or IPMI) for out‑of‑band management.
• Write infrastructure automation in Ansible (YAML, Jinja2) and Python, and work in Git with pull requests, code review, and tests.
• Have working networking knowledge: L2/L3 fundamentals, VLANs, LACP, switch configuration, and optics and copper cabling at 100G and above.
• Have built or operated observability stacks such as Prometheus, Grafana, or VictoriaMetrics.
• Have handled incident response and rolled software and firmware updates across a fleet without breaking it.
• Are physically hands‑on: you rack and unrack heavy servers with a lift, run cabling, and are at home in an active lab.
• Are genuinely excited about hardware, including early, experimental hardware that doesn't come with a vendor playbook.
• Work well with electrical, mechanical, facilities, and software engineers, and write things down so the next person can repeat your work.
• Are able to work on‑site in San Jose and travel to our Bay Area lab and data center sites.

Strong candidates may also have experience with

• Operating GPU, accelerator, or HPC clusters, especially liquid‑cooled ones (in‑rack CDUs, facility liquid cooling, leak detection and automated shutdown).
• OCP hardware, including Open Rack v3, DC bus bar power, and OpenBMC.
• Hardware bring‑up and NPI environments (EVT, DVT, PVT) where the platform is still changing.
• High‑performance networking: RoCE/RDMA fabrics, Tomahawk‑class switches, Arista EOS or SONiC, and AEC, DAC, and optical cabling at 400G.
• Slurm and Kubernetes cluster operations.
• Storage engineering with NVMe, mdadm RAID, and NFS, including performance benchmarking.
• Asset management or DCIM systems.
• Colocation build‑outs: cross‑connects, carrier circuits, and remote‑hands coordination.
• Identity and access for infrastructure, such as FreeIPA, RADIUS, or TACACS+.
• Terraform, and running mixed Linux distributions (RHEL, Ubuntu) side by side.

Benefits

• Medical, dental, and vision packages with generous premium coverage
• $500 per month credit for waiving medical benefits
• Housing subsidy of $2.5k per month for those living within walking distance of the office
• Relocation support for those moving to San Jose (Santana Row)
• Various wellness benefits covering fitness, mental health, and more
• Daily lunch + dinner in our office
• Unlimited compute budget subject to ROI justification

How we're different

Etched believes in the Bitter Lesson. We are the first inference-focused frontier AI system. Our addressable market is the entirety of inference, unlike many of our competitors.