Senior AI Infra Engineer: Sandbox & Scale for Agents

Echelon; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health insurance

401(k)

Team meals

Unlimited PTO

Job summary

Echelon is building the AI platform for Business Operations in San Francisco. We are hiring an engineer to design and operate the secure execution substrate for our agents, scaling ephemeral sandboxes, optimizing startup performance, and ensuring durable long-running workloads without compromising tenant isolation.

You will own sandbox lifecycle systems, filesystem primitives, and orchestration services, while aligning with security and observability goals in a multi-tenant environment.

Qualifications

• 5+ years building production infrastructure, distributed systems, or execution runtimes.
• Ownership of containerized workloads in a multi-tenant production environment.
• Strong Linux systems knowledge across processes, filesystems, networking and performance.
• Experience with Kubernetes and infrastructure as code on AWS or Azure.
• Strong programming ability in Go, Rust, TypeScript, or Python.
• Experience designing for retries, idempotency, backpressure, observability and safe rollouts.
• Security instincts for executing untrusted or model-generated work.

Responsibilities

• Design and operate sandbox lifecycle systems for agent code execution and tool use.
• Improve sandbox startup time, density, scheduling, warm pools, caching, and resource use.
• Build fast, reliable filesystem primitives for large artifacts and state.
• Enforce tenant isolation, network policy, secrets boundaries, quotas, and least-privilege access.
• Make long-running agent jobs durable through checkpointing, retries and recovery.
• Scale orchestration and control-plane services during bursts and growth.
• Build observability for resource pressure, failures, latency, and queue health.
• Run load tests, capacity plans, and incident reviews; fix root causes.

Skills

Distributed systems

Linux internals

Kubernetes

Cloud (AWS/Azure)

Go / Rust / Python

Security for untrusted work

Tools

Firecracker

gVisor

Kata Containers

Temporal

Kafka

Job description

Echelon is building the AI platform for Business Operations in San Francisco. We are hiring an engineer to design and operate the secure execution substrate for our agents, scaling ephemeral sandboxes, optimizing startup performance, and ensuring durable long-running workloads without compromising tenant isolation.

You will own sandbox lifecycle systems, filesystem primitives, and orchestration services, while aligning with security and observability goals in a multi-tenant environment.