We are working with a next-generation AI compute start-up building foundational technology for custom AI accelerators. With a proven engineering team and a working core platform already in place, they are entering a high-growth phase—transitioning from core development to full production delivery.
The Role
You will architect and own the agentic infrastructure layer of the platform—building the gateways, runtime orchestration, tool integrations, and security boundaries that allow autonomous agents to run safely and reliably in production.
What You’ll Do
- Deploy & Scale Agent Runtimes: Harden and operate open-source agent runtimes and gateways across internal clusters and enterprise environments, integrating them with identity, networking, and observability stacks.
- Build Core Tooling & MCP Integrations: Develop custom plugins and Model Context Protocol (MCP) servers that connect autonomous agents directly to engineering workflows, CI/CD pipelines, and cluster scheduling.
- Architect Agent Security: Design zero-trust governance for tool-using agents, covering granular permissions, sandboxing, secrets management, audit logging, and supply-chain vetting for third-party plugins.
- Drive Telemetry & Evaluations: Build observability around token usage, latency, and cost per task, alongside automated regression evaluations and failure analysis for upstream releases.
- Collaborate Across the Stack: Partner with runtime and compiler teams to translate real-world agent workloads into system-level requirements, while contributing fixes upstream to keep internal forks minimal.
About you
- Production Infrastructure Engineer: You have designed, deployed, and maintained backend or distributed systems that stayed resilient under real-world load, with strong command of Linux, containers, Kubernetes, and SSO.
- TypeScript & Python Builder: You have deep proficiency in TypeScript and Node.js for building agent gateways and control planes, paired with solid Python for ML tooling and evaluation pipelines.
- Open-Source Agent Contributor: You have a public record of meaningful contributions (merged PRs, issue triage, or RFC discussions) to open-source agent frameworks or MCP server/SDK repositories.
- Systems & Security Mindset: You understand the internals of tool calling, memory management, and always-on execution, and you design proactively around least privilege and prompt injection risks.
- AI-Native Workflow: You use AI-assisted coding tools daily to ship and debug faster, and you communicate clearly through actionable design docs.
Bonus
- Maintainer or reviewer status on an open-source agent project, or authorship of a widely used MCP server or plugin.
- Experience serving LLMs at scale (inference engines, batching, quantization, and accelerator scheduling) or operating clusters with mixed hardware.
- Familiarity with GPU programming models or ML compiler stacks (e.g., LLVM, MLIR, Triton).
- Background in red-teaming autonomous systems or building agent evaluation benchmarks.