Member of Technical Staff

Castleisland; New York (NY); Hybrid; USD 120,000 - 150,000; Full time

Job summary

Castleisland in New York, NY is seeking a skilled engineer to work on AI agent orchestration and infrastructure at scale. You'll orchestrate hundreds of agent runs, design engaging environments, and stay at the forefront of agent evaluation methodologies.

The ideal candidate has experience in container orchestration, distributed systems, and running LLM workloads efficiently under load. You'll work with a talented team and engage directly with AI researchers to innovate cutting-edge solutions.

Qualifications

• Experience with Kubernetes to manage container orchestration.
• Built or maintained message-driven systems like SQS, Kafka.
• Experience running LLM workloads under heavy load.

Responsibilities

• Monitor and manage agent orchestration at scale.
• Design realistic environments for agent evaluations.
• Support new integrations with external orchestration frameworks.

Skills

Container orchestration

Distributed systems

LLM infrastructure

Kubernetes

Message-driven architectures

Job description

About Us

Chakra Labs' mission is to encode human taste into intelligence. We build high-fidelity trajectories and environments for frontier AI research, working with several of the top labs.

Our work sits at the intersection of post-training, agent environments, data quality, and research infrastructure. We care about building systems that make models better in ways that are measurable, useful, and hard to fake.

What You'd Work On

• Agent orchestration at scale. Hundreds of agent runs at once, each with its own stateful environment. 100M tokens per minute across the fleet. You own the dispatch layer: SQS, concurrency control, failure handling.

• Environment and task design. We need environments that feel real and scenarios that actually push agents to their limits. You'd figure out how to build new evaluations and design the tasks that test what matters, not just what's easy to measure.

• New frontiers. The agent evaluation space is moving fast. You'd stay on that edge, supporting new environment modalities and shipping integrations with external orchestration frameworks.

About You

• Container orchestration. You're comfortable running Kubernetes or similar in production. Auto-scaling, pod lifecycle, persistent storage, networking. You can figure out why something won't schedule and reason about resource contention.

• Distributed systems. You've built or maintained message-driven architectures. SQS, Kafka, or similar. You know how to keep jobs moving when things back up, retry without duplicating, and fail without losing work.

• LLM infrastructure. You've run LLM workloads at scale. Token instrumentation, rate limit handling, prompt caching, multi-provider routing. You've built the plumbing between models and external tools, and you know what it takes to keep it all running under load.

• Experience. No hard rule. Ideally at least 3 years at this level, but less works if the above sounds like you.

What Makes This Different

• It's infra, but the workload is AI agents. You're monitoring model behavior alongside pod health, debugging token throughput alongside network throughput.

• Our customers are AI researchers and labs. You'd work directly with the people pushing the frontier of what agents can do, and build the infrastructure they run it on.

• Ownership, not theater. You own whole systems, not tickets in a queue. One week you're shipping a new environment type, the next you're scaling the dispatch layer to handle 10x the throughput. You will ship your work to real customers, and build things that didn't exist a month ago.

• The team. Our team is ex-Stripe, Snap, AWS, Microsoft - you'll work with a small team who has years of shipping high-impact products over the last decade.

• Cutting edge. You will get to touch the latest and greatest technologies across the data, AI, and infrastructure stack.