Staff AI Engineer — Autonomous Agents & Evaluation

Openkrill; United States; Hybrid; USD 200,000 - 300,000; Full time

Job summary

Openkrill is seeking a senior engineer to join a small, senior team building autonomous AI agents for executive productivity. You will develop new agent capabilities and create reliable evaluation methods to ensure real-world tasks are completed accurately before release.

You will influence end-to-end agent performance, ship capabilities in familiar tools, and build repeatable evaluation workflows while collaborating across implementation, evaluation, and iteration in a fast-moving environment.

Qualifications

• 2–11 years of relevant experience with progression.
• Experience building or evaluating production AI/ML systems, especially agents or LLM-powered products.
• Strong Python skills and familiarity with Django, React, and TypeScript.
• Experience with evaluation and monitoring tools such as Braintrust or Raindrop.
• Comfort working across implementation, evaluation, and iteration in a fast-moving environment.

Responsibilities

• Build agents that complete delegated work end to end without requiring the user to check each step.
• Develop trustworthy evaluations that measure how accurately agents perform different tasks.
• Ship new agent capabilities in the tools people already use, supported by evidence that they are ready.
• Investigate production failures and deliver durable, validated fixes.
• Build large-scale failure detection to identify known patterns and unexpected issues.
• Improve agent recovery when a tool or website fails.
• Create repeatable workflows so engineers can measure whether agent changes improve performance.

Skills

Python

Django

React

TypeScript

Tools

Braintrust

Raindrop

Job description

Openkrill is seeking a senior engineer to join a small, senior team building autonomous AI agents for executive productivity. You will develop new agent capabilities and create reliable evaluation methods to ensure real-world tasks are completed accurately before release.

You will influence end-to-end agent performance, ship capabilities in familiar tools, and build repeatable evaluation workflows while collaborating across implementation, evaluation, and iteration in a fast-moving environment.