Remote AI Agent Orchestrator

Outlier; San Jose (CA); Remote; USD 57,000 - 81,000; Part time

Benefits offered by this job

Missions incentives

AI tool access ($200+/month)

Job summary

Outlier is offering a remote freelance role to help build and evaluate autonomous agent systems. You will craft evaluative rubrics, debug agent traces, and stress test multi-turn agents in live environments.

The work embraces modular software design and rigorous feedback to improve AI agent performance. Ideal candidates have 2+ years in backend engineering, AI automation, and complex integrations, with a track record of production-grade systems and attention to detail.

Qualifications

• 2+ years of experience in backend engineering, AI automation, or complex systems integration.
• Proven ability to build and maintain production-grade software with modular separation (e.g., distinct services for data parsing, logic processing, and reporting).
• Practical experience building for live, non-mocked environments and handling multi-turn system interactions.
• Outstanding attention to detail and the ability to provide clear, high-density technical feedback on complex system behaviors.

Responsibilities

• Write Eval Rubrics
• Debug Agent Traces
• Stress Test Agent

Skills

2+ years experience

Backend engineering

AI automation

Complex systems integration

Tech feedback

Job description

Outlier is offering a remote freelance role to help build and evaluate autonomous agent systems. You will craft evaluative rubrics, debug agent traces, and stress test multi-turn agents in live environments.

The work embraces modular software design and rigorous feedback to improve AI agent performance. Ideal candidates have 2+ years in backend engineering, AI automation, and complex integrations, with a track record of production-grade systems and attention to detail.