AI Evaluation Engineer: RL Environments & Agents

MaxIT Consulting - Max Corporate Group; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

MaxIT Consulting - Max Corporate Group seeks an AI Evaluation Engineer to design and build environments, evaluation systems, and infrastructure for training and assessing long-horizon enterprise AI agents. The role focuses on creating realistic agent environments and robust evaluation pipelines in a San Francisco setting.

The candidate will work across languages and stacks, shipping production-quality infrastructure and addressing reward design and trajectory analysis.

Qualifications

• Hands-on experience with AI environments and evaluation systems.
• Experience with RL infrastructure or agent-training systems.
• Strong software engineering fundamentals.
• Ability to build and ship production-grade infrastructure.
• Knowledge of evaluation methods, reward design, and graders.

Responsibilities

• Design evaluation environments for long-horizon enterprise agent workflows.
• Define tasks, state, tools, graders, and reward signals for evaluation.
• Build high-fidelity representations of complex enterprise software environments.
• Develop infrastructure for rollouts, orchestration, and grader pipelines.
• Measure correctness and efficiency across multi-step agent behavior.
• Investigate evaluation failures and reward-quality issues.
• Build production-quality systems beyond notebook research.

Skills

AI environments

Reinforcement learning infra

Software engineering

Infrastructure shipping

Evaluation methodology

Cross-language work

Job description

MaxIT Consulting - Max Corporate Group seeks an AI Evaluation Engineer to design and build environments, evaluation systems, and infrastructure for training and assessing long-horizon enterprise AI agents. The role focuses on creating realistic agent environments and robust evaluation pipelines in a San Francisco setting.

The candidate will work across languages and stacks, shipping production-quality infrastructure and addressing reward design and trajectory analysis.