AI Evaluation Engineer – RL Environments & Agents

MaxIT Consulting - Max Corporate Group; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

MaxIT Consulting - Max Corporate Group in San Francisco is seeking an AI Evaluation Engineer focused on reinforcement learning and agents to design environments, evaluation systems, and the infrastructure used to train and assess long-horizon enterprise AI workflows.

You will build high‑fidelity representations of complex software environments, implement graders and reward signals, and deliver production‑quality tools for rollouts, trajectory inspection, and robust evaluation.

Qualifications

• Hands-on experience with AI environments, evaluations, reinforcement learning infrastructure, or related agent-training systems.
• Strong software engineering fundamentals.
• Demonstrated ability to build and ship technical infrastructure.
• Understanding of evaluation methodology, reward design, graders, and agent trajectories.
• Ability to work across languages and technology stacks based on system requirements.

Responsibilities

• Design evaluation environments for long-horizon enterprise agent workflows.
• Define tasks, state, tools, graders, and reward signals used to evaluate and improve agents.
• Build high-fidelity representations of complex enterprise software environments.
• Develop infrastructure for rollouts, orchestration, trajectory inspection, and grader pipelines.
• Measure both correctness and efficiency across multi-step agent behavior.
• Investigate evaluation failures, reward-quality issues, and agent behavior.
• Build production-quality systems rather than notebook-only research prototypes.

Skills

AI environments

RL infrastructure

Software engineering

Evaluation methodology

Multi-language proficiency

Job description

MaxIT Consulting - Max Corporate Group in San Francisco is seeking an AI Evaluation Engineer focused on reinforcement learning and agents to design environments, evaluation systems, and the infrastructure used to train and assess long-horizon enterprise AI workflows.

You will build high‑fidelity representations of complex software environments, implement graders and reward signals, and deliver production‑quality tools for rollouts, trajectory inspection, and robust evaluation.