AI Evals Engineer – Reward Design & Agent Environments

MaxIT Consulting - Max Corporate Group

San Francisco (CA)

Presencial

USD 120.000 - 180.000

Jornada completa

Hace 7 días
Sé de los primeros/as/es en solicitar esta vacante

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Descripción de la vacante

MaxIT Consulting - Max Corporate Group, based in San Francisco, seeks a Research Engineer to build RL agent environments, evaluation systems, and supporting infrastructure for long-horizon enterprise AI tasks. You will shape tasks, state, tools, graders, and reward signals to drive robust agent evaluation.

The role emphasizes production-quality systems, cross-team collaboration, and hands-on engineering across languages and stacks. On-site with a strong collaboration culture.

Formación

  • Hands-on experience with AI environments, evaluations, reinforcement learning infrastructure, or related agent-training systems.

Responsabilidades

  • Design evaluation environments for long-horizon enterprise agent workflows.
  • Define tasks, state, tools, graders, and reward signals used to evaluate and improve agents.
  • Build high-fidelity representations of complex enterprise software environments.
  • Develop infrastructure for rollouts, orchestration, trajectory inspection, and grader pipelines.
  • Measure both correctness and efficiency across multi-step agent behavior.
  • Investigate evaluation failures, reward-quality issues, and agent behavior.
  • Build production-quality systems rather than notebook-only research prototypes.

Conocimientos

AI environments
RL infrastructure
Software engineering
Infrastructure development
Cross-language

Descripción del empleo

San Francisco, California | Primarily On-site

We are seeking an Research Engineer – RL Infrastructure & Agent Environments to build the environments, evaluation systems, and supporting infrastructure used to train and assess long-horizon enterprise AI agents.

The Opportunity

You will work on the engineering and research problems behind realistic agent environments, post-training systems, and reliable evaluation of complex multi-step workflows.

Key Responsibilities
  • Design evaluation environments for long-horizon enterprise agent workflows.
  • Define tasks, state, tools, graders, and reward signals used to evaluate and improve agents.
  • Build high-fidelity representations of complex enterprise software environments.
  • Develop infrastructure for rollouts, orchestration, trajectory inspection, and grader pipelines.
  • Measure both correctness and efficiency across multi-step agent behavior.
  • Investigate evaluation failures, reward-quality issues, and agent behavior.
  • Build production-quality systems rather than notebook-only research prototypes.
Required Qualifications
  • Hands-on experience with AI environments, evaluations, reinforcement learning infrastructure, or related agent-training systems.
  • Strong software engineering fundamentals.
  • Demonstrated ability to build and ship technical infrastructure.
  • Understanding of evaluation methodology, reward design, graders, and agent trajectories.
  • Ability to work across languages and technology stacks based on system requirements.
Candidate Profile

A PhD is not required. Strong engineering and shipped environment or evaluation systems are more important than academic credentials or publication history.

Seniority

The opportunity is open to exceptional new graduates, early-career engineers, and experienced senior candidates. Selection is based primarily on engineering strength and relevant technical work.

Work Arrangement

The role is anchored in San Francisco with a strong preference for in-person collaboration. Limited flexibility may be considered case by case for exceptional candidates.

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

AI Evaluation Engineer – Reinforcement Learning & Agents
AI Evaluation Engineer – Reinforcement Learning & Agents

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140.000 - 210.000
Member of Technical Staff – RL Environments & Evals
Member of Technical Staff – RL Environments & Evals

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 170.000 - 250.000
Evals Research Engineer – LLM Agents & RL Environments
Evals Research Engineer – LLM Agents & RL Environments

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 150.000 - 190.000
Research Engineer – RL Infrastructure & Agent Environments
Research Engineer – RL Infrastructure & Agent Environments

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 150.000 - 190.000
RL Environments & Evaluation Infrastructure Engineer
RL Environments & Evaluation Infrastructure Engineer

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140.000 - 200.000
RL Environments Engineer
RL Environments Engineer

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140.000 - 200.000
Agent Evaluation Infrastructure Engineer
Agent Evaluation Infrastructure Engineer

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 160.000 - 230.000
AI Evaluation Engineer: RL Environments & Agents
AI Evaluation Engineer: RL Environments & Agents

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140.000 - 210.000
AI Engineer, RL & Evals
AI Engineer, RL & Evals

raydar • California

Presencial
USD 175.000 - 275.000
Competitive equity
Forward Deployed AI Engineer – Agents & Research Systems
Forward Deployed AI Engineer – Agents & Research Systems

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140.000 - 210.000