Remote Production AI Agent Engineer

Monte Carlo; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Remote-first culture

Competitive compensation

Job summary

Monte Carlo is hiring to build the products that tell enterprises whether their AI agents can be trusted. You would end-to-end research, prototyping, and production work, turning ambiguous problems into measurable experiments and reliable features.

You will design agent-powered capabilities, implement safe evaluation infrastructure, and partner with data science and product teams to align what agents should do with what they can do.

Qualifications

• Built agents in production with real autonomy.
• Maintained eval frameworks with golden datasets and regression suites.
• Strong Python and ML/data science background.
• Solve ambiguous problems by designing experiments and testing quickly.
• Regularly use AI tools to write code and conduct research.
• Prefer shipping quickly over polishing perfect demos.

Responsibilities

• Take an open problem end-to-end from research to production.
• Design and ship agent-powered features and integrate with the platform.
• Build evaluation infrastructure with datasets and scoring.
• Own retrieval and context pipelines, instrument agent behavior in production.
• Collaborate with data science on detection quality and with PM on scope.
• Set technical bar for building with LLMs, patterns and guardrails.

Skills

Production agents

Python

AI tooling

Experiment design

Problem solving

Tools

Eval framework

Job description

Monte Carlo is hiring to build the products that tell enterprises whether their AI agents can be trusted. You would end-to-end research, prototyping, and production work, turning ambiguous problems into measurable experiments and reliable features.

You will design agent-powered capabilities, implement safe evaluation infrastructure, and partner with data science and product teams to align what agents should do with what they can do.