Applied AI Engineer

Monte Carlo

United States

À distance

USD 180 000 - 240 000

Plein temps

Il y a 2 jours
Soyez parmi les premiers à postuler

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Avantages offerts par ce poste

Remote-first environment
Competitive compensation
Equity

Résumé du poste

Monte Carlo is seeking a backend/model‑layer engineer to own end‑to‑end ML features that power agent trust. You’ll move from research and prototyping to production, partnering with engineering and data science to ship reliable, scalable components that improve agent behavior and reliability.

You’ll work primarily on the backend with a focus on ML systems, experimentation, and production-grade tooling. A strong Python background and several years in ML or data science are essential, with

Qualifications

  • 3+ years in ML, data science, or software engineering.
  • Experience building production ML systems.
  • Strong Python proficiency and ability to reason about model behavior.

Responsabilités

  • Research, prototype, and productionize agent-powered features.
  • Design and ship features like root-cause analysis and incident triage.
  • Build evaluation infrastructure with golden datasets and regression suites.
  • Manage retrieval and context pipelines over metadata and logs for production clarity.
  • Collaborate with data science and PM to align ML capabilities with product goals.

Connaissances

Python
ML background

Outils

Snowflake
Databricks
dbt
Airflow

Description du poste

About Monte Carlo

Monte Carlo is the agent trust platform that autonomously observes, fixes, and continuously improves agents and the data that powers them — so enterprises can scale trust at the speed of AI. Bayes, our AI teammate, does the work: detecting, triaging, troubleshooting, and fixing issues in AI agents and the data powering them. The result is self‑improving agents that get better with every fix instead of drifting. More than 400 enterprises, including Amazon, PepsiCo, CNN, Nasdaq, and JetBlue, rely on Monte Carlo. Learn more at montecarlo.ai.

The Role

We're building the products that tell enterprises whether their AI agents can be trusted — and we need someone who works end to end, from an ambiguous problem statement through research, prototyping, and production. You'd get the problem, not the spec: research the approaches, prototype, prove what works, build it, and integrate it into the platform alongside our engineering and data science teams. This role exists because agent observability moved from roadmap to revenue faster than anyone predicted, and the work is now on the critical path.

What You'll Do
  • Take an open problem end‑to‑end — from research and prototyping through production, killing what doesn't work before it becomes someone's roadmap.
  • Design and ship agent-powered features — root‑cause analysis, incident triage, monitor generation — and integrate them into the platform with our engineering team.
  • Build the eval infrastructure that makes those features safe to change: golden datasets, regression suites, offline and online scoring, and the judgment calls about what \"good\" means.
  • Own retrieval and context pipelines over customer metadata, lineage, and query history, and instrument agent behavior in production — traces, failure taxonomies, cost and latency budgets — to close the loop on quality.
  • Partner with data science on detection quality and experiment design, and with PM on what an agent should do versus what it merely can do.
  • Set the technical bar for how we build with LLMs — patterns, guardrails, and the internal tooling other engineers reuse.
What We're Looking For

You've built agents in production with real users. Not integrated a framework. Not worked on a team that had one. Built them — agents with real autonomy and internal loops, where the model uses tools and decides what to do next without a human in the middle, and you kept them running once real users showed up. RAG with a wrapper doesn't count. Neither does a set of MCP tools point at an API. You've run evals and monitored agents after launch. Agents are non-deterministic, so normal tests don't work on them. You've owned an eval framework — golden datasets, regression suites, offline and online scoring — not a folder of one‑off scripts. And you've watched agents in production, not just in dev.

Python, plus an ML or data science background.

Python is your daily language and you're solid on the backend, though you don't need to be a distributed systems specialist. You understand models well enough to reason about how they behave — you're not an application engineer calling someone else's API.

3+ years in ML, data science, or software engineering — at companies that ship quickly. Startups, AI‑native teams, or high‑growth product companies where the release cadence is measured in weeks, not quarters.

If all of your experience sits inside large, process‑heavy organizations with the platform already built for you, this seat will be a hard adjustment.

You work from a problem, not a spec. Handed an ambiguous problem statement, you design the experiment, build the smallest version to test it, and take what works into production. You use AI tools every day. Claude or its equivalents are part of how you write code and do research, not something you tried once.

This is backend and model layer work, by the way — no frontend. You'd rather ship than polish. Most of this work needs a good answer quickly, not a perfect one eventually. You can tell which problems are the exception and deserve real depth — and you'll say no to the version that demos well and falls apart in production.

Nice to have:

statistics and hypothesis testing, applied rather than theoretical.

Building and maintaining MCP servers.

Experience in the data and cloud space — Snowflake, Databricks, dbt, Airflow.

This Is Not For You

If Your AI work is retrieval with a wrapper, or MCP tools pointed at an API — nothing that decides and acts on its own.

Your LLM experience is prototypes, notebooks, and demos that never carried production traffic.

You need a fully specified problem before you start, or you're uncomfortable with the ambiguity of a category being invented in real time #BI-REMOTE.

Why Monte Carlo

We created the data observability category, and we're doing it again with agent trust — you'll build where the market is forming, not where it's settled.

  • Series D, $236M raised, backed by Accel, Redpoint, Notable Capital, ICONIQ Growth, and Salesforce Ventures.
  • Customers include HubSpot, Fox, Nasdaq, Toast, and Mercado Libre — your work ships to enterprises with real stakes.
  • Snowflake Partner of the Year and a verified connector in Anthropic's Claude AI directory.

Remote‑first by design since day one, and recognized as a Best Workplace for it.

Competitive compensation, equity, and a remote‑first environment.

Come As You Are.

Equality

Equality is a core tenet of Monte Carlo's culture.

We are committed to building an inclusive global team that represents a variety of backgrounds, perspectives, beliefs, and experiences.

Monte Carlo is an equal‑opportunity employer.

We celebrate diversity and are committed to creating an inclusive environment for all employees.

Recognition

Monte Carlo Named 2025 Databricks Data Governance Partner of the Year.

We were recently recognized as the #1 Data Observability Platform by G2 for the 4th consecutive quarter. See our G2 reviews here!

Monte Carlo Named to G2's Best Software Products of 2026.

Monte Carlo was featured on Database Trends and Applications (DBTA’s) Trend‑Setting Products for 2025!

We are super proud to be named the 2026 Best Place to Work by Built In!

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Applied Forward Deployed Engineer (Post-Sales)
Applied Forward Deployed Engineer (Post-Sales)

Monte Carlo • États-Unis

À distance
USD 150 000 - 190 000
Remote-first culture
Sr. Sales Engineer (Forward Deployed Engineer - Pre Sales)
Sr. Sales Engineer (Forward Deployed Engineer - Pre Sales)

Monte Carlo • États-Unis

À distance
USD 150 000 - 210 000
Senior Growth Account Executive (Remote)
Senior Growth Account Executive (Remote)

Monte Carlo • États-Unis

À distance
USD 1 000 000 - 1 500 000
Remote-first culture
Equity
Competitive compensation
Applied Forward Deployed Engineer (Post-Sales)
Applied Forward Deployed Engineer (Post-Sales)

FDE Times • États-Unis

À distance
USD 150 000 - 220 000
Senior Account Executive, Key Accounts
Senior Account Executive, Key Accounts

Monte Carlo • États-Unis

À distance
USD 120 000 - 190 000
Strategic Account Executive, AMER
Strategic Account Executive, AMER

Monte Carlo • États-Unis

À distance
USD 140 000 - 190 000
Remote-first
Enterprise Account Executive AMER
Enterprise Account Executive AMER

Monte Carlo • États-Unis

À distance
USD 120 000 - 180 000
Strategic Sales Development Representative
Strategic Sales Development Representative

Monte Carlo • Town of Boston (NY)

À distance
USD 55 000 - 85 000
Enterprise Sales Manager, Americas
Enterprise Sales Manager, Americas

Monte Carlo • États-Unis

À distance
USD 180 000 - 260 000
Enterprise Sales Manager, Americas (Remote)
Enterprise Sales Manager, Americas (Remote)

Monte Carlo • États-Unis

À distance
USD 360 000 - 400 000