Director of MLE - Build AI Evaluation Function at 2AI

Evidence Action

United States

Remote

USD 142,000 - 162,000

Full time

12 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

2AI is seeking a Director of Monitoring, Learning, and Evaluation to build the MLE function from the ground up. You will set standards, design measurement systems, and push frontier AI methods to accelerate evidence generation for Agriculture and Health programs.

You will lead a growing team, oversee budget, and communicate findings to funders and partners from the US, with international travel expected. This is a founding role with broad scope.

Qualifications

  • 10+ years of progressive experience in monitoring, learning, and evaluation in global health or development.
  • Led rigorous quantitative evaluation and built or reshaped an MLE function.
  • Experience implementing AI-enabled MLE approaches and frontier AI tools.

Responsibilities

  • Set MLE strategy and standards for Agriculture, Health, and the Incubator.
  • Push frontier on AI-enabled MLE approaches, including AI-assisted evidence synthesis.
  • Build and lead the MLE team and recruit initial hires.
  • Assess the evidence base for early-stage programs and stress-test hypotheses.
  • Review outcome and cost-effectiveness models for live programs.
  • Design measurement systems and rapid experimentation within live programs.
  • Drive data-driven decision-making across the portfolio.
  • Own the MLE budget and resourcing plan.
  • Represent 2AI's MLE work to funders and partners.

Skills

MLE leadership
Quantitative analysis
Data-driven decisions
AI in MLE

Education

Advanced degree in economics/statistics/public health

Tools

R
Python
Stata

Job description

Director of MLE - Build AI Evaluation Function at 2AI — Remote

Founding-Role Opportunity: This is a rare chance to build an MLE function from zero at a mission-driven AI organization with high-profile advisors (including Anthropic's CEO and a Nobel Laureate). Founding directors at organizations of this trajectory carry outsized influence on culture, standards, and future team composition — a significant career accelerator.

Compensation vs. Market: The $142,000–$162,000 USD range is competitive for a senior MLE director in the nonprofit/global development sector, where equivalent roles at large INGOs typically range from $120,000–$155,000. It is modestly below pure tech-sector equivalents, but the mission premium and equity-equivalent founding influence are meaningful compensating factors.

Skills You'll Develop:

  • Practical application of frontier AI tools (LLMs, agentic pipelines) to real-world evaluation — a fast-growing, high-demand skill set.
  • Executive leadership: budget ownership, team-building, and board/funder communication.
  • Cross-sector fluency spanning global health, agriculture, AI labs, and international development finance.

Green Flags: Evidence Action's track record (530M+ people reached), mission clarity, and the spin-out structure suggest serious organizational ambition. The honest acknowledgment of rapid growth "dynamism" shows cultural self-awareness.

Potential Red Flags: As a startup spinning out of a parent organization, benefits and systems are still being established. Title and scope are explicitly still being defined — candidates should negotiate carefully and get role boundaries in writing. The 15–20% travel requirement and broad mandate may challenge work-life balance in the early months.

About The AI Access Initiative (2AI)

We're at an inflection point in artificial intelligence, presenting both tremendous potential opportunity and risk for people in developing countries. AI capabilities are improving on a timescale measured in months, while development systems often move on timescales measured in decades. Much of the technology needed for positive impact exists today, but few organizations exist to bridge research, deployment, and scaled implementation: we must act now to ensure AI's benefits reach people in poverty in LMICs.

Incubated at Evidence Action, we're scaling The AI Access Initiative — an organization focused on scaling AI-enabled 'big bets' to benefit tens or hundreds of millions of people in poverty in low- and middle-income countries (LMICs). We operate at the intersection of global development actors, top AI labs, and leading researchers to drive meaningful access to AI's benefits for the 3.5 billion people living in poverty globally. We create "public good," open-sourced playbooks, toolkits, and insights that define how to design and launch tractable and impactful AI-enabled programs. Given the scale of opportunity, we expect our portfolio to expand substantially, but to start, we're scaling two programs focused on AI in Agriculture and AI in Health.

We are led by former Evidence Action CEO Kanika Bahl, a founding member of Anthropic's Long-Term Benefit Trust, and advised by Nobel Laureate Michael Kremer; Dario Amodei, CEO of Anthropic; and Kent Walker, President, Global Affairs for Alphabet and Google. This work builds on Evidence Action's track record reaching 530M+ people with cost-effective, evidence-based programs across 9 countries in Africa and Asia, with a focus on last-mile delivery.

The Role

We are seeking a Director of Monitoring, Learning, and Evaluation (MLE) to build and lead 2AI's MLE function from the ground up. You will set the evidence standard for our live Agriculture and Health programs and for new interventions developed by our Incubator, and you will be accountable for the technical rigor of what we measure, learn, and report.

This is a founding role. The team, systems, and protocols do not exist yet, and you will design them. We are also looking for someone who pushes on how frontier AI can make MLE faster, cheaper, and more rigorous, and who shows our team and the wider field what is possible.

You Will:

  • Set MLE strategy and standards: Define 2AI's approach to measurement and evaluation across Agriculture, Health, and the Incubator, and hold a consistently high bar for rigor from design through dissemination, including minimum standards for how we act on weak or emerging evidence, uncertainty, and key assumptions.
  • Push the frontier on AI-enabled approaches to MLE: Test and create industry best practices for AI-enabled MLE approaches, and share learnings with the sector. Explore frontier AI tools and techniques, such as LLM-assisted evidence synthesis and agentic analysis pipelines, to improve the quality, speed, and cost of evaluation.
  • Build and lead the MLE team: Design the team structure, recruit the first hires, and develop them into a high-performing group.
    • Set a clear performance bar and build a culture of active feedback.
    • Provide technical supervision and mentorship on diagnosing and solving complex problems.
  • Assess the evidence base for early-stage programs (~30% of your time): Provide advisory support to 2AI's Incubator (new program development unit) to strengthen rigor and consistency as we assess and select new interventions.
    • Review existing evidence with the Incubator team and stress-test hypotheses.
    • Serve as a thought partner on program assessment and pilot design, identifying where 2AI's program design aligns with or departs from the existing evidence and ensuring pilots are designed to be measurable and focused on the most important learning goals.
    • Identify evidence gaps and develop stage-appropriate learning plans, recognizing the limited evidence base for AI-enabled interventions.
  • Review outcome and cost-effectiveness models: Pressure-test cost-effectiveness analyses built by the Incubator team, checking how evidence feeds into key assumptions and whether results pass a reality check.
  • Build measurement systems for live programs: Design protocols and data collection processes grounded in operational realities, and ensure rigorous iteration as Agriculture and Health programs scale. Build rapid experimentation (such as A/B testing of AI tool features and delivery approaches) into live programs so teams can learn and iterate quickly.
  • Drive data-driven decision-making: Identify the highest-value questions across the portfolio, and make sure MLE findings reach the program and country teams making decisions on the ground.
  • Own the MLE budget and resourcing plan: Plan resources for a new function, make tradeoffs aligned with organizational priorities, and secure the buy-in and funding to sustain it.
  • Represent 2AI's evidence externally: Communicate methods, findings, and learning to funders and partners, invite and address external feedback, and publish MLE work as public goods.

Requirements

The ideal candidate has led rigorous quantitative evaluation in global health or international development, has built or reshaped an MLE function, and is energized by using frontier AI to do this work better. They favor implementation over theory, balance rigor with speed, and can make hard calls with incomplete data.

We are still actively scoping our roles and structure, reflecting the dynamism of this fast-growing phase of the organization. 2AI is open to candidates across a range of seniority levels for this role; title, scope, and compensation will be assessed candidly with each candidate throughout the process based on their experience and fit.

You will likely have:

  • 10+ years of progressive experience in monitoring, learning, and evaluation, including hands-on leadership of a rigorous quantitative function that enables data-driven decisions in global health or development.
  • An advanced degree in economics, statistics, public health, or another quantitative field, or a comparable mix of advanced training and technical experience. A PhD is optional.
  • Expertise in evaluation design and analysis, including statistically rigorous impact evaluations, process evaluations, and mixed-methods approaches, with experience managing large-scale quantitative evaluations from design through dissemination.
  • Fluency in interpreting existing evidence and translating it into informed program decisions.
  • A track record of building teams and developing senior technical talent, including setting performance standards and giving clear, frequent feedback.

You will also bring:

  • Strategic judgment: the instinct to identify problems early and the curiosity to ask the questions that clarify context.
  • Clear understanding of and experience in using AI in the MLE process. Experience with R, Python, Stata, or similar tools is a plus.
  • Quantitative modeling under uncertainty, including outcome and cost-effectiveness estimates for early-stage programs.
  • Comfort with ambiguity and experience setting up new systems, structures, and standards where none exist.
  • A bias to action: the ability to make tough choices among imperfect options and communicate the tradeoffs.
  • Technical communication: the ability to adjust style between MLE specialists, funders, government counterparts, and non-technical audiences to drive decisions.
  • Relationship-building with program teams, partners, and peer organizations, with an external presence through publications, conference talks, or similar platforms.

Nice to have:

  • In-the-field experience in low- and middle-income countries.
  • Experience supporting government partners with data-led decisions on program implementation.

Position Location

This role is open to candidates in the US, Canada, Kenya, and India. This role may require international travel (~15–20%) to partner countries. Employing entity, compensation, and benefits will depend on the selected candidate's location and local statutory requirements. Applicants must be legally authorized to work in the country where they are based. We are unable to sponsor or take over sponsorship of an employment visa at this time.

Evidence Action is an Equal Opportunity Employer with a commitment to diversity. All individuals, regardless of personal characteristics, are encouraged to apply.

The expected role range for this position is listed below. We consider multiple factors when determining the base salary for a role, including but not limited to: role scope, program budgets, internal equity, and a candidate's qualifications and/or prior experience.

The salary range listed reflects the US market. We also hire in other geographies, and compensation for candidates based outside the US will be adjusted to local market rates.

Note: Pay and benefits will be commensurate with the role specifications, local statutory requirements, and the cost of labor in the markets where we operate.

The pay range for this role is: $142,000 – $162,000 USD per year.

While we are currently incubated at Evidence Action, 2AI is in the process of spinning out as an independent organization. New employees will be hired directly by 2AI. We will offer a comprehensive benefits package, including health, vision, and dental, retirement plans, generous and flexible leave options, as well as other employee perks.

Use of AI in our hiring process: We believe strongly in the importance of human judgment throughout hiring, from screening to final decisions. We use AI to support our team across the process, including application review, note-taking, scheduling, and project management. AI does not make hiring decisions or reject candidates. We do not use models that train on inputs or candidate data. By submitting your application, you acknowledge that we will use your information for recruitment purposes, including through these AI-supported tools, as described in our privacy notice.

Is this role fully remote, or is office attendance required? The posting does not mandate office attendance and hiring is open across four countries (US, Canada, Kenya, India), strongly suggesting remote work is supported. However, the role requires approximately 15–20% international travel to partner countries. Can 2AI sponsor a work visa for this role? No. Applicants must already be legally authorized to work in their base country. 2AI is unable to sponsor or take over sponsorship of an employment visa at this time. What is the salary range and does it apply outside the US? The posted range is $142,000–$162,000 USD per year and reflects the US market. Candidates based in Canada, Kenya, or India will have compensation adjusted to local market rates. Is a PhD required for this role? A PhD is explicitly listed as optional. The role requires an advanced degree in economics, statistics, public health, or a related quantitative field, or a comparable combination of advanced training and technical experience. What does it mean that this is a 'founding role'? The MLE team, systems, and protocols do not yet exist. The Director will design the team structure, recruit initial hires, build measurement systems, and set the evidence standards for the entire organization from the ground up.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Director, Monitoring, Learning, and Evaluation, AI Access Initiative
Director, Monitoring, Learning, and Evaluation, AI Access Initiative

Evidence Action • United States

Remote
USD 142,000 - 162,000
Health, vision and dental benefits
Retirement plans
Generous leave options
+1
Manager / Senior Manager, Incubation (Evidence), The AI Access Initiative
Manager / Senior Manager, Incubation (Evidence), The AI Access Initiative

Evidence Action • United States

Remote
USD 100,000 - 135,000
Health insurance
Vision insurance
Dental insurance
+2
Manager / Senior Manager, Incubation, The AI Access Initiative
Manager / Senior Manager, Incubation, The AI Access Initiative

Evidence Action • United States

Remote
USD 100,000 - 135,000
Health, vision, and dental benefits
Retirement plans
Generous leave options
+1
Senior AI Incubator Lead — Scale AI for LMICs
Senior AI Incubator Lead — Scale AI for LMICs

Evidence Action • United States

Remote
USD 100,000 - 135,000
Health, vision, and dental benefits
Retirement plans
Generous leave options
+1
Growth Strategist Role at AI Nonprofit - $135K
Growth Strategist Role at AI Nonprofit - $135K

Evidence Action • United States

Remote
USD 125,000 - 135,000
Health, vision, dental
Retirement plans
Generous leave options
Senior Manager, Growth, The AI Access Initiative
Senior Manager, Growth, The AI Access Initiative

Evidence Action • United States

Remote
USD 125,000 - 135,000
Health, Vision & Dental
Retirement plans
Generous and flexible leave
+1
Chief Financial Officer, The AI Access Initiative
Chief Financial Officer, The AI Access Initiative

Evidence Action • United States

Remote
USD 240,000 - 270,000
Health
Vision
Dental
+3
Strategic CFO for Global AI Impact (Remote)
Strategic CFO for Global AI Impact (Remote)

Evidence Action • United States

Remote
USD 240,000 - 270,000
Health
Vision
Dental
+3
Chief People Officer, The AI Access Initiative
Chief People Officer, The AI Access Initiative

Evidence Action • United States

Remote
USD 220,000 - 240,000
Health, vision, dental
Retirement plans
Generous and flexible leave
+1
Senior Manager, Growth, The AI Access Initiative
Senior Manager, Growth, The AI Access Initiative

Evidence Action • Oregon

Hybrid
USD 125,000 - 135,000
Health
Vision
Dental
+3