Software Engineer

MongoDB; United States; On-site; USD 130,000 - 195,000; Full time

Job summary

MongoDB is seeking a software engineer to build tooling and evaluation systems behind its agent skills. You will design CLIs, libraries, and MCP integrations used in local development and CI, and integrate tooling into GitHub Actions.

In this role you will tackle loosely defined agent problems, convert them into durable internal systems, and ship observable metrics such as latency and token usage. You will collaborate across teams and own projects independently while improving correctness and

Qualifications

• Experience building eval harnesses, benchmark datasets, quality metrics, LLM-as-judge workflows, or human-review tooling
• Experience with Go, Python, JavaScript/TypeScript, Java, or C#
• 2+ years of experience building production software, developer tools, internal platforms, or automation systems
• Experience building CLIs, libraries, test infrastructure, static analysis, or CI/CD workflows
• Experience reasoning about correctness and safety with ambiguous input, nondeterministic output, false positives, or untrusted content
• Ability to design systems that are usable by developers and reliable in automation
• Experience with GitHub Actions security, secret handling, static rule engines, or policy enforcement
• Experience moving prototypes into production
• Comfort in an evolving R&D environment where the right abstraction emerges through prototypes and feedback
• Written and verbal communication, including explaining technical trade-offs and aligning stakeholders across teams
• Software engineering fundamentals in API design, testing, error handling, and maintainability
• Experience with agentic systems, LLM applications, prompt or rubric-based evaluation, or AI-assisted development

Responsibilities

• Build and maintain agent tooling, evaluation systems, and CI workflows
• Design evaluation datasets and workflows to compare agent behavior against baselines
• Create agent metrics and observability: latency, token usage, outcomes
• Design safety and quality gates for agent-authored content
• Collaborate with engineers, security, and product teams
• Ship tooling to test, review, and adopt agent skills
• Own projects independently while collaborating on shared systems

Skills

Eval harnesses

Go

Python

TypeScript

JavaScript

CI/CD

GitHub Actions

LLM eval

Agentic systems

Production software

Tools

GitHub Actions

Job description

• The Agent Research and Tooling team, part of MongoDB’s AI Builder Experience organization, owns the platform layer around agents: how teams author, distribute, evaluate, monitor, and improve agent skills and agent behavior
• We are hiring a software engineer to build and maintain the tooling, evaluation systems, and quality gates behind MongoDB’s agent skills
• This is a software engineering role at the intersection of developer tooling, applied AI, and software quality
• You will take loosely defined agent and tooling problems, break them into workable plans, and ship durable internal systems: command-line tools, reusable libraries, evaluation harnesses, and CI workflows
• Build and maintain agent skills and the infrastructure to validate, evaluate, publish, and maintain them
• Design evaluation datasets and workflows that compare agent behavior against a baseline and produce actionable quality signals
• Build agent metrics and observability: skill selection and routing, success and failure outcomes, tool calls, latency, and token usage
• Design safety and quality gates for agent-authored content: rule packs, static analysis, confidence thresholds, structured verdicts, and bounded suppression
• Create CLIs, libraries, and MCP integrations that other repositories adopt and that run in local development and CI
• Integrate tooling into GitHub Actions and other CI workflows, including secrets, annotations, exit codes, and artifacts
• Build code-generation quality checks, such as anti-pattern catalogs and linting for AI-generated MongoDB code
• Investigate real failures such as nondeterministic results, false positives, and unsafe generated guidance, and turn them into reusable improvements
• Collaborate with engineers, security partners, and product teams; communicate trade-offs, risks, and ownership across teams
• In your first year, you will:
• Ship tooling that makes agent skills or developer workflows easier to test, review, and adopt
• Improve the quality and interpretability of evaluations, not just their count
• Convert recurring manual work and fragile scripts into documented, reusable automation
• Make security, correctness, and operational trade-offs explicit in the designs you ship
• Earn adoption from partner teams through clear interfaces and reliable CI
• Own projects independently while collaborating on shared systems

• Experience building eval harnesses, benchmark datasets, quality metrics, LLM-as-judge workflows, or human-review tooling
• Experience with Go, Python, JavaScript/TypeScript, Java, or C#
• 2+ years of experience building production software, developer tools, internal platforms, or automation systems
• Experience building CLIs, libraries, test infrastructure, static analysis, or CI/CD workflows
• Experience reasoning about correctness and safety with ambiguous input, nondeterministic output, false positives, or untrusted content
• Ability to design systems that are usable by developers and reliable in automation
• Experience with GitHub Actions security, secret handling, static rule engines, or policy enforcement
• Experience moving prototypes into production
• Comfort in an evolving R&D environment where the right abstraction emerges through prototypes and feedback
• Written and verbal communication, including explaining technical trade-offs and aligning stakeholders across teams
• Software engineering fundamentals in API design, testing, error handling, and maintainability
• Experience with agentic systems, LLM applications, prompt or rubric-based evaluation, or AI-assisted development