Staff Software Engineer – Testing & Evals Frameworks

Idme; McLean, Mountain View (VA, CA); On-site; USD 150,000 - 190,000; Full time

Job summary

ID.me in McLean, VA is seeking a Staff Engineer who will own the architecture and strategy for evaluation frameworks, automation tools, and testing infrastructure across services. It requires deep engineering rigor, mentoring, and collaboration with security and product teams to embed quality at every development stage.

You will lead AI-assisted testing practices and drive CI/CD quality gates, ensuring reliable, secure software delivery for a platform that verifies identities at scale.

Qualifications

• 10+ years of software engineering experience in modern web apps and distributed systems.
• Proficiency in Python, TypeScript, JavaScript, and Java.
• Hands-on with Playwright, Jest, API automation, or evaluation frameworks.
• Experience integrating automated tests into CI/CD pipelines (GitHub Actions, ArgoCD).
• Proven ability to lead cross-team technical initiatives and mentor engineers.
• Strong written and verbal communication across engineering, product, and leadership.

Responsibilities

• Own test architecture and long-term automation strategy across services.
• Define end-to-end and integration testing for distributed systems.
• Lead testing-in-production through canaries, phased rollouts, and automated rollbacks.
• Own security-focused test automation (OAuth2, RBAC, secrets).
• Integrate automated testing into CI/CD pipelines org-wide (GitHub Actions).
• Drive AI eval culture and use AI to accelerate test authoring and reviews.
• Collaborate with Security, Platform and Product teams to embed quality throughout development.

Skills

Programming languages (Python/TypeS/JS

Test frameworks

CI/CD

Cross-team leadership

Mentoring

Communication

Education

10+ years software engineering

Master's degree in CS

PhD with 5+ years

Tools

Playwright

Jest

API automation

GitHub Actions

ArgoCD

Locust

Gatling

Job description

Company Overview ID.me is the next-generation digital identity wallet that simplifies how individuals securely prove their identity online. Consumers can verify their identity with ID.me once and seamlessly login across websites without having to create a new login and verify their identity again. Over 175 million users experience streamlined login and identity verification with ID.me at 22 federal agencies, 50 state government agencies, and more than 100 healthcare organizations. Over 600 consumer brands use ID.me to verify communities and user segments to honor service and build more authentic relationships. ID.me's technology meets the federal standards for consumer authentication set by the Commerce Department and is approved as a NIST 800-63-3 IAL2 / AAL2 credential service provider by the Kantara Initiative. ID.me is committed to "No Identity Left Behind" to enable all people to have a secure digital identity. To learn more, visit https://network.id.me/. ID.me is a full-time, in-office culture. Unless a specific job description explicitly states otherwise, all roles are on-site five days per week at one of our offices in McLean, VA; Mountain View, CA; New York City, NY; or Tampa, FL. Certain roles - such as field-based sales or other remote-by-design positions - may have different work arrangements as noted in their individual postings. At ID.me, we embrace the thoughtful use of AI tools in our daily work and there are even occasions where we leverage AI in our hiring process. However, during the interview process, we want to understand your individual skills and experiences. Therefore, we have guidelines on how AI can be appropriately used during your application and interviews which can be found here.

Position Overview

This Staff Engineer role sits at the intersection of engineering, testing, AI evals, and developer experience. You will own the architecture and strategy for the evaluation frameworks, automation tools, and testing infrastructure that give ID.me engineering teams the confidence to ship quickly and safely across functional, end-to-end, LLM/eval, and security-focused testing practices, as well as the CI/CD quality gates the org depends on. It requires deep engineering rigor, the judgment to set testing strategy for complex, asynchronous, event-driven systems, and the ability to build tooling and standards that other teams adopt widely. You will raise the quality bar across the engineering org, mentor senior and mid-level engineers, champion AI-assisted testing practices, and partner directly with engineering, product, and security teams to embed quality into every stage of development for a platform that verifies the identity of millions of users.

Key Responsibilities

• Own Test Architecture & Strategy: Set the architecture and long-term strategy for automation frameworks and testing infrastructure (Playwright, Python, Jest, API automation) across ID.me's services, and ensure comprehensive, reliable coverage org-wide.
• Lead Testing Strategy for Distributed Systems: Define end-to-end and integration testing strategy for complex, asynchronous, event-driven, and eventually-consistent systems, including workflow orchestration platforms such as Temporal.
• Drive Testing-in-Production as an Org Standard: Build and champion canary deployments , progressive/phased rollouts , synthetic monitoring, and automated rollback triggers to validate real-world behavior safely before full release.
• Own Security-Focused Test Automation: Set the standard for testing authentication/authorization flows (OAuth 2.0, OIDC), RBAC/ReBAC, secrets management, and tenant isolation.
• CI/CD Quality Gates: Integrate automated testing into CI/CD pipelines (e.g., GitHub Actions) org-wide to give every team fast, reliable feedback.
• Drive AI & Evals Culture: Build robust evaluation pipelines for AI-assisted features and LLM agents, while applying AI tools to accelerate test authoring, test plan generation, and code review across the org.
• Cross-Team Collaboration: Partner with Security, Platform, and Product teams as a testing strategist, advising on testable code structure, coverage gaps, and quality checks throughout the development lifecycle.
• Mentorship: Guide senior and mid-level engineers on test architecture, testing strategy, and quality practices.
• Debug & Root Cause: Debug and root-cause issues across frontend, backend, APIs, and distributed services, and drive fixes to closure.
• Reliability & Eval Metrics: Define and track reliability, quality, and eval accuracy metrics (e.g., time to detect, time to remediate, output quality scores) for the systems you evaluate and test.

Minimum Qualifications

• 10+ years of experience (or equivalent: 10+ years with a Master's degree, or a PhD with 5+ years) as a software engineer in modern web applications and distributed systems.
• Proficiency in Python/TypeScript/JavaScript/Java.
• Hands-on experience with Playwright, Jest, API automation, or evaluation frameworks for software/AI systems.
• Experience integrating automated tests into CI/CD pipelines (e.g., GitHub Actions, ArgoCD).
• Proven ability to lead cross-team technical initiatives and influence engineering standards.
• Track record of mentoring senior engineers and raising the quality bar across teams.
• Strong written and verbal communication across engineering, product, and leadership.

Preferred Qualifications

• API & Systems Testing: REST APIs, microservices, distributed and event-driven architectures; workflow orchestration (e.g., Temporal).
• Identity & Access Management: OAuth 2.0, OIDC, SAML 2.0, SCIM 2.0; enterprise IdPs (Okta, Entra ID, Auth0, Ping); RBAC/ReBAC (Zanzibar, SpiceDB) and multi-tenant authorization.
• Security & Compliance: AuthN/authZ testing, secrets management, tenant isolation, least-privilege access; IDV/KYC and PII/compliance validation.
• Testing Strategy & Ops: Canary/phased rollouts, automated rollback, contract testing (Pact), performance/load testing (Locust, k6, Gatling).
• Engineering & Platform: Ja