ML Engineer, Retrieval & Grounded Generation

redcellpartners; United States; Remote; USD 110,000 - 160,000; Full time

Job summary

Red Cell Partners is seeking an experienced QA Engineer to own quality for an AI-enabled decision-support system at DEFCON AI. You will lead the test strategy, automation, and evidence that requirements are met, reporting to the Product Lead with daily collaboration with engineering and platform teams.

You will build scalable automated tests, validate performance under expected load, and ensure release governance with clear acceptance criteria and traceable evidence for every deployment.

Qualifications

• 5+ years in software quality engineering or test automation.
• Strong coding ability for test automation (Python or JS/TS).

Responsibilities

• Own the system test plan covering functional, integration, performance and load testing.
• Translate product acceptance criteria into executable tests.
• Review acceptance criteria with product before development; flag ambiguities.
• Hold teams accountable to acceptance criteria and define required evidence.
• Work daily with engineering; test what is built and report findings.
• Plan and coordinate user acceptance testing with the customer.
• Build and maintain automated test suites and CI/CD gates.
• Establish automated coverage standards and report against them.
• Create performance/load tests to validate under expected load.
• Maintain regression coverage across releases.
• Create labeled evaluation sets with SMEs.
• Design and run audits to verify automated decisions.
• Measure misses vs hits and report trends.
• Document calculation methods and evidence for releases.
• Produce evidence independently of model builders.

Skills

QA ownership

Test automation

CI/CD

Python

JavaScript/TypeScript

Education

Bachelor's degree (or equivalent)

Tools

Playwright

Cypress

Selenium

pytest

Job description

About Us

Red Cell Partners is an incubation firm building and investing in rapidly scalable technology‑led companies that are bringing revolutionary advancements to market in three distinct practice areas: healthcare, cyber, and national security. United by a shared sense of duty and deep belief in the power of innovation, Red Cell is developing powerful tools and solutions to address our Nation's most pressing problems.

ABOUT DEFCON AI

RESILIENCE IN THE FACE OF DISRUPTION. DEFCON AI is an insights company that leverages artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems.In today's dynamically changing world, DEFCON AI's technology aligns outcomes with operational goals, better decision making, and empowers customers to anticipate assess, and mitigate the impacts of disruptions.

About the Role

As QA Engineer you will own quality for an AI-enabled decision‑support system: the test plan, the automated coverage, and the evidence that what was delivered is what was specified. Product defines the acceptance criteria. You build the executable version of it , and you hold engineering, platform, and everyone else delivering the work accountable to it.

This role sits on the product side of the organization, not the engineering side. You will report through the Product Lead and take direction from product, while working with engineering and platform every day. A tester embedded in engineering tends to verify what was built, because the implementation is what they see. This role verifies what was specified. A test suite written from the code will pass while still missing what the customer asked for.

That reporting line does not lower the engineering bar. You will write code and build real automation rather than managing test cases, and you will need enough fluency in the application code to test it well. As the first dedicated quality engineering hire you are establishing the approach rather than inheriting one, so we want someone with a clear point of view on how testing should work. We build with current tooling and expect the same, including AI assistance in our own engineering practice.

This is a fully remote role with occasional travel (up to 25%) to DEFCON AI HQ, customer sites, and vendor facilities as required.

Key Responsibilities

Test Strategy and Acceptance Criteria

• Own the system test plan covering functional, integration, performance, and load testing, and maintain it as a formal deliverable
• Translate the acceptance criteria set by product into executable tests
• Review acceptance criteria with product before development begins, and identify requirements that are ambiguous or not verifiable as written
• Hold engineering, platform, and partner teams accountable to the acceptance criteria, and define what evidence must exist before a change is considered complete
• Work alongside the engineering team day to day, close enough to test their work well and independent enough to report what the testing shows
• Plan and coordinate user acceptance testing with the customer, and track findings through to resolution

Test Automation and Coverage

• Build and maintain automated test suites, and integrate them into the CI/CD pipeline as release gates
• Establish and hold an automated coverage standard, and report against it
• Build performance and load testing that validates the system under its expected user load
• Maintain regression coverage across releases

Measurement and Evidence

• Build and maintain labeled evaluation sets, working with subject‑matter experts whose time is limited
• Design and run sampled audits confirming that automated decisions were correct
• Measure and report what the system misses alongside what it gets right
• Measure how often users override the system, and report the trend
• Document how each committed measure is computed, on what data, and what evidence accompanies a release
• Produce this evidence independently of the people building the models

Release Governance

• Maintain the version inventory across builds, models, configurations, and thresholds
• Produce test reports, evaluation records, and rollback criteria for each release
• Work with platform engineering to confirm that test environments match production closely enough for results to be meaningful
• Hold the gate: no release ships without the evidence its acceptance requires

Required Qualifications

• 5+ years in software quality engineering, test automation, or software engineering with substantial dedicated test ownership
• A clear point of view on how testing should be done , and the judgment to know which practices are worth holding to. You will be setting the standard here rather than following one
• Strong coding ability in a language and framework used for test automation , for example Python with pytest, or JavaScript or TypeScript with Playwright, Cypress, or Selenium. Python is what most of this stack is written in. You should also be ab