Senior AI QA Engineer - Eval & Production

Curative HR; Austin (TX); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Curative Health Plan (100% employer‑s­

$0 copays and $0 deductibles

Preventive and primary care built in

Mental health support

One-on-one care navigation

Chronic condition programs

Maternity and family planning support

Telehealth access 24/7/365

Pharmacy benefits

Dental and vision coverage

Life and disability coverage

Generous PTO and 11 holidays

401K for full-time

Job summary

Curative is hiring a Senior Software Test Engineer to ensure fast delivery with top-tier quality. You will work with AI agents and eval suites to measure agent behavior, task success, and regression coverage.

You’ll test how real users interact with the product, from end-to-end flows to edge cases, and collaborate with business teams to close gaps between what was asked for and what ships. Expect to own the eval infrastructure, perform hands-on exploratory testing, and shape quality across

Qualifications

• 5+ years in software quality, testing, or engineering with real range.
• Fluent with LLM evaluation and programmatic graders.
• Hands-on experience with AI agents and failure modes.
• Strong debugging and root-cause analysis skills.
• Clear written communication with business stakeholders.

Responsibilities

• Own the eval platform, including datasets, graders, and CI integration.
• Perform hands-on exploratory testing across end-to-end workflows.
• Validate we built the right thing by aligning with business stakeholders.
• Develop domain expertise within Curative's business processes.
• Investigate production signals, reproduce incidents, and propose fixes.

Skills

Software QA

Exploratory testing

LLM evaluation

AI agents

Debugging

Python

TypeScript

Stakeholder communication

Ambiguity tolerance

Observability

Tools

Python

TypeScript

LangSmith

Claude Code

Job description

Curative is hiring a Senior Software Test Engineer to ensure fast delivery with top-tier quality. You will work with AI agents and eval suites to measure agent behavior, task success, and regression coverage.

You’ll test how real users interact with the product, from end-to-end flows to edge cases, and collaborate with business teams to close gaps between what was asked for and what ships. Expect to own the eval infrastructure, perform hands-on exploratory testing, and shape quality across