Test Lead (AI Validation & Integration)

Digital Forms; United States; Remote; USD 110,000 - 160,000; Full time

Job summary

Digital Forms seeks an experienced Test Lead to define and drive the Testing & AI Validation Strategy for a regulated AI+Salesforce integration. You will own QA strategy, plan, and hands-on testing across functional, non-functional, regression, and UAT to ensure safe, reliable outputs.

You will coordinate with client teams, align with GDPR/HITRUST standards, manage test environments and data, and integrate tests into CI/CD with clear QA metrics.

Qualifications

• Fluent English required.
• QA strategy experience for complex solutions.
• End-to-end testing for integrated systems.
• AI/ML/LLM testing experience.
• Test data management across Dev/Test/Prod.
• Traceability from requirements to tests to defects to releases.
• Experience testing NFRs: performance, reliability, security basics.
• Strong privacy/data handling (PII/PHI minimization, masking).
• Hands-on automation (Python/Java/JS with pytest/JUnit).
• QA metrics tracking and reporting.
• Jira proficiency and strong communication skills.
• Full-time availability with overlap 14:00-18:00 US time.

Responsibilities

• Define and own the QA and AI Validation strategy for a regulated AI+Salesforce project.
• Plan and execute functional, regression, exploratory, performance, and load tests.
• Coordinate end-to-end validation with client teams for Salesforce integration.
• Integrate testing activities into the CI/CD pipeline and manage test data across environments.
• Establish traceability from requirements to test cases to defects to releases.

Skills

QA strategy

End-to-end testing

AI/ML/LLM testing

Test data management

Traceability

NFR testing

Privacy & data handling

Automation (Python/Java/JS)

QA metrics

Jira proficiency

Communication

Overlap with USA client hours

Education

ISTQB certification

Tools

Jira

CI/CD tooling

AWS

Job description

We are looking for an experienced Test Lead to drive the Quality and Validation strategy for a high-impact solution, in a highly regulated industry, involving AI components and Salesforce integration. In this role, you will own both definition and delivery of the Testing & AI Validation Strategy. You will be responsible for defining how solution outputs are verified for correctness, reliability, and safe operation, bridging the gap between newly developed AI Agents components and the client’s Salesforce environment.

What you’ll do

• Strategy & Planning: Own the QA Strategy: Design, plan and dDevelop the comprehensive Testing and AI Validation Strategy, covering FunctionalUnit, E2E, Integration, Regression, and User Acceptance Testing (UAT).
• Address NFR and align against standards (e.g. GDPR, HIPAA, HITRUST, etc.).
• AI Validation Logic: Define validation plans for AI components, specifically determining what constitutes "correct" output, setting success thresholds, and establishing evaluation methods (including handling hallucinations and grounding rules).
• Environment & Data: Create detailed plans for Test Environments (Dev/Test/Prod) and define the Test Data Strategy to ensure sufficient coverage for standard scenarios, edge cases, and ingestion failures.
• Execution & Reporting: Hands-on Testing: Execute functional, regression, exploratory, performance, and load tests.
• Integration Coordination: Coordinate "Joint End-to-End Validation" efforts, collaborating with the client team to manage testing of Salesforce integration.
• Pipeline Integration: Integrate testing activities into the CI/CD pipeline for continuous quality assurance.
• Process Management: Manage Jira test case structures aligned to user stories and acceptance criteria.
• Metrics: Provide regular reporting, tracking key QA metrics such as defect ratios.

About the project, in short. Is it a new project or maintenance? This is a new project for a client that we have already established collaboration with during other projects. We are finalizing the Discovery Phase for this project and moving into Delivery. We are developing a data validation and automation engine that processes structured and unstructured inputs against a complex logic framework. The system utilizes AI models to interpret user-submitted data, identifying discrepancies and missing information before mapping the results to a structured output format. Key components include automated ingestion, rule-based validation logic, and integration with existing enterprise architecture in a highly regulated industry.

Technical Requirements (must-have)

• Fluent English
• QA Strategy: Experience defining end-to-end testing strategies for complex solutions
• End-to-End Testing: Experience in designing and executing E2E tests for complex, integrated systems using any modern testing technology
• AI/ML/LLM Testing Experience: Proven ability to validate AI models
• Test Data Management: Experience strategizing and managing test data across multiple environments (Dev/Test/Prod)
• Traceability mindset: linking requirements → test cases → evidence → defects → release decisions.
• Experience testing NFRs: performance, reliability, auditability, security basics.
• Strong understanding of privacy and sensitive data handling in testing (PII/PHI minimization, masking/anonymization)
• Hands-on automation experience (at least one strong stack): e.g., Python/Java/JS + frameworks (pytest/JUnit/etc.)
• QA Metrics: Ability to track and report on specific QA metrics, such as defect ratios
• Good communication skills (written and verbal)
• Jira Proficiency
• Full time availability, with overlap during 14:00-18:00 - to ensure seamless collaboration with USA based client

Additional Requirements (nice-to-have)

• Familiar with AWS platform
• ISTQB or similar certifications
• Previous experience in fast-paced consulting or client-facing projects

Highlights

• Lead QA and AI validation for a regulated AI + Salesforce project, owning strategy and hands-on testing across E2E, integration and NFR with CI/CD and close client collaboration.