Senior SDET for AI Agents | Testing, CI/CD & Equity

Nexxa.ai; Sunnyvale (CA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Equity

Job summary

Nexxa.ai is seeking a Senior SDET/QA engineer in Sunnyvale, CA to build and maintain automated evaluation frameworks for AI agents. You will develop test suites, simulators, and CI/CD quality gates to ensure robust agent performance and safety.

The role requires strong Python skills, experience with distributed systems, APIs, and LLM-based products, and the ability to work across teams on test infrastructure, evaluation methods, and failure analysis.

Qualifications

• Requires at least five years of SDET, QA, or software engineering experience focused on testing.
• Strong programming skills (Python preferred) and experience building test infrastructure.
• Experience testing distributed systems, APIs, and LLM-based products.
• Proficiency in manual and automated testing, CI/CD and containerized environments.
• Experience with evaluation methods, dataset management, and LLM agent failure modes.

Responsibilities

• Build and maintain automated evaluation frameworks, test suites, simulators, and CI/CD quality gates for AI agents.
• Define quality metrics and release criteria for meaningful deployments.
• Manage evaluation datasets and investigate failures to improve agent reliability and safety.
• Collaborate with engineering and deployment teams to enhance testing capabilities across platforms.
• Participate in hands-on exploratory, regression, and adversarial testing efforts.

Skills

Python

Test Automation

Exploratory Testing

LLM Evaluation

AI Agent Testing

CI/CD

API Testing

Integration Testing

End-to-End Testing

Test Framework Development

Dataset Management

Red Teaming

Performance Testing

Tools

Docker

Kubernetes

Observability and Tracing

Job description

Nexxa.ai is seeking a Senior SDET/QA engineer in Sunnyvale, CA to build and maintain automated evaluation frameworks for AI agents. You will develop test suites, simulators, and CI/CD quality gates to ensure robust agent performance and safety.

The role requires strong Python skills, experience with distributed systems, APIs, and LLM-based products, and the ability to work across teams on test infrastructure, evaluation methods, and failure analysis.