Senior SDET for AI Agents & Industrial Automation

Nexxa; San Francisco (CA); On-site; USD 120,000 - 190,000; Full time

Job summary

Nexxa is seeking a Senior / Staff SDET to own quality for AI agent systems in industrial settings. You will build evaluation harnesses, simulation environments, and CI/CD quality gates, while also probing agent behavior manually.

Collaboration with ML, backend, and Forward Deployed Engineers defines what “good” looks like for high‑stakes deployments. You will design tests that cover planning, tool calls, multi‑turn reasoning, and failure modes, while creating dashboards to track ground‑truth

Qualifications

• 5+ years as an SDET, QA engineer, or software engineer with a testing focus.
• Experience in manual and automated testing, including exploratory testing and bug reporting.
• Strong Python programming with clean, scalable code for tools and pipelines.
• Experience building test frameworks and CI/CD infrastructure from scratch.
• Hands‑on with CI/CD systems and containerized test environments (Docker/Kubernetes).
• Experience testing APIs, microservices, and LLM‑based products.

Responsibilities

• Design, build, and maintain test frameworks, evaluation harnesses, and regression suites for LLM‑based agents.
• Create automated pipelines to run evals on model changes and integrate into CI/CD.
• Develop simulators and sandbox environments for industrial systems and tool APIs.
• Implement API, contract, integration, and E2E tests for services and orchestration layers.
• Develop performance, load, and resilience tests for agent workflows.
• Perform hands‑on exploratory testing and review traces to judge reasoning quality.
• Lead manual regression and release verification in staging and customer‑like environments.
• Contribute to security testing via red‑teaming and cross‑functional collaboration.

Skills

Python

Exploratory testing

Test strategy

CI/CD

Tools

Docker

Kubernetes

GitHub Actions

GitLab CI

Jenkins

LangSmith

Job description

Nexxa is seeking a Senior / Staff SDET to own quality for AI agent systems in industrial settings. You will build evaluation harnesses, simulation environments, and CI/CD quality gates, while also probing agent behavior manually.

Collaboration with ML, backend, and Forward Deployed Engineers defines what “good” looks like for high‑stakes deployments. You will design tests that cover planning, tool calls, multi‑turn reasoning, and failure modes, while creating dashboards to track ground‑truth