SDET - Automation Engineer - AI

Aegistech; New Jersey; Hybrid; USD 120,000 - 180,000; Full time

Job summary

Aegistech in Northern New Jersey is looking for a self-driven Software Development Engineer in Test (SDET II) consultant with hands-on experience testing AI-powered applications, Generative AI solutions, and Agentic AI systems. This role is hybrid onsite 3 days a week in Northern NJ.

Local candidates only/NO RELOCATION. This position requires 8+ years in software quality engineering, strong Python, PyTest, Pandas, and UI automation (Playwright/Selenium), plus API and CI/CD experience.

Qualifications

• 8+ years in software quality engineering, testing, or SDET in Agile environments.
• Strong proficiency in Python, PyTest, Pandas and OO programming.
• 6+ years UI automation experience with Playwright or Selenium.
• 6+ years testing RESTful APIs and microservices.
• Hands-on with CI/CD pipelines and DevOps practices.
• Experience with SQL and NoSQL databases.
• Experience with AWS or other cloud platforms.
• Understanding AI testing concepts including prompt engineering and hallucination detection.
• Knowledge of LLMs, RAG, AI agents, MCP and AI-assisted workflows.
• Ability to develop QA strategies and automation for AI-driven apps.
• Analytical, problem-solving mindset and quality-first culture.

Responsibilities

• Design and execute functional, integration, system, regression, and automation testing.
• Apply Shift-Left testing by involvement in requirements and early test planning.
• Build, maintain, and enhance UI automation frameworks using Python, PyTest, Playwright.
• Perform manual and automated API testing with industry-standard tools.
• Collaborate with developers, product owners, analysts, and DevOps to ensure quality.
• Work with SQL and NoSQL databases for backend validation and data testing.
• Support CI/CD pipelines and ensure tests are integrated into deployments.
• Create test plans, test data strategies, and quality metrics dashboards.
• Manage defect lifecycle and report quality risks clearly.
• Develop AI test datasets, benchmarks, and automated validation approaches.
• Experience with performance and load testing using JMeter or similar tools.
• Knowledge of AI observability, model monitoring, and evaluation platforms.

Skills

Python

PyTest

Pandas

OO Programming

Playwright

Selenium

RESTful APIs

CI/CD

DevOps

SQL

NoSQL

AWS

Prompt engineering

LLMs

AI observability

AI evaluation

Tools

Playwright

Selenium

JMeter

New Relic

Datadog

Splunk

CloudWatch

Job description

Our client in Northern New Jersey is seeking a self-driven Software Development Engineer in Test (SDET II) consultant with experience testing AI-powered applications, Generative AI solutions, and Agentic AI systems. This role is responsible for designing and executing quality engineering activities, developing automation solutions, validating APIs and integrations, supporting CI/CD and release readiness, and promoting a quality-first engineering mindset across traditional and AI-driven platforms.

This position is hybrid onsite 3 days a week in Northern NJ. Local candidates only/NO RELOCATION.

This role is not eligible for employer-sponsored work authorization (e.g., H-1B, OPT, CPT, etc.) or transfer. Candidates must have valid U.S. work authorization.

Key Experience Requirements:

• 8+ years of software quality engineering, test automation, or SDET experience in Agile development environments.
• Strong proficiency in Python, Pytest, Pandas, and object-oriented programming principles.
• 6+ years of experience with UI automation frameworks such as Playwright or Selenium.
• 6+ years of experience testing RESTful APIs and microservices.
• Hands-on experience with CI/CD pipelines and DevOps practices.
• Experience working with SQL and NoSQL databases.
• Experience with AWS or other cloud-based platforms.
• Understanding prompt engineering, prompt validation, hallucination detection, bias testing, guardrail validation, and AI output evaluation.
• Knowledge of LLMs, Retrieval-Augmented Generation (RAG), AI agents, MCP, and AI-assisted workflows.
• Ability to develop QA strategies, test frameworks, and automation solutions for AI-driven applications.
• Analytical, problem-solving mindset with continuous learning and quality-first engineering habits.

Key Responsibilities:

• Design and execute functional, integration, system, regression, and automation testing activities.
• Apply Shift-Left testing practices by participating in requirement reviews and defining test scenarios early in the SDLC.
• Build, maintain, and enhance UI automation frameworks using Python, PyTest, and Playwright.
• Perform manual and automated API testing using industry-standard tools and frameworks.
• Collaborate with developers, product owners, business analysts, and DevOps teams to ensure product quality.
• Work with SQL and NoSQL databases for backend validation and data testing.
• Support CI/CD pipelines and ensure automated tests are integrated into deployment processes.
• Create test plans, test cases, test data strategies, and quality metrics dashboards.
• Manage defect lifecycle activities and provide clear reporting of quality risks.
• Develop AI test datasets, benchmarks, evaluation frameworks, and automated validation approaches.
• Experience with performance and load testing using JMeter or similar tools.
• Knowledge of AI observability, model monitoring, and evaluation platforms.
• Experience with monitoring tools such as New Relic, Datadog, Splunk, or CloudWatch.

Preferred Qualifications:

• Experience testing Generative AI and Agentic AI applications.
• Ability to validate AI outputs for accuracy, reliability, and business relevance.
• Knowledge of AI agent workflows, prompt testing, hallucination detection, and model evaluation.
• Understanding AI safety, guardrails, security, compliance, and responsible AI practices.
• Experience with LangChain, LangGraph, MCP, OpenAI, Anthropic, Azure OpenAI, Kiro or Amazon Bedrock.
• Familiarity with AI governance, risk management, and AI quality metrics.