Senior Test Engineer with Agentic AI

NTT America, Inc.; City of Rome (NY); Hybrid; USD 25,000 - 38,000; Full time

Benefits offered by this job

Private Health Insurance

Coaching School

Remote/Hybrid/Office options

Sponsored certifications

Team events

Job summary

NTT DATA Romania is seeking a Senior Test Engineer to lead the end‑to‑end testing lifecycle for AI solutions, including strategy, test plans, SIT, UAT, and performance coordination. You will manage regression suites, coordinate with stakeholders, and drive high‑quality delivery across testing phases.

You will operate in a matrix setup with remote, hybrid, or office work options, and contribute to education of business users on the test strategy, while leveraging tools like Azure DevOps and

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or equivalent experience.
• AI testing experience is mandatory.
• AI agents orchestration know how is mandatory.
• Test data analysis competency.
• Iterative, Agile, or similar methodology experience.
• Excellent stakeholder communication skills, able to summarise a complicated data set into meaningful progress reports.
• Experience with industry standard test management tools (MS Azure DevOps).
• Proficient at defect management workflows and processes.
• ISEB/ISTQB or equivalent certification in testing.
• Experience working in the Insurance industry.
• A strong understanding of how teams build and maintain low-code or script-assisted automation using tools like Katalon.
• Tracks KPIs and integrates automation and NFT checks into quality governance reporting.
• Understand CI/CD tools such as ADO, GitHub Actions, or GitLab CI/CD.
• Ensure functional test cases, automation packs, and test data align with continuous integration workflows.
• Experience with automation planning, automation-first strategies, or low-code/no-code automation tools.

Responsibilities

• Testing of AI solutions generated in Pega and Azure, and using AIS/Integro for integrations.
• Active participation in test environment preparation discussions.
• Develop end-to-end Test Strategy accounting for downstream regression tests, SIT, UAT etc.
• Develop test strategy and plans for the project.
• Define test objectives, scope and deliverables.
• Coordinate with stakeholders to ensure testing requirements are met.
• Manage the test team and ensure efficient preparation and execution of test cases.
• Firm understanding of all the project level integrations and ensuring the SIT & UAT environment includes all required systems, components, and test data.
• Ensure all prerequisites for SIT & UAT execution are met, and execution proceeds in accordance with the SIT & UAT schedule.
• Define the SIT & UAT Schedule which will support delivery against the defined project outcomes.
• Lead education of the business users/testers on agreed test strategy and approaches.
• Provide support to the business on the creation of robust end-to-end business test scenarios.
• Produce SIT & UAT Schedule in alignment with the Project Plan, whilst ensuring availability and buy-in.
• Co-ordinate SIT & UAT execution activities across all lines of business, as well as business units (i.e. Operations, Claims & Finance).
• Management of defects raised during SIT & UAT execution until resolution, in accordance with the agreed SIT & UAT Management Process.
• Track progress of SIT & UAT execution and produce Daily Status Reports for circulation amongst Project and Business Stakeholders.
• Implementation of SIT & UAT Entry/Exit Quality Gates, including the facilitation of any remedial tasks/actions.
• Chair regular SIT & UAT Progress Meetings with the delegated Business Testers, IT Rep and Business Analyst.
• Produce SIT & UAT Summary Report
• Regular updates to Project Manager on progress to plan
• Escalate risks and issues to Project Manager
• Provide estimations for testing - timelines and resource

Skills

AI testing

AI orchestration

Test data analysis

Agile/Scrum

Stakeholder comms

Azure DevOps

Defect management

ISTQB/ISEB

Insurance industry

Low-code automation

CI/CD tools

Automation planning

Education

Bachelor/Master in CS/Eng

Tools

Azure DevOps

Katalon

Job description

Who we are

As a Senior Test Engineer with Agentic AI, you will lead the end‑to‑end testing lifecycle, including defining the overall test strategy, developing comprehensive test plans, and creating new SIT test cases. Your responsibilities include maintaining and executing SIT regression suites, overseeing business‑driven UAT test cases, and coordinating performance test scripts when required. You will manage the testing RAID log, ensure clear reporting through SIT and UAT test summary reports, and drive high‑quality delivery across all testing phases.

What you'll be doing

• Testing of AI solutions generated in Pega and Azure, and using AIS/Integro for integrations

• Active participation in test environment preparation discussions

• Develop end-to-end Test Strategy accounting for downstream regression tests, SIT, UAT etc

• Develop test strategy and plans for the project

• Define test objectives, scope and deliverables

• Coordinate with stakeholders to ensure testing requirements are met

• Manage the test team and ensure efficient preparation and execution of test cases

• Firm understanding of all the project level integrations and ensuring the SIT & UAT environment includes all required systems, components, and test data

• Ensure all prerequisites for SIT & UAT execution are met, and execution proceeds in accordance with the SIT & UAT schedule

• Define the SIT & UAT Schedule which will support delivery against the defined project outcomes

• Lead education of the business users/testers on agreed test strategy and approaches

• Provide support to the business on the creation of robust end-to-end business test scenarios

• Produce SIT & UAT Schedule in alignment with the Project Plan, whilst ensuring availability and buy-in

• Co-ordinate SIT & UAT execution activities across all lines of business, as well as business units (i.e. Operations, Claims & Finance)

• Management of defects raised during SIT & UAT execution until resolution, in accordance with the agreed SIT & UAT Management Process

• Track progress of SIT & UAT execution and produce Daily Status Reports for circulation amongst Project and Business Stakeholders

• Implementation of SIT & UAT Entry/Exit Quality Gates, including the facilitation of any remedial tasks/actions

• Chair regular SIT & UAT Progress Meetings with the delegated Business Testers, IT Rep and Business Analyst

• Produce SIT & UAT Summary Report

• Regular updates to Project Manager on progress to plan

• Escalate risks and issues to Project Manager

• Provide estimations for testing - timelines and resource

What you'll bring along

• Bachelor’s or Master’s degree in Computer Science, Engineering, or equivalent experience

• AI testing experience is mandatory

• AI agents orchestration know how is mandatory

• Test data analysis competency

• Iterative, Agile, or similar methodology experience

• Excellent stakeholder communication skills, able to summarise a complicated data set into meaningful progress reports

• Experience with industry standard test management tools (MS Azure DevOps)

• Proficient at defect management workflows and processes

• ISEB/ISTQB or equivalent certification in testing

• Experience working in the Insurance industry

• A strong understanding of how teams build and maintain low-code or script-assisted automation using tools like Katalon

• Tracks KPIs and integrates automation and NFT checks into quality governance reporting

• Understand CI/CD tools such as ADO, GitHub Actions, or GitLab CI/CD

• Ensure functional test cases, automation packs, and test data align with continuous integration workflows

• Experience with automation planning, automation-first strategies, or low-code/no-code automation tools

• Ability to validate non-functional aspects with performance Lead (performance baselines, resilience under load, platform constraints)

• Knowledge of Scrum, Agile is essential

• Good speaking capability in English

What’s in it for you

New beginnings can be a challenge. We promise a smooth integration and a supportive mentor

Pick your working style: choose from Remote, Hybrid or Office work opportunities

Early bird or night owl? Our projects have different working hours to suit your needs

Nobody is born an expert. Sharpen your tech skills with our sponsored certifications, trainings and top e-learning platforms

We want you to stay healthy! Enjoy our Private Health Insurance – it’s custom‑made for you

A clear mind is a healthy mind. Attend individual coaching sessions or go one step further by joining our accredited Coaching School

Make the most of our epic parties or themed events – they’re lovingly designed for our people and their families

NTT DATA Romania is an equal opportunity employer and considers all applicants regardless to race, color, religion, citizenship, national origin, ancestry, age, sex, sexual orientation, gender identity, genetic information, physical or mental disability, veteran or marital status, or any other characteristic protected by law. We are committed to creating a diverse and inclusive environment for all employees.

What’s in it for you

• New beginnings can be a challenge. We promise a smooth integration and a supportive mentor

• Pick your working style: choose from Remote, Hybrid or Office work opportunities

• Early bird or night owl? Our projects have different working hours to suit your needs

• Nobody is born an expert. Sharpen your tech skills with our sponsored certifications, trainings and top e-learning platforms

• We want you to stay healthy! Enjoy our Private Health Insurance – it’s custom‑made for you

• A clear mind is a healthy mind. Attend individual coaching sessions or go one step further by joining our accredited Coaching School

• Make the most of our epic parties or themed events – they’re lovingly designed for our people and their families

Make this the place you grow

Your unique talent is what matters. NTT DATA Romania is an equal opportunity employer and considers all applicants regardless to race, color, religion, citizenship, national origin, ethnicity, age, gender, sexual orientation, gender identity, genetic information, physical or mental disability, veteran or marital status, or any other characteristic.

#LI-GS1