Quality Engineering (AI & LLM Systems) / Test Automation / Evaluation & Validation Engineering
NNIT
Get more replies from employers
Send a job-specific resume in minutes.
Benefits offered by this job
Job summary
NNIT's AI Center of Excellence in the Philippines seeks an Evaluation & Validation Engineer to design evaluation frameworks, validate enterprise AI, and ensure production readiness across global clients.
You will collaborate with AI engineers, solution architects, consultants, and business stakeholders to measure AI quality, detect model drift, and support responsible AI governance within an Agile, enterprise IT context.
Qualifications
- At least 4 years of Quality Engineering, Test Automation, Validation, AI Engineering, or Software Engineering experience.
- Experience designing automated testing or evaluation frameworks.
- Experience validating enterprise software or AI applications.
- Experience working within Agile delivery environments.
Responsibilities
- Design AI evaluation frameworks for enterprise applications and define quality metrics, benchmarks, and acceptance criteria.
- Build scalable validation processes with automated evaluation and regression testing.
- Collaborate with AI engineers, architects, consultants, and product owners to meet business, operational, and regulatory expectations.
Skills
Tools
Job description
As an Evaluation & Validation Engineer, you'll evaluate, validate, and improve enterprise AI solutions to ensure they are accurate, reliable, secure, and ready for production. Working within NNIT's AI Center of Excellence, you'll establish evaluation frameworks, quality standards, and validation processes that build confidence in AI systems deployed across global organizations.
You'll collaborate with AI engineers, solution architects, consultants, product owners, and business stakeholders across Europe, Asia, and other regions to evaluate AI solutions throughout their lifecycle—from solution design and development to deployment, monitoring, and continuous improvement.
Success in this role requires analytical thinking, structured problem solving, and a strong understanding of AI evaluation methodologies, software quality engineering, and enterprise delivery.
Develop evaluation methodologies that measure AI quality, reliability, and business readiness.
- Design evaluation frameworks for enterprise AI applications.
- Define quality metrics, benchmarks, and acceptance criteria.
- Evaluate LLMs, AI agents, and Retrieval-Augmented Generation (RAG) solutions.
- Develop datasets and test cases for automated AI evaluations.
- Support Responsible AI and AI governance initiatives.
Build scalable validation processes that improve AI quality and production readiness.
- Develop automated evaluation and regression testing frameworks.
- Detect hallucinations, model drift, and quality degradation.
- Improve AI reliability through continuous benchmarking and testing.
- Build reusable validation assets, test suites, and engineering standards.
- Recommend improvements to AI quality and engineering practices.
Collaborate with technical and business teams to ensure AI solutions meet business, operational, and regulatory expectations.
- Work closely with AI engineers, architects, consultants, and product owners.
- Translate business expectations into measurable evaluation criteria.
- Participate in solution reviews and production readiness assessments.
- Communicate evaluation results, quality findings, and technical risks.
- Support continuous improvement across the AI Center of Excellence.
Success in this role means you:
- Improve the quality, accuracy, and reliability of enterprise AI solutions.
- Reduce production issues through proactive evaluation and validation.
- Establish reusable evaluation frameworks and engineering standards.
- Increase stakeholder confidence in AI solutions.
- Support the continued growth and technical maturity of NNIT's AI Center of Excellence.
- At least 4 years of Quality Engineering, Test Automation, Validation, AI Engineering, or Software Engineering experience.
- Experience designing automated testing or evaluation frameworks.
- Experience validating enterprise software or AI applications.
- Experience working within Agile delivery environments.
- Experience working within consulting, enterprise IT, or global delivery organizations is highly desirable.
- OpenAI Evals
- DeepEval
- LangSmith Evaluation
- RAGAS
- TruLens
- AI Quality Metrics
- LangChain
- LangGraph
- AI Agents
- Retrieval-Augmented Generation (RAG)
- Python
- Git
- CI/CD
- REST APIs
- Test Automation Frameworks
Many of our customers operate in highly regulated industries where quality, traceability, compliance, and governance are essential for successful AI adoption.
Experience supporting one or more of the following industries is advantageous:
- Life Sciences
- Pharmaceuticals
- Biotechnology
- Medical Devices
- Insurance
- Manufacturing
Exposure to Responsible AI, Computer System Validation (CSV), GxP, FDA, EMA, HIPAA, GDPR, ISO 27001, SOC 2, or similar governance and compliance frameworks is beneficial but not required.
Joining the AI Center of Excellence gives you the opportunity to build the data foundation that powers enterprise AI and digital transformation for global organizations while expanding your expertise across cloud data platforms, Artificial Intelligence, enterprise consulting, and global delivery.
- Build Solutions That Matter
- Work Across Borders
- Grow Your Expertise
- Shape the Future
NNIT is a fast-growing IT company with ambitious goals and 1,600 employees. At NNIT, you can make your mark on some of the most challenging projects in the industry, on our journey forward as a company, and on your own career. You’ll work in agile teams with other passionate people in a culture with room for independent thinking and individual responsibility.
We supply IT services to large enterprises that demand high levels of quality and security, and we take on the responsibility for our clients' IT as if it were our own. We live our values every day, so you should be able to see yourself in them. We are: Conscience driven. Value adding. Open and honest.
Our everyday work is characterized by an open and honest atmosphere, and we put emphasis on creating a challenging yet fun working environment. We pay significant attention to job satisfaction, and personal and professional development.
On top of the statutory benefits stipulated by the local law, NNIT DLSP employees enjoy the following benefits upon joining:
- Competitive Compensation
- 13th Month Pay
- Performance Bonus (discretionary)
- Rice Subsidy
- Clothing and Laundry Allowance
- Medical Allowance (Employee and Dependent)
- HMO (Employee and Dependent/s)
- Life Insurance
- Vacation Leaves and Sick Leaves
- Emergency Loan
- Year-End Gift
- Wedding Gift
- First Baby Gift
- Breakfast once a week
- Fruits, Snacks / Well-stocked pantry
- Flexible Working Hours
- Hybrid set up
Related positions:
- AI Validation Engineer
- Quality Assurance Engineer
- Test Automation Engineer
- Validation Engineer
- Software Quality Engineer
1. By applying to this job ad, you acknowledge and agree for your information to be collected by NNIT. Rest assured, any information you share will only be used for recruitment/hiring purposes.
2. This position is available only to those with current authorization to live and work in the Philippines.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
