Director, AI Reliability & Performance Engineering

Novartis Ireland; Dublin; On-site; EUR 102,000 - 190,000; Full time

Benefits offered by this job

Insurance plans

Retirement plans

Wellbeing resources

Global recognition programs

Flexible and hybrid working options

14 weeks parental leave

Job summary

Novartis is seeking a Director of AI Systems Reliability, Testing & Performance in Dublin. The role leads the evaluation, validation, benchmarking, monitoring, and continuous improvement of AI systems across Novartis Development, ensuring reliability, security, and fitness-for-use throughout their lifecycle.

The successful candidate will provide technical leadership in evaluation science, reliability engineering, validation, adversarial testing, and performance assessment, guiding governance,

Qualifications

• Minimum 10+ years in AI, ML, software engineering, quality, risk, validation, or technology governance.
• Experience deploying or overseeing business-critical AI systems.
• Experience establishing governance, controls, quality standards, or operational frameworks.
• Experience defining evaluation approaches and performance metrics.
• Experience managing operational risk in regulated environments.

Responsibilities

• Define evaluation methodologies for AI systems, models, agents, digital twins and simulation environments.
• Establish benchmark suites, evaluation datasets and testing harnesses across AI initiatives.
• Define objective measures for quality, reliability, robustness, grounding and task success.
• Ensure evaluation approaches are scientifically rigorous, reproducible and comparable.
• Build a common evidence framework for assessing AI capabilities, limitations and fitness-for-use.
• Establish approaches for hallucination testing, failure-mode analysis and robustness testing.
• Develop production readiness criteria for AI systems operating in Development environments.
• Support technical validation activities required for regulated AI systems.
• Ensure AI systems are tested under realistic operating conditions with objective validation evidence.
• Define monitoring and performance assessment frameworks across Development AI systems.

Skills

AI/ML

Software Eng

Quality

Risk Governance

Validation

Tech Governance

Job description

Novartis is seeking a Director of AI Systems Reliability, Testing & Performance in Dublin. The role leads the evaluation, validation, benchmarking, monitoring, and continuous improvement of AI systems across Novartis Development, ensuring reliability, security, and fitness-for-use throughout their lifecycle.

The successful candidate will provide technical leadership in evaluation science, reliability engineering, validation, adversarial testing, and performance assessment, guiding governance,