Director, AI Reliability & Performance

Novartis; Dublin; On-site; EUR 102,000 - 190,000; Full time

Benefits offered by this job

Hybrid work options

Parental leave 14 weeks

Insurance & retirement plans

Job summary

Novartis in Dublin, Ireland seeks a Director for AI Systems Reliability, Testing & Performance to lead how AI systems are evaluated, tested, validated and monitored across Development. You will define evidence, benchmarks and production-readiness criteria to ensure AI reliability and fitness-for-use.

The role requires senior technical leadership, collaboration with governance, product and platform teams, and a deep focus on measurable AI performance in regulated environments.

Qualifications

• Work Experience 10+ years in AI, machine learning, software engineering, quality, risk, validation, or technology governance.
• Experience deploying or overseeing business-critical AI systems.
• Experience establishing governance, controls, quality standards, or operational frameworks.
• Experience defining evaluation approaches and performance metrics.

Responsibilities

• Define evaluation methodologies for AI systems, models, agents, digital twins, and simulation environments across Development.
• Establish benchmark suites, evaluation datasets, and testing harnesses used across AI initiatives.
• Define objective measures for quality, reliability, robustness, grounding, agent effectiveness, and task success.
• Ensure evaluation approaches remain scientifically rigorous, reproducible, and comparable across AI systems.
• Build a common evidence framework for assessing AI capabilities, limitations, and fitness-for-use.
• AI Testing & Validation: Establish approaches for hallucination testing, failure-mode analysis, robustness testing, and behavioral validation.
• Define validation methodologies for agentic systems, digital twins, simulation environments, and human-in-the-loop workflows.
• Develop production readiness criteria for AI systems operating in Development environments.

Skills

AI leadership

Reliability engineering

Validation & testing

Governance & risk

Stakeholder collaboration

Education

10+ years in AI/ML/engineering/quality/governance

Job description

Novartis in Dublin, Ireland seeks a Director for AI Systems Reliability, Testing & Performance to lead how AI systems are evaluated, tested, validated and monitored across Development. You will define evidence, benchmarks and production-readiness criteria to ensure AI reliability and fitness-for-use.

The role requires senior technical leadership, collaboration with governance, product and platform teams, and a deep focus on measurable AI performance in regulated environments.