Senior AI Evaluation Engineer - CI/CD & Platform

Workday, Inc. (old)

Ireland

On-site

EUR 110,000 - 150,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Workday, Inc. is seeking a Senior AI Evaluation Engineer to own the continuous evaluation architecture and release gating for our enterprise AI Skills platform.

You will build MLOps/CI-CD deployment pipelines and automated evaluation harnesses to ensure production AI skills execute safely and remain schema‑compliant across live tenant environments. You will bridge AI skill development with production platform engineering, leveraging modern evaluation frameworks to assess trajectories, test

Qualifications

  • 1+ years of hands-on experience evaluating or building Generative AI applications.
  • Experience with evaluation frameworks and observability tools.
  • 4+ years of professional software engineering in Python—CI/CD OR AI evaluation, plus familiarity with the other.
  • Schema validation libraries (JSON Schema, Pydantic) and test automation.
  • Strong engineering discipline in test coverage, regression prevention, and documentation.

Responsibilities

  • Own the continuous evaluation architecture and release gates for the enterprise AI Skills platform.
  • Build MLOps/CI-CD deployment pipelines and automated eval harnesses.
  • Bridge AI skill development with production platform engineering and governance.
  • Architect and maintain CI/CD build pipelines, versioning, and runner environments for automated gates.
  • Ingest production traces into benchmark datasets to close test coverage gaps.

Skills

Generative AI evaluation
CI/CD pipelines
Python production systems
Technical communication

Education

Bachelor's degree in CS/Engineering

Tools

LangSmith
Braintrust
DeepEval
Promptfoo
Phoenix

Job description

Workday, Inc. is seeking a Senior AI Evaluation Engineer to own the continuous evaluation architecture and release gating for our enterprise AI Skills platform.

You will build MLOps/CI-CD deployment pipelines and automated evaluation harnesses to ensure production AI skills execute safely and remain schema‑compliant across live tenant environments. You will bridge AI skill development with production platform engineering, leveraging modern evaluation frameworks to assess trajectories, test

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Evaluation Engineer — CI/CD & Platform
Senior AI Evaluation Engineer — CI/CD & Platform

Workday, Inc. (old) • Dublin

Hybrid
EUR 96,000 - 144,000
Stock grants
Flexible work options
Senior AI Evaluation Engineer (CICD & Platform)
Senior AI Evaluation Engineer (CICD & Platform)

Workday, Inc. (old) • Ireland

On-site
EUR 110,000 - 150,000
Senior Applied AI Engineer - AI Skill Authoring
Senior Applied AI Engineer - AI Skill Authoring

Workday, Inc. (old) • Ireland

On-site
EUR 90,000 - 150,000
Senior Applied AI Engineer — Enterprise AI Skills & Tooling
Senior Applied AI Engineer — Enterprise AI Skills & Tooling

Workday, Inc. (old) • Dublin

Hybrid
EUR 96,000 - 144,000
Flex Work policy
Applied AI Engineer, Enterprise Skills Platform
Applied AI Engineer, Enterprise Skills Platform

Workday, Inc. (old) • Ireland

On-site
EUR 90,000 - 120,000
Enterprise AI Engineer: AI Skills & API Orchestration
Enterprise AI Engineer: AI Skills & API Orchestration

Workday, Inc. (old) • Dublin

On-site
EUR 72,000 - 108,000
Senior DevEx Engineer: AI Skills Platform & IDE Tools
Senior DevEx Engineer: AI Skills Platform & IDE Tools

Workday, Inc. (old) • Dublin

On-site
EUR 85,000 - 130,000
Applied AI Engineer
Applied AI Engineer

Workday, Inc. (old) • Ireland

On-site
EUR 90,000 - 120,000
Senior Applied AI Engineer
Senior Applied AI Engineer

Workday, Inc. (old) • Ireland

On-site
EUR 90,000 - 150,000
Senior DevEx Engineer - AI Platform & MCP Tools
Senior DevEx Engineer - AI Platform & MCP Tools

Workday, Inc. (old) • Ireland

Hybrid
EUR 86,000 - 130,000
Top locations