AI Evaluation Infrastructure & Production Readiness Lead

Ports North; Baltimore (MD); On-site; USD 138,000 - 207,000; Part time

Job summary

Korn Ferry, in partnership with a financial services client, seeks an AI Evaluation Infrastructure & Production Readiness Lead for a contract role. You will shape evaluation frameworks, establish go/no-go criteria, and build scalable testing infra across governance, risk, and compliance teams.

You will lead cross-functional efforts to ensure AI systems are accurate, safe, and production-ready, with emphasis on LLM evaluation, bias testing, and regulatory alignment for enterprise deployment.

Qualifications

• Bachelor's Degree in CS, Engineering, IT, Data Science or related field.
• 7+ years of professional work experience in AI/ML engineering, software quality engineering, data science, or ML.
• Experience building/testing enterprise LLM/ML evaluation frameworks.

Responsibilities

• Lead development of enterprise AI evaluation and production readiness framework with standardized testing, governance, and go/no-go criteria.
• Design scalable AI evaluation infrastructure including evaluation harnesses, data sets, regression testing and bias/fairness validation.
• Collaborate with AI Engineering, Risk, Compliance, Legal, and Model Governance to translate requirements into repeatable AI testing methods and reporting.

Skills

LLM evaluation

Regression testing

Statistical analysis

Production validation

AI testing methodologies

Cross-functional collaboration

Education

Bachelor's Degree in CS/Engineering/Data Science

Job description

We have partnered with our client in their search for an AI Evaluation Infrastructure & Production Readiness Lead (Contract position).

Responsibilities

• Lead the development of client's enterprise AI evaluation and production readiness framework, establishing standardized testing, validation, governance, and go/no-go criteria that ensure AI systems are accurate, safe, compliant, and ready for enterprise deployment.
• Design and build scalable AI evaluation infrastructure, including evaluation harnesses, golden datasets, regression testing, hallucination and grounding validation, bias/fairness testing, adversarial testing, drift detection, automated monitoring, and production readiness controls.
• Partner with AI Engineering, Risk, Compliance, Legal, and Model Governance teams to translate regulatory and business requirements into repeatable AI testing methodologies, vendor evaluation standards, and evidence-based reporting for enterprise AI governance.

Skills Required

• 7+ years of experience in AI/ML Engineering, Software Quality Engineering, Data Science, or Machine Learning, with expertise in LLM evaluation, regression testing, statistical analysis, and production validation.
• Hands-on experience building and testing enterprise LLM/ML evaluation frameworks, including hallucination detection, grounding validation, bias/fairness testing, experimental design, AI safety, policy adherence, and production readiness.
• Strong expertise in AI testing methodologies, statistical experimentation, evaluation metrics, and acceptance criteria, with experience partnering across AI Engineering, Risk, Compliance, and Legal to implement enterprise AI governance and quality controls.

Education & Work Experience

• Bachelor's Degree (pref. Computer Science, Engineering, IT, Data Science, or related fields)
• 7+ years of Professional Work Experience

Title: AI Evaluation Infrastructure & Production Readiness Lead

Location: Remote

Client Industry: Financial Services

Compensation: $100+/hr

About Korn Ferry

Korn Ferry unleashes potential in people, teams, and organizations. We work with our clients to design optimal organization structures, roles, and responsibilities. We help them hire the right people and advise them on how to reward and motivate their workforce while developing professionals as they navigate and advance their careers. To learn more, please visit Korn Ferry at www.Kornferry.com