Head of AI Evaluation Engineering

Mundelein Elementary School District 75

Chicago, Northern (IL, KY)

Hybrid

USD 148,000 - 240,000

Full time

14 days+
Application generator

An application made for this job — a tailored resume and cover letter that speak straight to the posting.

Get past ATS filters

Benefits offered by this job

Medical, dental, and vision benefits
401(k) savings plans
Health Savings Account (HSA)
Flexible Spending Accounts (FSAs)
Career Development
Incentive bonus

Job summary

Caterpillar Inc. is seeking an AI Engineering leader within the Cat Digital AI Engineering team to drive evaluation and validation of cutting-edge generative AI solutions, intelligent agents, and digital assistants.

You will shape product development and guide engineering practices across hybrid cloud and embedded environments, focusing on scalable, reliable AI systems. Responsibilities include providing clear technical direction, overseeing team performance, and ensuring quality across all AI

Qualifications

  • Proven experience leading software quality, test automation, validation, and AI evaluation teams.

Responsibilities

  • Provide strong technical direction to align the team with company goals for evaluating advanced AI projects.
  • Oversee performance of individuals and the team, identifying training and development needs.
  • Own the quality of AI engineering products and ensure robustness and high standards.
  • Establish and supervise engineering best practices to maintain development excellence.

Skills

AI leadership
GenAI architecture
CI/CD
AI evaluation
Team coaching
Observability

Education

CS/AI-related advanced degree

Tools

Azure
AWS
SageMaker
LangChain
LangGraph
Langfuse
Arize

Job description

Caterpillar Inc. is seeking an AI Engineering leader within the Cat Digital AI Engineering team to drive evaluation and validation of cutting-edge generative AI solutions, intelligent agents, and digital assistants.

You will shape product development and guide engineering practices across hybrid cloud and embedded environments, focusing on scalable, reliable AI systems. Responsibilities include providing clear technical direction, overseeing team performance, and ensuring quality across all AI

Get your free, confidential resume review.
or drag and drop your file here.
Similar jobs

Similar jobs worth comparing

AI Evaluation Engineering Lead
AI Evaluation Engineering Lead

Caterpillar Inc • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
HSA
+3
GenAI Evaluation Engineering Lead
GenAI Evaluation Engineering Lead

Caterpillar Brazil • Chicago (IL), Northern (KY)

Hybrid
USD 148,000 - 240,000
GenAI Evaluation Engineering Lead
GenAI Evaluation Engineering Lead

Caterpillar, Inc. • Westminster (CO)

On-site
USD 148,000 - 240,000
Medical benefits
Paid time off
401(k) plans
+8
AI Evaluation Engineering Lead
AI Evaluation Engineering Lead

Caterpillar Inc. • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
Dental and vision
401(k)
+8
AI Evaluation Engineering Lead
AI Evaluation Engineering Lead

Caterpillar Financial Services Corporation • Broomfield (CO)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
PTO and holidays
+2
AI Engineering Manager — Cloud-Native & AI Tools Leader
AI Engineering Manager — Cloud-Native & AI Tools Leader

Caterpillar Financial Services Corporation • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical, dental, and vision benefits
Paid time off (Vacation, Holidays)
401(k) savings plans
+5
Gen AI Delivery Lead - Senior Data Scientist
Gen AI Delivery Lead - Senior Data Scientist

Caterpillar Inc. • Chicago (IL)

On-site
USD 113,000 - 183,000
Medical, dental, and vision benefits
401(k) with company match
Paid time off
Manager; AI Evaluation Engineering
Manager; AI Evaluation Engineering

Mundelein Elementary School District 75 • Chicago (IL), Northern (KY)

Hybrid
USD 148,000 - 240,000
Medical, dental, and vision benefits
401(k) savings plans
Health Savings Account (HSA)
+3
Manager; AI Evaluation Engineering
Manager; AI Evaluation Engineering

Caterpillar Inc • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
HSA
+3
Senior Gen AI Engineer — Edge & Cloud Pipelines
Senior Gen AI Engineer — Edge & Cloud Pipelines

Caterpillar Inc • Chicago (IL)

On-site
USD 113,000 - 183,000
Medical, dental, and vision benefits
Paid time off
401(k) savings plans
+6