AI Evaluation Engineering Lead

Caterpillar

Broomfield (CO)

On-site

USD 148,000 - 240,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical benefits
401(k) plan
PTO and holidays
Life Insurance
Career development

Job summary

Caterpillar Financial Services Corporation is seeking a leader for the AI Engineering team. You will guide a group evaluating and validating advanced generative AI solutions, shaping AI-driven experiences that redefine customer and dealer interactions with technology.

The role emphasizes deep GenAI architecture, ML engineering, and modern software practices, including observability and scalable deployment. Located in the US with full-time responsibilities in a technology-focused team.

Qualifications

  • Proven experience leading software quality, test automation, validation, and AI evaluation teams.
  • Experience delivering GenAI solutions across hybrid cloud and edge environments.
  • Deep knowledge of modern AI observability, telemetry, and evaluation methodologies.

Responsibilities

  • Provide strong technical direction to ensure alignment with company goals and AI evaluation objectives.
  • Oversee performance of individuals and the team, addressing training and development needs.
  • Own the quality of AI engineering products to ensure robustness and high standards.
  • Establish and supervise engineering best practices for consistent development.

Skills

Team leadership
GenAI architecture
ML/AI engineering
CI/CD
AI evaluation
Observability

Tools

Azure
AWS
GCP
LangChain
LangSmith
SageMaker
Bedrock

Job description

Caterpillar Financial Services Corporation is seeking a leader for the AI Engineering team. You will guide a group evaluating and validating advanced generative AI solutions, shaping AI-driven experiences that redefine customer and dealer interactions with technology.

The role emphasizes deep GenAI architecture, ML engineering, and modern software practices, including observability and scalable deployment. Located in the US with full-time responsibilities in a technology-focused team.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Evaluation Engineering Lead
AI Evaluation Engineering Lead

Caterpillar Inc • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
HSA
+3
Enterprise AI Architect: Agentic Systems Lead
Enterprise AI Architect: Agentic Systems Lead

Caterpillar Inc • Irving (TX)

On-site
USD 173,000 - 259,000
Medical benefits
PTO (Paid time off)
401(k) savings plan
+8
AI Engineering Manager: Cloud-Native Agents & Tools
AI Engineering Manager: Cloud-Native Agents & Tools

Aimlroles • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical, dental & vision benefits
401(k) savings plan
Health Savings Account (HSA)
+4
Senior AI Enterprise Architect — Agentic Systems
Senior AI Enterprise Architect — Agentic Systems

Caterpillar • Broomfield (CO)

On-site
USD 173,000 - 259,000
Medical, dental, and vision benefits
401(k) savings plans
Tuition Reimbursement
Senior AI Architect for Agentic Enterprise AI
Senior AI Architect for Agentic Enterprise AI

Caterpillar Inc. • Westminster (CO)

On-site
USD 173,000 - 259,000
Medical, dental, and vision benefits
401(k) savings plans
Health Savings Account (HSA)
+8
Manager; AI Evaluation Engineering
Manager; AI Evaluation Engineering

Caterpillar Inc • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
HSA
+3
Manager; AI Evaluation Engineering
Manager; AI Evaluation Engineering

Caterpillar • Broomfield (CO)

On-site
USD 148,000 - 240,000
Medical benefits
401(k) plan
PTO and holidays
+2
Senior AI Analytics Lead, R&D Prototyping
Senior AI Analytics Lead, R&D Prototyping

Caterpillar • Chicago (IL)

On-site
USD 159,000 - 259,000
Medical, dental and vision benefits
401(k) savings plans
Paid time off (PTO)
+3
Cloud-Native AI Engineering Manager
Cloud-Native AI Engineering Manager

Caterpillar Inc • Chicago (IL)

On-site
USD 148,000 - 240,000
Medical benefits
Dental benefits
Vision benefits
+5
Lead AI Factory Architect for Autonomous Systems
Lead AI Factory Architect for Autonomous Systems

Caterpillar Inc. • Irving (TX)

On-site
USD 159,000 - 259,000
Medical, dental, and vision benefits
Paid time off plan
401(k) savings plans
+1