AI Evaluation Engineering Lead

Caterpillar Inc.; Chicago (IL); On-site; USD 148,000 - 240,000; Full time

Benefits offered by this job

Medical benefits

Dental and vision

401(k)

HSA

FSAs

Career development

Incentive bonus

Disability benefits

Life Insurance

Parental leave

Tuition Reimbursement

Job summary

Caterpillar Inc. in Chicago, IL, seeks a leadership role within Cat Digital to evaluate and validate cutting-edge GenAI solutions and guide the development of AI-driven experiences for customers and dealers.

You will drive engineering best practices, ensure high-quality AI products, and scale processes across hybrid cloud and edge environments with a focus on LLMs, tool orchestration, and rigorous evaluation methodologies.

Qualifications

• Experience leading software quality, test automation, validation and AI evaluation teams.
• Delivery of enterprise-scale software and GenAI solutions across hybrid cloud and edge environments.
• Strong knowledge of modern software engineering practices and AI tooling.

Responsibilities

• Provide technical direction to ensure alignment with goals and successful evaluation of AI projects.
• Oversee performance of team members and identify training needs.
• Own the quality of AI engineering products and enforce best practices in development.

Skills

GenAI architecture

LLMs

Prompt engineering

AI evaluation

Observability

CI/CD

Tools

Azure

AWS

GCP

SageMaker

Job description

Caterpillar Inc. in Chicago, IL, seeks a leadership role within Cat Digital to evaluate and validate cutting-edge GenAI solutions and guide the development of AI-driven experiences for customers and dealers.

You will drive engineering best practices, ensure high-quality AI products, and scale processes across hybrid cloud and edge environments with a focus on LLMs, tool orchestration, and rigorous evaluation methodologies.