Lead AI Automation Engineer, Data Annotation Services

Caterpillar Inc.; Irving (TX); On-site; USD 128,000 - 209,000; Full time

Benefits offered by this job

Medical benefits

Paid time off

401(k) plans

HSA

FSAs

Career development

Bonus

Relocation assistance

Visa sponsorship

Job summary

Caterpillar Inc. is seeking a Lead AI Automation Engineer to guide a scrum team building automated annotation and captioning for robotics, autonomy, CV, and Vision-Language-Action systems. You’ll blend software, ML, and data engineering to deploy scalable labeling and QA solutions on real-world heavy equipment.

You’ll work on AI models, data pipelines, and platforms like NVIDIA Cosmos Curator to enable automated labeling, captioning, and data-driven improvements at scale in a production setting.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, AI, ML, Robotics, or related field.
• Experience leading technical delivery for software or AI engineering teams.
• Experience developing production-grade software using Python and modern practices.
• Experience designing distributed systems, APIs, and data pipelines.
• Experience training, fine-tuning, or deploying ML models.
• Strong problem-solving skills across technical disciplines.
• 3+ years creating/ managing camera and LiDAR datasets for perception or ML.
• 3+ years data annotation and QA for autonomous systems or CV projects.

Responsibilities

• Lead the technical execution of a scrum team focused on annotation automation and AI-driven data production.
• Design, develop, and deploy automated labeling and captioning solutions for perception and VLA datasets.
• Train, fine-tune, and evaluate AI models to support auto-labeling and QA automation.
• Build data pipelines and workflows for automated labeling, human review, QA, and publication.
• Extend platforms like NVIDIA Cosmos Curator and Cosmos Reason and other AI automation tech.
• Integrate Caterpillar data systems with external annotation providers and human-in-the-loop workflows.
• Drive reliability, scalability, observability, and operational excellence of annotation services.
• Mentor engineers and provide day-to-day technical leadership.

Skills

Leadership

Python

Distributed systems

Data pipelines

ML models

Problem solving

Cross-functional collaboration

AI automation

Education

Bachelor's degree in Computer Science or related field

Tools

Labelbox

Supervisely

RoboFlow

PyTorch

TensorFlow

ONNX

Weights & Biases

MLflow

Job description

Job Description

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Career Area

Technology, Digital and Data

Job Description

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Help Build the Future of Caterpillar –

At Caterpillar, technology always has a purpose, which is to solve our customers' toughest challenges. Through Cat Technology, we are solving problems by building the intelligence layer that connects machines, data, and people to make jobsites safer, more productive, and more sustainable. By combining deep domain expertise in physical systems with software, connectivity, autonomy, and AI, we deliver solutions that work in the real world - on real jobsites, at global scale.

You'll build and deploy against one of the most unique data foundations - over 1.6 million connected assets generating real-world data daily. These data and platform capabilities are enabling the development of AI models, edge computing architectures, and software systems that scale across fleets, products, and industries. The result will be a new generation of machines that continuously learn, improve, and deliver performance at scale.

Be Part of What's Next in Autonomous Construction Sites

Construction autonomy is one of the most complex challenges in applied AI, and at Caterpillar, advancements in physical AI, simulation, sensing, and edge computing are turning things that once felt impossible - intelligent machines operating in dynamic jobsites - into reality. Our connected ecosystem brings together massive volumes of high-quality data to create a foundation where engineers like you can build and deploy against.

If this work motivates you, we invite you to join our team. In these roles, you'll work at the intersection of the physical and digital worlds. You'll help design and deliver intelligent systems that enable machines to perceive their environment, make informed decisions, and support safer, more productive operations.

We are seeking a Lead AI Automation Engineer to provide technical leadership for a scrum team responsible for developing annotation automation solutions used to create training data for robotics, autonomy, computer vision, and Vision-Language-Action (VLA) systems. This role combines software engineering, machine learning, and data engineering to build automated annotation and captioning capabilities for real-world heavy equipment and construction environments. You will help deploy and extend technologies such as NVIDIA Cosmos Curator and Cosmos Reason while also developing Caterpillar-specific AI models that enable automated labeling, automated captioning, and AI-driven quality assurance. This is a hands-on technical leadership role focused on turning complex AI and data challenges into scalable production solutions.

What You’ll Do

• Lead the technical execution of a scrum team focused on annotation automation and AI-driven data production.
• Design, develop, and deploy automated labeling and captioning solutions for perception and VLA datasets.
• Train, fine-tune, and evaluate AI models to support auto-labeling, auto-captioning, and annotation quality automation.
• Build data pipelines and workflows that move datasets through automated labeling, human review, quality validation, and dataset publication.
• Implement and extend platforms such as NVIDIA Cosmos Curator, Cosmos Reason, and other emerging AI automation technologies.
• Develop solutions that integrate Caterpillar data systems with external annotation providers and human-in-the-loop workflows.
• Drive reliability, scalability, observability, and operational excellence across annotation automation services.
• Mentor engineers and provide day-to-day technical leadership for the team.

What Makes This Role Unique

Most AI automation solutions are not trained to understand heavy equipment, jobsites, sensor data, or machine operations. In this role, you'll help develop the AI capabilities needed to label and describe Caterpillar-specific data at scale. You'll work on challenges involving computer vision, multimodal AI, robotics, autonomy, and Vision-Language-Action systems while helping create the data foundation that powers the future of intelligent machines.

Basic Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Machine Learning, Robotics, or a related field.
• Experience leading technical delivery for software or AI engineering teams.
• Experience developing production-grade software solutions using Python and modern software engineering practices.
• Experience designing and implementing distributed systems, APIs, and data processing pipelines.
• Experience training, fine-tune, or deploying machine learning models.
• Strong problem-solving skills and ability to work across multiple technical disciplines.
• 3+ years of experience generating and managing camera and LiDAR sensor datasets for perception or machine learning applications, including data collection pipeline design and ground truth generation.
• 3+ years of experience in data annotation and quality assurance for autonomous systems or computer vision projects, with demonstrated ability to define annotation guidelines, identify labeling defects, and drive measurable improvements in dataset accuracy.
• Experience using annotation and dataset management tools such as Labelbox, Supervisely, RoboFlow or similar toolchains.
• Experience using machine learning frameworks and training tools such as PyTorch, TensorFlow, ONNX, Weights & Biases, MLflow or similar frameworks.

Preferred Qualifications

• Experience with computer vision, robotics, autonomy, Physical AI, or multimodal AI systems.
• Experience with AI-assisted annotation, auto-labeling, or machine learning data pipelines.
• Experience training or fine-tuning foundation models, vision models, or multimodal models.
• Experience with NVIDIA Cosmos Curator, Cosmos Reason, or similar AI automation platforms.
• Experience with Vision-Language-Action (VLA) datasets and workflows.
• Experience with sensor data such as cameras, LiDAR, radar, or autonomous machine datasets.
• Experience with cloud platforms, MLOps, and machine learning deployment pipelines.
• Experience building human-in-the-loop AI workflows and quality automation systems.

Additional Details

• This position requires the candidate to work full-time at the Irving, TX office (Dallas)
• Domestic relocation assistance is available for this position.
• Visa sponsorship is available with this position.

Why Caterpillar

You'll work on some of the most challenging applications of AI in the physical world, helping build systems that transform vast amounts of machine and sensor data into the intelligence that powers autonomous and AI-enabled equipment. If you're excited about applying AI to real-world machines operating in complex environments, we'd like to hear from you.

Summary Pay Range

$128,470.00 - $208,770.00

Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar.

Benefits

Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits.

• Medical, dental, and vision benefits*
• Paid time off plan (Vacation, Holidays, Volunteer, etc.)*
• 401(k) savings plans*
• Health Savings Account (HSA)*
• Flexible Spending Accounts (FSAs)*
• Health Lifestyle Programs*
• Employee Assistance Program*
• Voluntary Benefits and Employee Discounts*
• Career Development*
• Incentive bonus*
• Disability benefits
• Life Insurance
• Parental leave
• Adoption benefits
• Tuition Reimbursement
• These benefits also apply to part-time employees

This position requires working onsite five days a week.

Relocation is available for this position.

Visa sponsorship is available for eligible applicants.

Posting Dates

Any offer of employment is conditioned upon the successful completion of a drug screen.

Caterpillar is an Equal Opportunity Employer, Including Veterans and Individuals with Disabilities. Qualified applicants of any age are encouraged to apply.