Lead AI Automation Engineer - Data Annotation (Onsite)

Caterpillar; Irving (TX); On-site; USD 128,000 - 209,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits

Paid time off plan (Vacation, Holidays

401(k) savings plans

Health Savings Account

Flexible Spending Accounts

Career Development

Incentive bonus

Relocation assistance

Job summary

Caterpillar in Irving, TX is seeking a Lead AI Automation Engineer to provide technical leadership for a scrum team building annotation automation solutions for robotics, autonomy, computer vision, and VLA systems.

You will deploy automated labeling and captioning, train AI models, and develop data pipelines that move data through labeling, review, and publication. This role emphasizes scalable production solutions on heavy equipment data.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, AI, ML, Robotics or related field.
• Experience leading technical delivery for software or AI engineering teams.
• Experience developing production‑grade software solutions using Python and modern software engineering practices.
• Experience designing and implementing distributed systems, APIs, and data processing pipelines.
• Experience training, fine‑tuning, or deploying machine learning models.
• Strong problem‑solving skills and ability to work across multiple technical disciplines.
• 3+ years of experience generating and managing camera and LiDAR sensor datasets for perception or machine learning applications, including data collection pipeline design and ground truth generation.
• 3+ years of experience in data annotation and quality assurance for autonomous systems or computer vision projects, with demonstrated ability to define annotation guidelines, identify labeling defects, and drive measurable improvements in dataset accuracy.
• Experience using annotation and dataset management tools such as Labelbox, Supervisely, RoboFlow or similar toolchains.
• Experience using machine learning frameworks and training tools such as PyTorch, TensorFlow, ONNX, Weights & Biases, MLflow or similar frameworks.

Responsibilities

• Lead the technical execution of a scrum team focused on annotation automation and AI‑driven data production.
• Design, develop, and deploy automated labeling and captioning solutions for perception and VLA datasets.
• Train, fine‑tune, and evaluate AI models to support auto‑labeling, auto‑captioning, and annotation quality automation.
• Build data pipelines and workflows that move datasets through automated labeling, human review, quality validation, and dataset publication.
• Implement and extend platforms such as NVIDIA Cosmos Curator, Cosmos Reason, and other emerging AI automation technologies.
• Develop solutions that integrate Caterpillar data systems with external annotation providers and human‑in‑the‑loop workflows.
• Drive reliability, scalability, observability, and operational excellence across annotation automation services.
• Mentor engineers and provide day‑to‑day technical leadership for the team.

Skills

Leadership

Python

Distributed systems

APIs

Data pipelines

MLOps

Problem solving

Cross-functional collaboration

Software engineering

Camera & LiDAR data handling

Education

Bachelor's degree in Computer Science, Software Engineering, AI/ML, Robotics or related field

Tools

Labelbox

Supervisely

RoboFlow

PyTorch

TensorFlow

ONNX

Weights & Biases

MLflow

Job description

Caterpillar in Irving, TX is seeking a Lead AI Automation Engineer to provide technical leadership for a scrum team building annotation automation solutions for robotics, autonomy, computer vision, and VLA systems.

You will deploy automated labeling and captioning, train AI models, and develop data pipelines that move data through labeling, review, and publication. This role emphasizes scalable production solutions on heavy equipment data.