AI Automation Engineer - Perception Data Annotation

Socket.dev; Irving (TX); On-site; USD 98,000 - 158,000; Full time

Benefits offered by this job

Relocation assistance

Visa sponsorship

Job summary

This Caterpillar Inc. role focuses on building and deploying automated labeling and captioning for perception datasets, training AI models, and creating data pipelines to scale annotation workflows.

You will integrate Caterpillar data systems with external providers and ensure scalable, reliable AI automation across autonomous machine data. Candidate will work on AI/ML models, computer vision, multimodal AI, and Vision-Language-Action systems applied to heavy equipment data, with on-site

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Machine Learning, Robotics, or a related field.
• Experience developing production‑grade software solutions using Python, modern software engineering practices and containerization platforms (e.g. Docker, Kubernetes)
• Experience designing and implementing distributed systems, APIs, and data processing pipelines.
• Experience training, fine‑tuning, or deploying machine learning models.
• Strong problem‑solving skills and ability to work across multiple technical disciplines.
• Experience generating and managing camera and LiDAR sensor datasets for perception or machine learning applications, including data collection pipeline design and ground truth generation.
• Experience in data annotation and quality assurance for autonomous systems or computer vision projects, with demonstrated ability to define annotation guidelines, identify labeling defects, and drive measurable improvements in dataset accuracy.
• Experience using annotation and dataset management tools such as Labelbox, Supervisely, RoboFlow or similar toolchains.
• Experience using machine learning frameworks and training tools such as PyTorch, TensorFlow, ONNX, Weights & Biases, MLflow or similar frameworks.

Responsibilities

• Develop and deploy automated labeling and captioning solutions for perception and VLA datasets.
• Train, fine-tune, and evaluate AI models to support auto-labeling, auto-captioning, and annotation quality automation.
• Build data pipelines and workflows that move datasets through automated labeling, human review, quality validation, and dataset publication.
• Implement and extend platforms such as NVIDIA Cosmos Curator, Cosmos Reason, and other emerging AI automation technologies.
• Develop solutions that integrate Caterpillar data systems with external annotation providers and human-in-the-loop workflows.
• Ensure reliability, scalability, observability, and operational excellence across annotation automation services.

Skills

Python programming

Distributed systems

APIs and data pipelines

Machine learning

Docker

Kubernetes

PyTorch

TensorFlow

ONNX

Weigths & Biases

Education

Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Machine Learning, Robotics, or a related field

Tools

NVIDIA Cosmos Curator

Cosmos Reason

Labelbox

Supervisely

RoboFlow

Job description

This Caterpillar Inc. role focuses on building and deploying automated labeling and captioning for perception datasets, training AI models, and creating data pipelines to scale annotation workflows.

You will integrate Caterpillar data systems with external providers and ensure scalable, reliable AI automation across autonomous machine data. Candidate will work on AI/ML models, computer vision, multimodal AI, and Vision-Language-Action systems applied to heavy equipment data, with on-site