Lead Data Engineer – Physical AI Platform (AWS/Python)

Caterpillar Inc.; Irving (TX); On-site; USD 128,000 - 209,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits

Paid time off

401(k) matching

Health Savings Account (HSA)

Tuition Reimbursement

Job summary

Caterpillar Inc. is seeking a Lead Data Engineer in Irving, TX to design, build, and maintain scalable data pipelines and cloud-based data platforms. You will work in an agile team to improve data architecture, performance, and reliability across enterprise data solutions.

The role emphasizes real-time and batch data processing, integration frameworks, and automated testing. Strong AWS, Python, and SQL skills are essential for success and collaboration with cross-functional teams.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, or related field.
• 8+ years of experience in data engineering or related disciplines with increasing responsibility.
• Experience with Caterpillar data platforms such as Helios Data Platform.
• Strong Python development experience in production environments.
• Experience leading teams to build high-throughput, scalable data pipelines.

Responsibilities

• Actively collaborate with Principal Software Engineers and Data Architects to define solution architecture.
• Lead the design and optimization of scalable data pipelines and microservices in Python.
• Advance cloud-native data ingestion and streaming using AWS services like Kinesis, S3, DynamoDB.
• Own data integration frameworks and source data pipelines for CI Autonomy initiatives.
• Translate complex requirements into scalable data architectures and workflows.
• Establish automated testing, data quality controls, and validation frameworks.
• Lead monitoring, performance tuning, and root-cause analysis of data platforms using CloudWatch.

Skills

Python

Java

AWS

SQL

Data Pipelines

Microservices

Data Streaming

CI/CD

Jira

Jenkins

Azure DevOps

Education

Bachelor’s degree in Computer Science, Computer Engineering, or related field

Tools

AWS data services

Azure DevOps

Jira

Jenkins

CI/CD pipelines

Helios Data Platform

Job description

Caterpillar Inc. is seeking a Lead Data Engineer in Irving, TX to design, build, and maintain scalable data pipelines and cloud-based data platforms. You will work in an agile team to improve data architecture, performance, and reliability across enterprise data solutions.

The role emphasizes real-time and batch data processing, integration frameworks, and automated testing. Strong AWS, Python, and SQL skills are essential for success and collaboration with cross-functional teams.