Data Engineer for Physical AI Platform (Autonomy)

Caterpillar Inc; Irving (TX); On-site; USD 98,000 - 158,000; Full time

Job summary

Caterpillar Inc. in Irving, TX is seeking a Data Engineer to design, build, and maintain scalable data pipelines and cloud-based data systems in an agile environment.

You will work with Python, Java, AWS, and CI/CD to deliver reliable data solutions, ensure data quality, and enable real-time decision making across distributed platforms. You'll contribute to building a robust data foundation, integrate APIs and streaming data, and collaborate with cross-functional teams to drive performance and

Qualifications

• Bachelor's degree in CS/CE or related field.
• 2+ years development on modern data platforms.
• 3+ years developing and deploying Java or Python solutions in production.
• Experience building high throughput, scalable data pipelines.
• Strong hands-on with AWS data services (Kinesis, S3, DynamoDB, EventBridge).
• Proficiency in SQL, data quality practices.
• Proficiency with CI/CD tools (Azure DevOps, Jira, Jenkins).
• Experience developing microservices for real-time data ingestion.
• Experience with relational and NoSQL databases.
• Ability to ensure data integrity across distributed systems.
• Experience with API development and secure integrations (OAuth 2.0).
• Experience with monitoring, testing, and automation in large-scale data environments.

Responsibilities

• Design, build, and optimize data pipelines and microservices using Python for real-time and batch processing.
• Develop cloud-based data ingestion solutions using AWS services (Kinesis, S3, DynamoDB, EventBridge).
• Build and maintain data integration workflows for source pipelines aligned to CI Autonomy.
• Translate business requirements into reliable data workflows, mappings, and system designs.
• Implement automated testing and validation to ensure data integrity across distributed systems.
• Monitor and troubleshoot pipelines using CloudWatch to maintain reliability.

Skills

Python

Java

AWS

SQL

CI/CD

NoSQL

APIs

Microservices

Data pipelines

Education

Bachelor's degree in CS/CE

Tools

Azure DevOps

Jira

Jenkins

CloudWatch

Kinesis

DynamoDB

S3

Job description

Caterpillar Inc. in Irving, TX is seeking a Data Engineer to design, build, and maintain scalable data pipelines and cloud-based data systems in an agile environment.

You will work with Python, Java, AWS, and CI/CD to deliver reliable data solutions, ensure data quality, and enable real-time decision making across distributed platforms. You'll contribute to building a robust data foundation, integrate APIs and streaming data, and collaborate with cross-functional teams to drive performance and