Data Engineer – Physical AI & Edge Platform

Caterpillar Inc.; Chicago (IL); On-site; USD 98,000 - 158,000; Full time

Benefits offered by this job

Medical benefits

401(k) savings plans

Health Savings Account

Job summary

Caterpillar Inc. is seeking a Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud-based data systems in an agile environment. You will focus on performance, reliability, and data quality while shaping the future of autonomous construction data platforms.

You will work on real-time and batch processing, integrating AWS services and building end-to-end data workflows. Collaboration across teams and a strong emphasis on data integrity will be essential.

Qualifications

• Bachelor’s degree in CS, CE, or related field required.
• 2+ years development on modern, large-scale data platforms.
• 3+ years developing and deploying Java or Python solutions.
• Experience building high-throughput, scalable data pipelines.
• Strong hands-on experience with AWS data services (Kinesis, S3, DynamoDB, EventBridge).
• Proficiency in SQL and data quality practices.
• Experience with CI/CD tools like Azure DevOps, Jira, Jenkins.
• Experience developing microservices for real-time data ingestion.
• Ability to ensure data integrity across distributed systems.
• Experience with API development and secure integrations (OAuth 2.0).
• Experience with monitoring, testing, and automation in large-scale data environments.

Responsibilities

• Design, build, and optimize data pipelines and microservices using Python for real-time and batch processing
• Develop cloud-based data ingestion solutions using AWS services (Kinesis, S3, DynamoDB, EventBridge)
• Build and maintain data integration workflows for source data pipelines aligned to CI Autonomy
• Translate business requirements into reliable data workflows, mappings, and system designs
• Implement automated testing and validation to ensure data integrity across distributed systems
• Monitor and troubleshoot pipelines using CloudWatch to maintain reliability

Skills

Decision Making

Effective Communication

Software Development

Software Development Life Cycle

Software Integration Engineering

Software Product Design/Architecture

Software Product Technical Knowledge

Software Product Testing

Education

Bachelor’s degree in Computer Science/Computer Engineering/related field

Tools

AWS data services

SQL

CI/CD tools (Azure DevOps, Jira, Jenkins)

Relational & NoSQL databases

Job description

Caterpillar Inc. is seeking a Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud-based data systems in an agile environment. You will focus on performance, reliability, and data quality while shaping the future of autonomous construction data platforms.

You will work on real-time and batch processing, integrating AWS services and building end-to-end data workflows. Collaboration across teams and a strong emphasis on data integrity will be essential.