Data Engineer – Real-Time AI & Cloud Data Pipelines

Caterpillar; Chicago (IL); On-site; USD 98,000 - 158,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits

401(k) savings plans

Health Savings Account (HSA)

Flexible Spending Accounts (FSAs)

Relocation assistance

Visa sponsorship available

Job summary

Caterpillar in Chicago, IL seeks a Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud-based data systems in an agile environment, focusing on performance and data quality. You will work with Python and Java, AWS Kinesis, S3, DynamoDB, and EventBridge to ingest and process data, implement tests, and ensure reliability across distributed systems.

This full-time onsite role offers relocation assistance and a comprehensive benefits package.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, or related field.
• 3+ years developing and deploying Java or Python solutions to production.
• Experience building high-throughput, scalable data pipelines.
• Strong hands-on experience with AWS data services (Kinesis, S3, DynamoDB, EventBridge).
• Proficiency in SQL and data quality/validation practices.
• Experience with CI/CD tools (Azure DevOps, Jira, Jenkins).
• Ability to design microservices that support real-time data ingestion.

Responsibilities

• Design, build, and optimize data pipelines and microservices using Python and Java.
• Develop cloud-based data ingestion solutions on AWS.
• Build and maintain data integration workflows aligned to CI Autonomy.
• Translate business requirements into reliable data workflows and designs.
• Implement automated testing and validation to ensure data integrity.
• Monitor pipelines and troubleshoot to maintain reliability.

Skills

Python

Java

Distributed data systems

Cloud computing

Data pipelines

Performance optimization

Data quality assurance

CI/CD practices

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS

Kinesis

S3

DynamoDB

EventBridge

CI/CD tools

Jira

Jenkins

Job description

Caterpillar in Chicago, IL seeks a Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud-based data systems in an agile environment, focusing on performance and data quality. You will work with Python and Java, AWS Kinesis, S3, DynamoDB, and EventBridge to ingest and process data, implement tests, and ensure reliability across distributed systems.

This full-time onsite role offers relocation assistance and a comprehensive benefits package.