Data Engineer — Python Pipelines & AWS CI/CD

Quest Oracle Community; Peoria, Northern (IL, KY); Hybrid; USD 89,000 - 145,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits*

401(k) savings plans*

Paid time off

Job summary

Caterpillar Inc. is seeking a Data Engineer for the Platform Data Engineering team to develop Python data pipelines that power business data objects for applications. You will build, deploy, and test end-to-end data pipelines and contribute to CI/CD practices in a hands-on role.

Based in Chicago, IL or Peoria, IL, you will collaborate with cross-functional teams to interpret design requirements, optimize data flows, and ensure high-quality software delivery in a fast-paced environment.

Qualifications

• Proficient in Python data engineering and OOP principles.
• Experience designing and developing data pipelines.
• Experience with Git version control and CI/CD pipelines.
• Knowledge of AWS services used in data engineering (Lambda, S3, DynamoDB, RDS).
• Experience with relational and NoSQL databases.

Responsibilities

• Develop Python data pipelines to build business data objects for applications.
• Build and deploy CI/CD pipelines.
• Perform unit and integration testing; ensure software quality.
• Interpret design requirements for engineering implementation.
• Collaborate across functional teams to identify growth opportunities.

Skills

Python

OOP

Git

CI/CD

AWS

PostgreSQL

NoSQL

DynamoDB

MongoDB

S3

Sagemaker

RDS

Linux

Tools

Jenkins

GitHub Actions

Azure DevOps

Job description

Caterpillar Inc. is seeking a Data Engineer for the Platform Data Engineering team to develop Python data pipelines that power business data objects for applications. You will build, deploy, and test end-to-end data pipelines and contribute to CI/CD practices in a hands-on role.

Based in Chicago, IL or Peoria, IL, you will collaborate with cross-functional teams to interpret design requirements, optimize data flows, and ensure high-quality software delivery in a fast-paced environment.