Senior Data Engineer: Python Pipelines & Cloud CI/CD

Caterpillar Brazil; Peoria (IL); On-site; USD 112,710 - 183,140; Full time

Benefits offered by this job

Medical, dental, and vision benefits

401(k) savings plans

Health Savings Account (HSA)

Job summary

Caterpillar Inc. is seeking a Senior Data Engineer to join the Helios Data Engineering team in Peoria, IL or Chicago, IL. You will develop Python data pipelines and data objects used by business applications, while building robust CI/CD pipelines and ensuring high-quality software delivery.

The role requires extensive Python and OOP experience, embedding models, and strong knowledge of Git and CI/CD tools. Candidates should be comfortable working with AWS components and multiple databases to

Qualifications

• Extensive software development experience with Python and OOP principles.
• Experience embedding models and chunking data for scalable pipelines.
• Strong knowledge of Git version control and CI/CD practices.
• Familiarity with AWS services (Bedrock, Lambda, S3, Sagemaker).
• Experience with relational and NoSQL databases (PostgreSQL, DynamoDB, MongoDB or Oracle).

Responsibilities

• Develop Python data pipelines and build data objects to support applications.
• Build and deploy CI/CD pipelines and automate deployment processes.
• Translate design requirements into robust engineering implementations.
• Perform unit and integration testing to ensure software quality.
• Collaborate with cross-functional teams to improve data engineering components.

Skills

Embedding models

OOP

Python

Git

CI/CD tooling

Tools

Jenkins

GitHub Actions

Azure DevOps

AWS Bedrock

AWS Lambda

AWS S3

AWS Sagemaker

AWS IAM

AWS RDS

PostgreSQL

DynamoDB

MongoDB

Oracle

Job description

Caterpillar Inc. is seeking a Senior Data Engineer to join the Helios Data Engineering team in Peoria, IL or Chicago, IL. You will develop Python data pipelines and data objects used by business applications, while building robust CI/CD pipelines and ensuring high-quality software delivery.

The role requires extensive Python and OOP experience, embedding models, and strong knowledge of Git and CI/CD tools. Candidates should be comfortable working with AWS components and multiple databases to