Senior Data Engineer — Python Pipelines, CI/CD & GenAI

Caterpillar; Peoria (IL); Hybrid; USD 113,000 - 183,000; Full time

Benefits offered by this job

Medical, dental, vision benefits

401(k) savings plans

Career development

Job summary

Caterpillar is seeking a Senior Data Engineer to join the Data Engineering team in Chicago/Peoria/Dallas. You will develop Python data pipelines that support applications and build data objects used across Cat Digital initiatives.

You will design and deploy CI/CD pipelines, implement source-to-target mapping, and perform unit/integration tests while leveraging AWS services, Snowflake, and GenAI tools. This onsite role requires five days a week at designated locations.

Qualifications

• Experience designing and developing software applications in Python.
• Experience with Git version control on medium to large teams.
• Deploying software using CI/CD tools such as Jenkins, GitHub Actions, Azure DevOps, etc.
• Experience with AWS components like Lambda, Kinesis, EventBridge, SageMaker, IAM and RDS.

Responsibilities

• Develop Python data pipelines that build business data objects.
• Build and deploy CI/CD pipelines and manage pipeline code.
• Perform unit and integration testing to ensure quality.

Skills

Python

Git

CI/CD tooling

AWS

Snowflake

GenAI tools

Tools

GitHub Actions

Azure DevOps

Jenkins

AWS Lambda

AWS Kinesis

SageMaker

S3

Snowflake

GenAI tools

Job description

Caterpillar is seeking a Senior Data Engineer to join the Data Engineering team in Chicago/Peoria/Dallas. You will develop Python data pipelines that support applications and build data objects used across Cat Digital initiatives.

You will design and deploy CI/CD pipelines, implement source-to-target mapping, and perform unit/integration tests while leveraging AWS services, Snowflake, and GenAI tools. This onsite role requires five days a week at designated locations.