Lead Data Engineer: Cloud Pipelines & Real-Time Data (AWS)

Caterpillar; Irving (TX); On-site; USD 128,000 - 209,000; Full time

Benefits offered by this job

Health benefits

401(k) plan

Paid time off

Tuition reimbursement

Bonus

Career development

Job summary

Caterpillar Inc. in Irving, TX is seeking a Lead Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud data platforms delivering high-quality data for multiple teams.

You will drive architecture, performance, reliability, and continuous improvement in a fast-paced, Agile environment. Responsibilities include leading design of data pipelines, orchestrating real-time and batch processing, and ensuring data integrity across distributed ecosystems while

Qualifications

• Bachelor's degree in a related field and 8+ years of data engineering experience.
• Strong experience with Python, Java, SQL and cloud data platforms.
• Proven ability to lead teams and deliver scalable data solutions.

Responsibilities

• Design, build, and maintain scalable data pipelines and microservices.
• Lead cloud-based data platforms and ensure data quality and reliability.
• Collaborate with stakeholders to translate requirements into scalable architectures.
• Implement testing, monitoring, and automation for data systems.

Skills

Python

Java

SQL

AWS data services

CI/CD

Data pipelines

Education

Bachelor’s degree in Computer Science, Computer Engineering, or related field

Tools

Kinesis

S3

DynamoDB

EventBridge

Azure DevOps

Jira

Jenkins

Job description

Caterpillar Inc. in Irving, TX is seeking a Lead Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud data platforms delivering high-quality data for multiple teams.

You will drive architecture, performance, reliability, and continuous improvement in a fast-paced, Agile environment. Responsibilities include leading design of data pipelines, orchestrating real-time and batch processing, and ensuring data integrity across distributed ecosystems while