Senior AWS Data Engineer: Real-Time ETL & Redshift

Programmers.io; Malvern (Chester County); On-site; USD 110,000 - 160,000; Full time

Job summary

Programmers.io is seeking an experienced AWS Data Engineer to design, build, and maintain scalable ETL pipelines within the AWS ecosystem. You will leverage AWS Lambda, Redshift, S3, Athena, and Glue, using Python and PySpark to transform and analyze large datasets.

You will collaborate with cross-functional teams to ensure data requirements are met, implement automation, monitor pipelines, and focus on performance, reliability, and cost efficiency in a fast-paced environment.

Qualifications

• 10+ years of experience in data engineering or similar roles.
• Hands-on with AWS data services and serverless architectures.
• Proficiency in Python and PySpark for data transformations.
• Experience delivering scalable ETL pipelines in cloud environments.

Responsibilities

• Design, develop, and maintain ETL workflows to load data into AWS Redshift and other cloud systems.
• Leverage AWS Lambda for real-time data processing and automation.
• Work with S3, Redshift, Athena, and Glue to support data pipelines.
• Use Python and PySpark to optimize data transformation logic and queries.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.

Skills

ETL design

Cloud architecture

Data modeling

Problem solving

Agile working

Tools

Git

CI/CD

AWS CloudFormation

S3

Athena

Glue

Redshift

Job description

Programmers.io is seeking an experienced AWS Data Engineer to design, build, and maintain scalable ETL pipelines within the AWS ecosystem. You will leverage AWS Lambda, Redshift, S3, Athena, and Glue, using Python and PySpark to transform and analyze large datasets.

You will collaborate with cross-functional teams to ensure data requirements are met, implement automation, monitor pipelines, and focus on performance, reliability, and cost efficiency in a fast-paced environment.