Senior AWS Data Engineer — Scale ETL & Real-Time Pipelines

Programmers.io; Malvern (Chester County); On-site; USD 120,000 - 160,000; Full time

Job summary

Programmers.io is seeking an experienced AWS Data Engineer to design and maintain scalable ETL pipelines in the AWS cloud. The role requires deep hands-on expertise with Lambda, Redshift, S3, Athena, Glue, and CI/CD practices, as well as strong Python and PySpark skills.

You will collaborate with cross-functional teams to translate data requirements into reliable, cost-efficient data architectures, monitor performance, and ensure security and compliance across data platforms.

Qualifications

• Experience with AWS services and data engineering in cloud environments.
• Strong Python and PySpark for data transformation and processing.
• Design and optimization of scalable ETL pipelines on AWS Redshift and S3.
• Knowledge of data security, privacy, and compliance frameworks.

Responsibilities

• Design, develop, and maintain robust ETL workflows to load data into Redshift.
• Use AWS Lambda for real-time data processing and automation.
• Work with S3, Athena, Glue to support data pipelines.
• Write Python and PySpark for data transformations and queries.
• Collaborate with cross-functional teams to define data requirements.
• Monitor performance, reliability, and cost of cloud infrastructure.
• Troubleshoot data pipelines to minimize downtime and ensure accuracy.
• Implement automation and monitoring tools to improve reliability.

Skills

AWS Lambda

Redshift

S3

Athena

Glue

Python

PySpark

ETL pipelines

CloudWatch

Data warehousing

CI/CD

CloudFormation

Git

Data security

Agile

Tools

CloudFormation

Git

CI/CD pipelines

Job description

Programmers.io is seeking an experienced AWS Data Engineer to design and maintain scalable ETL pipelines in the AWS cloud. The role requires deep hands-on expertise with Lambda, Redshift, S3, Athena, Glue, and CI/CD practices, as well as strong Python and PySpark skills.

You will collaborate with cross-functional teams to translate data requirements into reliable, cost-efficient data architectures, monitor performance, and ensure security and compliance across data platforms.