Senior AWS Data Engineer: ETL & Real-Time Pipelines

Diverse Lynx, LLC; Malvern (Chester County); On-site; USD 68,000 - 97,000; Full time

Job summary

Diverse Lynx LLC is seeking an experienced AWS Data Engineer to design and maintain scalable ETL pipelines in AWS, using Lambda, Redshift, S3, Athena and Glue.

You will work with Python and PySpark to transform data, ensure data security and compliance, and collaborate with cross-functional teams in an agile environment.

This hands-on role requires cloud architecture expertise and a focus on performance, reliability, and cost efficiency.

Qualifications

• Experience with AWS services including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
• Strong Python and PySpark for data transformation and processing.
• Proven ability to build scalable ETL pipelines in a cloud environment.
• In-depth understanding of relational databases, data warehousing, and Redshift performance tuning.
• Experience with Infrastructure as code (CloudFormation) and CI/CD pipelines.
• Knowledge of data security, data privacy, and compliance frameworks.

Responsibilities

• Design, develop, and maintain robust ETL workflows to load data into AWS Redshift and other cloud systems.
• Leverage AWS Lambda for real-time data processing and automation.
• Work with S3, Athena, and Glue to support data processing pipelines.
• Use Python and PySpark for data transformation logic and analytics.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Monitor cloud-based infrastructure for performance, reliability, and cost efficiency.
• Troubleshoot and resolve data pipeline issues to minimize downtime and ensure data accuracy.
• Implement automation and monitoring tools to improve reliability.

Skills

AWS Lambda

Redshift

Python

PySpark

CloudWatch

CI/CD

CloudFormation

Git

Data security

Tools

Git

CI/CD

CloudFormation

S3

Athena

Glue

Job description

Diverse Lynx LLC is seeking an experienced AWS Data Engineer to design and maintain scalable ETL pipelines in AWS, using Lambda, Redshift, S3, Athena and Glue.

You will work with Python and PySpark to transform data, ensure data security and compliance, and collaborate with cross-functional teams in an agile environment.

This hands-on role requires cloud architecture expertise and a focus on performance, reliability, and cost efficiency.