AWS Python Data Engineer: Scalable Data Pipelines

REALIGN LLC; Malvern (Chester County); On-site; USD 120,000 - 170,000; Full time

Job summary

REALIGN LLC is seeking an experienced AWS Python Data Engineer to design, develop, and support scalable data pipelines and data solutions in a fast-paced environment. The role emphasizes Python, SQL, AWS, ETL/ELT processes, and large-scale data processing to deliver robust data platforms.

The ideal candidate will design and optimize AWS-based data architectures, implement data quality checks, and collaborate with data analysts, scientists, and business stakeholders to drive data-driven

Qualifications

• Strong Python development experience.
• Hands-on AWS Data Services experience.
• ETL/ELT and data pipeline development experience.
• Experience with PySpark/Apache Spark.
• Experience with AWS Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions.
• Experience with data lakes, data warehouses, and large-scale data processing.
• Knowledge of data modeling and data integration.
• Experience with APIs, JSON, and cloud-based data platforms.
• Experience with CI/CD and version control tools such as Git.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize AWS-based data solutions and data architectures.
• Develop data processing and automation solutions using Python and PySpark.
• Work with AWS services such as Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions.
• Integrate data from APIs, databases, files, and other internal and external sources.
• Write and optimize SQL queries and support data modeling and data warehousing initiatives.
• Implement data quality checks and ensure data accuracy, integrity, and reliability.
• Troubleshoot data pipeline and production issues.
• Collaborate with data analysts, data scientists, and business stakeholders.
• Support CI/CD, version control, testing, and deployment processes.

Skills

Python

AWS Data Services

ETL/ELT

PySpark

Git / Version Control

API integration

Data modeling

CI/CD

Data quality checks

Tools

Git

Apache Airflow

Apache Spark / PySpark

AWS Glue

S3

Lambda

Redshift

EMR

Athena

Step Functions

Kafka

Kinesis

Job description

REALIGN LLC is seeking an experienced AWS Python Data Engineer to design, develop, and support scalable data pipelines and data solutions in a fast-paced environment. The role emphasizes Python, SQL, AWS, ETL/ELT processes, and large-scale data processing to deliver robust data platforms.

The ideal candidate will design and optimize AWS-based data architectures, implement data quality checks, and collaborate with data analysts, scientists, and business stakeholders to drive data-driven