AWS Python Data Engineer-6

REALIGN LLC; Malvern (Chester County); On-site; USD 120,000 - 170,000; Full time

Job summary

REALIGN LLC is seeking an experienced AWS Python Data Engineer to design, develop, and support scalable data pipelines and data solutions in a fast-paced environment. The role emphasizes Python, SQL, AWS, ETL/ELT processes, and large-scale data processing to deliver robust data platforms.

The ideal candidate will design and optimize AWS-based data architectures, implement data quality checks, and collaborate with data analysts, scientists, and business stakeholders to drive data-driven

Qualifications

• Strong Python development experience.
• Hands-on AWS Data Services experience.
• ETL/ELT and data pipeline development experience.
• Experience with PySpark/Apache Spark.
• Experience with AWS Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions.
• Experience with data lakes, data warehouses, and large-scale data processing.
• Knowledge of data modeling and data integration.
• Experience with APIs, JSON, and cloud-based data platforms.
• Experience with CI/CD and version control tools such as Git.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize AWS-based data solutions and data architectures.
• Develop data processing and automation solutions using Python and PySpark.
• Work with AWS services such as Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions.
• Integrate data from APIs, databases, files, and other internal and external sources.
• Write and optimize SQL queries and support data modeling and data warehousing initiatives.
• Implement data quality checks and ensure data accuracy, integrity, and reliability.
• Troubleshoot data pipeline and production issues.
• Collaborate with data analysts, data scientists, and business stakeholders.
• Support CI/CD, version control, testing, and deployment processes.

Skills

Python

AWS Data Services

ETL/ELT

PySpark

Git / Version Control

API integration

Data modeling

CI/CD

Data quality checks

Tools

Git

Apache Airflow

Apache Spark / PySpark

AWS Glue

S3

Lambda

Redshift

EMR

Athena

Step Functions

Kafka

Kinesis

Job description

Job Description

We are looking for an experienced AWS Python Data Engineer to design, develop, and support scalable data pipelines and data solutions. The ideal candidate will have strong hands-on experience with Python, SQL, AWS, ETL/ELT processes, and large-scale data processing . Candidates with prior Vanguard experience are highly preferred .

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize AWS-based data solutions and data architectures.
• Develop data processing and automation solutions using Python and PySpark.
• Work with AWS services such as Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions .
• Integrate data from APIs, databases, files, and other internal and external sources.
• Write and optimize SQL queries and support data modeling and data warehousing initiatives.
• Implement data quality checks and ensure data accuracy, integrity, and reliability.
• Troubleshoot data pipeline and production issues.
• Collaborate with data analysts, data scientists, and business stakeholders.
• Support CI/CD, version control, testing, and deployment processes.

Required Skills

• Strong hands-on experience with Python .
• Strong experience with AWS Data Services .
• Experience with ETL/ELT and data pipeline development .
• Experience with PySpark/Apache Spark .
• Experience with AWS Glue, S3, Lambda, Redshift, EMR, Athena, and Step Functions .
• Experience with data lakes, data warehouses, and large-scale data processing.
• Knowledge of data modeling and data integration.
• Experience with APIs, JSON, and cloud-based data platforms.
• Experience with CI/CD and version control tools such as Git.

Preferred Skills

• Prior Vanguard experience is highly preferred .
• Experience with Apache Airflow.
• Experience with Apache Iceberg and Parquet.
• Knowledge of streaming technologies such as Kafka or Kinesis.
• Experience with data governance, security, and IAM.
• Financial services or investment management domain experience.