Data Engineer - AWS Spark ETL & Data Lake Architect

Peregrine Advisors; Washington; Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Health insurance

401(k) match

Unlimited PTO

Professional certifications

Job summary

Peregrine Advisors is seeking a data engineer to design and operate end-to-end data pipelines on AWS, ingesting, processing, storing, and validating data at scale. You will build Spark-based ETL with Glue/EMR, manage a performant S3 data-lake feeding Iceberg, Aurora PostgreSQL, and DynamoDB, and ensure data quality and lineage across the stack.

The role requires US citizenship and the ability to obtain a Public Trust, and is based in the Washington, DC area on a hybrid schedule.

Qualifications

• US citizenship and ability to obtain Public Trust
• Hybrid role based in the Washington, DC metro area
• 4+ years of data engineering experience
• Spark ETL on AWS (Glue, EMR) in Python/PySpark
• S3 data-lake design feeding Iceberg, Aurora PostgreSQL, and DynamoDB
• Event orchestration, data quality and lineage, monitoring
• Infrastructure-as-code (CloudFormation or Terraform)
• Basic proficiency in writing, PowerPoint, and Excel

Responsibilities

• Build end-to-end data pipelines on AWS (ingest, process, store, validate)
• Develop Spark-based ETL with Glue/EMR and manage S3-based data lake
• Enable Iceberg, PostgreSQL on Aurora, and DynamoDB with federated access via Trino
• Collaborate with cross-functional teams and maintain auditability in a regulated environment
• Contribute to reliability, monitoring, and data quality initiatives

Skills

AWS data engineering

Data quality & lineage

Regulated environment awareness

PowerPoint & Excel

Education

Bachelor's degree

Master's degree (preferred)

Tools

Spark ETL on AWS (Glue/EMR)

Python & PySpark

S3 data-lake design

Iceberg tables

Aurora PostgreSQL

DynamoDB

Trino

Lambda

Step Functions

SQS/SNS

Secrets management

Monitoring

Infrastructure-as-code (CloudFormation or Terraform)

Apache Ranger

Job description

Peregrine Advisors is seeking a data engineer to design and operate end-to-end data pipelines on AWS, ingesting, processing, storing, and validating data at scale. You will build Spark-based ETL with Glue/EMR, manage a performant S3 data-lake feeding Iceberg, Aurora PostgreSQL, and DynamoDB, and ensure data quality and lineage across the stack.

The role requires US citizenship and the ability to obtain a Public Trust, and is based in the Washington, DC area on a hybrid schedule.