Data Engineer: AWS Spark ETL & Data Lake Expert

Peregrine Advisors LLC; Washington (District of Columbia); Hybrid; USD 103,000 - 140,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) match

Life insurance

Disability insurance

Unlimited PTO

Certifications support

Job summary

Peregrine Advisors LLC is seeking a Data Engineer in the Washington, DC area to build scalable ETL pipelines on AWS using Spark, Glue, and EMR. You will design S3 data lakes, feed Apache Iceberg tables, and coordinate data across Aurora PostgreSQL and DynamoDB.

The role emphasizes data quality, governance, and scalable automation. You will work in a hybrid setting, interact with engineers and strategists, and contribute to a high-impact federal-ready data platform.

Qualifications

• 4+ years of data engineering experience.
• Bachelor's degree.
• Spark ETL on AWS (Glue, Amazon EMR) in Python and PySpark.
• S3 data-lake design (Parquet, partitioning, lifecycle) feeding Apache Iceberg tables, Amazon Aurora PostgreSQL, and DynamoDB.
• Event orchestration (Lambda, Step Functions, SQS/SNS) with secrets management and monitoring.
• Data quality, validation, and lineage.
• Infrastructure-as-code (CloudFormation or Terraform).
• Basic proficiency in writing, PowerPoint, and Excel.

Responsibilities

• Build ETL pipelines with Spark on AWS.
• Write processing in Python and PySpark.
• Design the S3 data-lake and Parquet layout feeding Iceberg tables.
• Connect PostgreSQL on Aurora and DynamoDB, with Trino for federated SQL.
• Deliver pipelines that keep data clean and trustworthy at scale.
• Automate data ingestion and storage design for downstream use.

Skills

Data engineering

Spark ETL

Python

PySpark

S3 data lake

AWS Lambda/Step Functions

Data quality

Terraform

Education

Bachelor's degree

Tools

Glue

Amazon EMR

Aurora PostgreSQL

DynamoDB

Trino

Iceberg

Job description

Peregrine Advisors LLC is seeking a Data Engineer in the Washington, DC area to build scalable ETL pipelines on AWS using Spark, Glue, and EMR. You will design S3 data lakes, feed Apache Iceberg tables, and coordinate data across Aurora PostgreSQL and DynamoDB.

The role emphasizes data quality, governance, and scalable automation. You will work in a hybrid setting, interact with engineers and strategists, and contribute to a high-impact federal-ready data platform.