AWS Lakehouse Data Engineer: Scalable AI/ML Pipelines

Guidehouse; Washington (District of Columbia); On-site; USD 113,000 - 188,000; Full time

Benefits offered by this job

Medical Insurance

401(k) Retirement Plan

Paid Holidays

Job summary

Guidehouse seeks an AWS Lakehouse Data Engineer to design, build, and operate a cloud-native data platform powering AI/ML, analytics, reporting, and data visualization. You will construct a scalable lakehouse on Amazon S3 using Iceberg/Parquet, enable ACID transactions, and provide fast interactive querying with AWS services.

You will implement batch and streaming ingestion, Python/PySpark ETL pipelines, metadata governance, and automated provisioning with CI/CD.

Qualifications

• Bachelor's degree in Engineering, IT, CS, Data Engineering or four years of equivalent practical experience.
• 6+ years of relevant experience in data engineering and cloud data platforms.
• Hands-on experience delivering AWS-native lakehouse architectures with S3, Glue, Athena, EMR, Lake Formation, and Redshift.

Responsibilities

• Design, implement, and operate batch and streaming data pipelines using Python and PySpark.
• Build a Delta Lakehouse-style platform on AWS with Iceberg and Parquet formats.
• Enable governance, metadata, lineage, and fine-grained access control using AWS services.
• Develop CI/CD and IaC for repeatable, secure deployments across environments.
• Collaborate with analytics, AI/ML, and security teams to meet regulatory and operational needs.

Skills

Python

PySpark

ETL/ELT pipelines

SQL

Data governance

Education

Bachelor's degree in Engineering / CS / Data Engineering

Tools

AWS Glue

Amazon Athena

Amazon EMR

Delta Lake / Iceberg

S3

Job description

Guidehouse seeks an AWS Lakehouse Data Engineer to design, build, and operate a cloud-native data platform powering AI/ML, analytics, reporting, and data visualization. You will construct a scalable lakehouse on Amazon S3 using Iceberg/Parquet, enable ACID transactions, and provide fast interactive querying with AWS services.

You will implement batch and streaming ingestion, Python/PySpark ETL pipelines, metadata governance, and automated provisioning with CI/CD.