AWS Lakehouse Data Engineer: Build Scalable Platform

Guidehouse; United States; Remote; USD 113,000 - 188,000; Full time

Benefits offered by this job

Medical, Rx, Dental & Vision Insurance

401(k) Retirement Plan

Parental Leave

Tuition Reimbursement

Employee Referral Program

Flexible Spending Accounts

Discretionary incentive bonus

Job summary

Guidehouse is seeking an AWS Lakehouse Data Engineer to design and operate a cloud-native data platform powering AI/ML, analytics, and visualization. You will build a scalable lakehouse on S3 with Iceberg/Parquet, implement CDC, and automate provisioning and CI/CD across environments.

You will also implement governance, metadata, access control, and quality checks, collaborating with data, security, and analytics teams to enable trusted data for decision making.

Qualifications

• Bachelor's degree or 4 years practical experience in lieu of degree.
• 6 years of relevant experience in data engineering or cloud platforms.
• Hands-on experience with AWS-native data lake/lakehouse architectures on S3.
• Strong PySpark and Python ETL/ELT development and testing.
• Experience with Apache Iceberg including ACID, snapshots, and time travel.
• Advanced SQL skills for analytics, reporting, and data visualization.
• Experience with metadata management, governance, and access controls.

Responsibilities

• Design and implement batch and streaming data ingestion pipelines.
• Develop and optimize ETL/ELT processes with Python and PySpark.
• Build Delta Lakehouse-style lakehouse on AWS with Iceberg and Parquet.
• Enable ACID transactions, schema evolution, and time travel for lakehouse data.
• Enable fast querying with Athena/EMR/Redshift and ensure performance/cost efficiency.
• Establish standard DEV/TEST/PROD environments and CI/CD for data pipelines.
• Implement data governance, access controls, lineage, and quality checks.

Skills

Python

PySpark

ETL/ELT

SQL

AWS-native services

Data modeling

Data governance

CI/CD

Education

Bachelor's degree or 4 years experience

Tools

Apache Iceberg

Apache Parquet

AWS Glue

Amazon Athena

Amazon EMR

AWS Lake Formation

Amazon Redshift

S3

Job description

Guidehouse is seeking an AWS Lakehouse Data Engineer to design and operate a cloud-native data platform powering AI/ML, analytics, and visualization. You will build a scalable lakehouse on S3 with Iceberg/Parquet, implement CDC, and automate provisioning and CI/CD across environments.

You will also implement governance, metadata, access control, and quality checks, collaborating with data, security, and analytics teams to enable trusted data for decision making.