Data Platform Engineer: Lakehouse & Automation

Guidehouse Digital, LLC; Washington (District of Columbia); On-site; USD 113,000 - 188,000; Full time

Job summary

Guidehouse is seeking a Data Infrastructure Engineer to build and operate a scalable data platform powering AI/ML analytics. You will design ingest pipelines, a modern lakehouse on AWS, and a governed metadata layer.

You will implement CI/CD for data pipelines, enforce security controls, and collaborate with cross-functional teams. A bachelor’s degree and 6+ years of experience are expected.

Qualifications

• Bachelor's degree in Engineering, IT, CS, or related field.
• Minimum 6 years of experience building production data pipelines or data platforms.
• Experience building a data lake/delta lake (lakehouse) on AWS.
• Proficiency in SQL and Python (or Scala/Java).
• Experience with metadata management, governance, and policy enforcement.
• Experience automating AWS provisioning with IaC across environments.
• Experience building or operating CI/CD pipelines for data workflows.
• Solid security fundamentals: IAM, encryption, secrets management, secure SDLC.

Responsibilities

• Build & operate batch and streaming data pipelines.
• Design and implement batch/streaming ingestion from APIs, databases, files, and events.
• Deliver a modern lakehouse on AWS S3 with Delta Lake concepts and schema evolution.
• Establish a managed metadata repository with cataloging, lineage, and quality checks.
• Implement policy-based access, data classification, retention, and secure handling.
• Develop automated cloud provisioning (IaC) and CI/CD for data pipelines.

Skills

SQL

Python

Scala/Java

Databricks

IaC (Terraform)

Git

CI/CD for data pipelines

Data governance

Security fundamentals

AWS

ETL/ELT design

Education

Bachelor's degree in Engineering/IT/CS or related field

Tools

Terraform

Git

Docker

Job description

Guidehouse is seeking a Data Infrastructure Engineer to build and operate a scalable data platform powering AI/ML analytics. You will design ingest pipelines, a modern lakehouse on AWS, and a governed metadata layer.

You will implement CI/CD for data pipelines, enforce security controls, and collaborate with cross-functional teams. A bachelor’s degree and 6+ years of experience are expected.