Architect of AI-Driven Databricks Data Pipelines

Booz Allen Hamilton; Bethesda (MD); On-site; USD 78,000 - 176,000; Full time

Benefits offered by this job

Health benefits

Professional development

Tuition assistance

Work-life programs

Job summary

Booz Allen Hamilton seeks a Databricks Data Engineer and Scientist to design, build, and maintain robust data pipelines for a critical enterprise Lakehouse. You will apply data science, AI, and analytics skills to develop end-to-end data flows, model data, and ensure security and compliance while collaborating with cross-functional teams.

The ideal candidate has 8+ years in ETL and data pipelines, 3+ years with Databricks and AWS, Python/PySpark experience, and a track record of optimizing

Qualifications

• 8+ years designing, developing, deploying, and managing ETL/data pipelines with robust logging and error handling.
• 3+ years building data pipelines with Databricks notebooks, clusters, and job orchestration (Delta Lake, Unity Catalog).
• 3+ years writing Python and PySpark for ETL scripts and transformations.
• Experience with AI-assisted coding tools to support data engineering and modeling tasks.
• Knowledge of Bronze/Silver/Gold medallion data architectures and data modeling.

Responsibilities

• Design, build, and maintain robust data pipelines for an enterprise Lakehouse.
• Develop and optimize end-to-end data flows and ETL processes.
• Design data models and ensure data security and compliance.
• Collaborate with cross-functional teams to turn raw data into actionable insights.
• Improve performance and scalability of data pipelines.

Skills

ETL design

Python

PySpark

SQL

AI-assisted coding tools

Education

Bachelor’s degree in CS/Engineering/Math/Physics/IS&T

Tools

Databricks

Delta Lake

Unity Catalog

AWS (S3, Glue, RDS)

Job description

Booz Allen Hamilton seeks a Databricks Data Engineer and Scientist to design, build, and maintain robust data pipelines for a critical enterprise Lakehouse. You will apply data science, AI, and analytics skills to develop end-to-end data flows, model data, and ensure security and compliance while collaborating with cross-functional teams.

The ideal candidate has 8+ years in ETL and data pipelines, 3+ years with Databricks and AWS, Python/PySpark experience, and a track record of optimizing