Databricks Data Engineer and Scientist

Booz Allen Hamilton; Denver (CO); On-site; USD 78,000 - 176,000; Full time

Job summary

Booz Allen Hamilton is seeking a Databricks Data Engineer and Scientist to design, build, and maintain data pipelines for a critical enterprise Lakehouse. You will apply data science and analytics skills, optimize ETL processes, and ensure data security and compliance while collaborating with cross-functional teams.

You will leverage Databricks, AWS, Python/PySpark, and AI-assisted coding tools to transform raw data into actionable insights and support enterprise analytics initiatives.

Qualifications

• 8+ years designing, developing, deploying, and managing ETL pipelines with robust logging and error handling.
• 3+ years building and optimizing data pipelines with Databricks notebooks/clusters/job orchestration (Delta Lake, Unity Catalog).
• 3+ years writing Python and PySpark for ETL scripts and transformations.
• Experience with AI-assisted coding tools to support data engineering and modeling tasks.
• Experience modeling data using Bronze/Silver/Gold medallion architecture.
• Experience with AWS services (Glue, S3, RDS).
• Knowledge of ETL performance optimization (partitioning, bucketing, caching) and troubleshooting.
• Ability to analyze legacy SQL and validate pipeline accuracy and performance.
• Ability to obtain Public Trust or Suitability/Fitness determination per client requirements.
• Bachelor's degree in relevant field.

Responsibilities

• Design, build, and maintain robust data pipelines for enterprise Lakehouse.
• Analyze pipelines, support data modeling, and improve performance and scalability.
• Collaborate with cross-functional teams on data security and compliance.
• Utilize Databricks and AWS to optimize end-to-end data flows.
• Turn raw data into actionable business insights.

Skills

ETL design

Databricks

Python/PySpark

AI-assisted coding

Data modeling

AWS

ETL optimization

Legacy SQL analysis

Public Trust clearance

Bachelor's degree

Education

Bachelor's degree in CS/CE/Engineering/Math/Physics/IS&T

Tools

Delta Lake

Unity Catalog

AWS Glue

S3

RDS

CloudWatch

Job description

Job Number: R0252302

Databricks Data Engineer and Scientist

The Opportunity:

As a Databricks Data Engineer and Scientist, you will design, build, and maintain robust data pipelines for a critical enterprise Lakehouse, applying your data science, AI, and analytics skills. You will develop and optimize end-to-end data flows, manage ETL processes, design data models, help ensure data security and compliance, collaborate with cross-functional teams, and improve performance and scalability. Your experience with Databricks and AWS, along with your SQL, Python, and AI-assisted coding skills, will help you analyze pipelines, support data modeling, and turn raw data into actionable insights across the business.

Work with us to use data for good.

Join us. The world can't wait.

You Have:

• 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines, with comprehensive logging and error handling to troubleshoot and resolve issues
• 3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake and Unity Catalog
• 3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations
• Experience using AI-assisted coding tools such as Claude Code or Cursor to support data engineering and data modeling tasks
• Experience with data modeling, including selecting dimensional or relational models, within a Bronze, Silver, and Gold medallion architecture
• Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS
• Knowledge of ETL performance optimization techniques, such as partitioning, bucketing, and caching, and experience troubleshooting query performance
• Ability to analyze and decompose complex legacy SQL, rebuild data pipelines or models, and compare results to validate accuracy and performance
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements
• Bachelor's degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology

Nice If You Have:

• Experience with AWS cloud-native data services such as AWS Lambda or Amazon CloudWatch that support serverless data processing
• Experience with metadata and data cataloging tools such as AWS Glue Data Catalog or Collibra to manage data assets and lineage
• Knowledge of data governance, data lineage, and compliance requirements such as GDPR and CCPA that support data quality, privacy, and security
• Ability to explain complex technical concepts to non-technical audiences, including architectural tradeoffs

Vetting:

Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client.

Compensation

At Booz Allen, we celebrate your contributions, provide you with opportunities and choices, and support your total well-being. Our offerings include health, life, disability, financial, and retirement benefits, as well as paid leave, professional development, tuition assistance, work-life programs, and dependent care. Our recognition awards program acknowledges employees for exceptional performance and superior demonstration of our values. Full-time and part-time employees working at least 20 hours a week on a regular basis are eligible to participate in Booz Allen's benefit programs. Individuals that do not meet the threshold are only eligible for select offerings, not inclusive of health benefits. We encourage you to learn more about our total benefits by visiting the Resource page on our Careers site and reviewing Our Employee Benefits page.

Salary at Booz Allen is determined by various factors, including but not limited to location, the individual's particular combination of education, knowledge, skills, competencies, and experience, as well as contract-specific affordability and organizational requirements. The projected compensation range for this position is $77,600.00 to $176,000.00 (annualized USD). The estimate displayed represents the typical salary range for this position and is just one component of Booz Allen's total compensation package for employees. This posting will close within 90 days from the Posting Date.

Identity Statement

As part of the hiring process, we will ask you to complete an identity verification process that leverages advanced biometrics and artificial intelligence to ensure authenticity and protect against identity fraud. You are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

Candidate AI Usage Policy

AI is a part of our daily work at Booz Allen, and we are committed to the responsible and ethical use of AI tools. However, we want to ensure a fair candidate process based on your own skills and knowledge. As part of this commitment, the use of artificial intelligence (AI) or other tools to assist with responses during interviews (whether in-person or virtual) is prohibited unless permission is explicitly provided.

Work Model

Our people-first culture prioritizes the benefits of collaboration. To support engagement and effective communication, employees working virtually are generally expected to have their cameras on during meetings.

• Remote: If this position is listed as remote, there may still be occasions when you are required to work in person at a Booz Allen or customer facility.
• Hybrid: If this position is listed as hybrid, you will be expected to work from a Booz Allen or customer facility, in alignment with your leadership's expectations and the needs of the role.
• Onsite: If this position is listed as onsite, work will primarily be performed full-time at a customer facility, where employees will collaborate directly with colleagues and customers as required by the role.

Commitment to Non-Discrimination

All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, local, or international law.