Databricks Data Engineering Lead - Delivery & DataOps

Guidehouse

McLean (VA)

On-site

USD 130,000 - 216,000

Full time

11 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical Insurance
Dental & Vision Insurance
401(k) Retirement Plan
Parental Leave
Mobility Stipend

Job summary

Guidehouse is seeking a Senior Data Engineer to lead Databricks data engineering and DataOps for client programs. You will design scalable pipelines across ingestion, transformation, and serving layers, and drive governance and data quality initiatives.

You will mentor engineers, steer architectural decisions, and coordinate with data architects, security, and business stakeholders. Travel up to 25% and the ability to obtain a Federal Public Trust are required.

Qualifications

  • Bachelor’s degree from an accredited university.
  • Minimum six or more years of experience in data engineering or analytics.
  • Hands-on Databricks solutions using Spark, PySpark, Python, SQL, Delta Lake, notebooks, workflows, and jobs.
  • Experience building scalable batch data pipelines and reusable ingestion/transformation frameworks.
  • Experience with enterprise data migration, mapping, reconciliation, validation, and defect remediation.
  • Experience with data-quality controls, metadata, lineage, logging, monitoring, and auditability.
  • Experience with Git-based development, automated testing, CI/CD, release management, and production stabilization.
  • Ability to obtain and maintain a Federal Public Trust.

Responsibilities

  • Lead Databricks data engineering and DataOps delivery across ingestion, transformation, orchestration, migration, integration, serving, and consumption patterns.
  • Translate architecture, data models, mappings, data contracts, and migration requirements into scalable technical designs and delivery plans.
  • Design and develop reusable pipelines, migration utilities, transformation frameworks, data-quality controls, reconciliation routines, and governed data products.
  • Establish standards for coding, testing, CI/CD, deployment, observability, logging, error handling, performance, security, and operational support.
  • Lead engineering reviews and troubleshoot Databricks job failures, pipeline bottlenecks, data-quality issues, and performance problems.
  • Support batch and streaming data-processing patterns using Databricks, Apache Spark, and Delta Lake.
  • Coordinate technical delivery with data architects, source-system teams, integration engineers, security specialists, governance leads, testing teams, and business stakeholders.
  • Lead technical estimates, capacity planning, backlog refinement, sprint execution, release readiness, and stabilization activities.
  • Mentor data engineers and ensure reusable code, design patterns, runbooks, and lessons learned are adopted across later modernization waves.
  • Communicate technical decisions, risks, dependencies, and delivery impacts to client and program leadership.

Skills

Databricks
Data engineering
DataOps
Apache Spark
Python
SQL
Delta Lake
CI/CD
Observability
Security

Education

Bachelor’s degree

Tools

Databricks
Apache Spark
Delta Lake
GitHub Actions
Azure DevOps

Job description

Guidehouse is seeking a Senior Data Engineer to lead Databricks data engineering and DataOps for client programs. You will design scalable pipelines across ingestion, transformation, and serving layers, and drive governance and data quality initiatives.

You will mentor engineers, steer architectural decisions, and coordinate with data architects, security, and business stakeholders. Travel up to 25% and the ability to obtain a Federal Public Trust are required.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Data Engineering Lead | DataOps & Delivery
Databricks Data Engineering Lead | DataOps & Delivery

Guidehouse • San Antonio (TX)

On-site
USD 130,000 - 216,000
Health insurance
Paid holidays
Bonus eligible
+10
Databricks Data Engineering Lead — Senior Consultant
Databricks Data Engineering Lead — Senior Consultant

Guidehouse • Arlington (VA)

On-site
USD 130,000 - 216,000
Medical insurance
401(k) retirement plan
Parental leave
Healthcare Data Engineering Lead (Databricks)
Healthcare Data Engineering Lead (Databricks)

Guidehouse • Boerne (TX)

On-site
USD 130,000 - 216,000
Medical insurance
401(k) retirement plan
Tuition reimbursement
+2
Lead Data Architect: Databricks Lakehouse & DataOps
Lead Data Architect: Databricks Lakehouse & DataOps

Jobtome • Universal City (TX)

On-site
USD 149,000 - 248,000
Medical, Rx, Dental & Vision Insurance
Parental Leave and Adoption Assistance
401(k) Retirement Plan
+2
Lead Data Architect, Databricks & DataOps
Lead Data Architect, Databricks & DataOps

Guidehouse • San Antonio (TX)

On-site
USD 150,000 - 190,000
Medical Insurance
Dental Insurance
Vision Insurance
+8
Managing Consultant, Databricks Data Engineering Lead
Managing Consultant, Databricks Data Engineering Lead

Guidehouse • San Antonio (TX)

On-site
USD 130,000 - 216,000
Health insurance
Paid holidays
Bonus eligible
+10
Cleared Data Engineer — Databricks & Cybersec Analytics
Cleared Data Engineer — Databricks & Cybersec Analytics

US101 Guidehouse Inc. • Arlington (VA)

On-site
USD 85,000 - 141,000
Medical, Rx, Dental & Vision Insurance
401(k) Retirement Plan
Parental Leave and Adoption Assistance
+1
Senior Data Architect & Lead Databricks Lakehouse Strategy
Senior Data Architect & Lead Databricks Lakehouse Strategy

Guidehouse • Tysons (VA)

On-site
USD 150,000 - 210,000
Medical Insurance
Rx Coverage
Dental & Vision Insurance
+4
Active-Secret Data Engineer: Databricks & Pipelines
Active-Secret Data Engineer: Databricks & Pipelines

Guidehouse Inc. • Arlington (VA)

On-site
USD 85,000 - 141,000
Medical Insurance
Dental & Vision
Holidays
+6
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000