Databricks Data Engineering Lead | DataOps & Delivery

Guidehouse

San Antonio (TX)

On-site

USD 130,000 - 216,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Paid holidays
Bonus eligible
Parental leave
401(k) plan
Life insurance
HSA/FSAs
Disability insurance
Student loan assistance
Tuition reimbursement
Learning opportunities
Employee referrals
Mobility stipend

Job summary

Guidehouse is seeking a senior Data Engineer to lead Databricks data engineering and DataOps delivery across ingestion, transformation, and migration patterns. You will translate requirements into scalable designs and govern data products while mentoring junior engineers.

In this client-facing role, you will set coding standards, oversee CI/CD, and drive performance, security, and operational stability across complex data solutions.

Qualifications

  • Bachelor’s degree from an accredited university.
  • Minimum SIX (6)+ years of experience.
  • Experience in data engineering, ETL/ELT, cloud data platforms, software development, or analytics engineering.
  • Hands-on experience designing and developing production Databricks solutions using Apache Spark, PySpark, Python, SQL, Delta Lake, notebooks, workflows, and jobs.
  • Experience building scalable batch data pipelines and reusable ingestion, transformation, and orchestration frameworks.
  • Experience with enterprise data migration, source-to-target mapping, data conversion, reconciliation, validation, and defect remediation.
  • Experience implementing data-quality controls, metadata capture, lineage, logging, monitoring, and technical auditability.
  • Experience with Git-based development, automated testing, CI/CD, release management, and production stabilization.
  • Experience leading data engineers and delivering complex data solutions in a client-facing environment.
  • Ability to obtain and maintain a Federal Public Trust.

Responsibilities

  • Lead Databricks data engineering and DataOps delivery across ingestion, transformation, orchestration, migration, integration, serving, and consumption patterns.
  • Translate architecture, data models, mappings, data contracts, and migration requirements into scalable technical designs and delivery plans.
  • Design and develop reusable pipelines, migration utilities, transformation frameworks, data-quality controls, reconciliation routines, and governed data products.
  • Establish standards for coding, testing, CI/CD, deployment, observability, logging, error handling, performance, security, and operational support.
  • Lead engineering reviews and troubleshoot Databricks job failures, pipeline bottlenecks, data-quality issues, and performance problems.
  • Support batch and, where applicable, streaming data-processing patterns using Databricks, Apache Spark, and Delta Lake.
  • Coordinate technical delivery with data architects, source-system teams, integration engineers, security specialists, governance leads, testing teams, and business stakeholders.
  • Lead technical estimates, capacity planning, backlog refinement, sprint execution, release readiness, and stabilization activities.
  • Mentor data engineers and ensure reusable code, design patterns, runbooks, and lessons learned are adopted across later modernization waves.
  • Communicate technical decisions, risks, dependencies, and delivery impacts to client and program leadership.

Skills

Databricks
Apache Spark
PySpark
Python
SQL
Delta Lake
Notebooks
Workflows
Jobs
Data Engineering
ETL/ELT

Education

Bachelor's degree

Tools

Git
Azure DevOps
GitHub Actions
Jenkins
Terraform
Unity Catalog
Delta Live Tables
Databricks Workflows

Job description

Guidehouse is seeking a senior Data Engineer to lead Databricks data engineering and DataOps delivery across ingestion, transformation, and migration patterns. You will translate requirements into scalable designs and govern data products while mentoring junior engineers.

In this client-facing role, you will set coding standards, oversee CI/CD, and drive performance, security, and operational stability across complex data solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Data Engineering Lead — Senior Consultant
Databricks Data Engineering Lead — Senior Consultant

Guidehouse • Arlington (VA)

On-site
USD 130,000 - 216,000
Medical insurance
401(k) retirement plan
Parental leave
Databricks Data Engineering Lead - Delivery & DataOps
Databricks Data Engineering Lead - Delivery & DataOps

Guidehouse • McLean (VA)

On-site
USD 130,000 - 216,000
Medical Insurance
Dental & Vision Insurance
401(k) Retirement Plan
+2
Healthcare Data Engineering Lead (Databricks)
Healthcare Data Engineering Lead (Databricks)

Guidehouse • Boerne (TX)

On-site
USD 130,000 - 216,000
Medical insurance
401(k) retirement plan
Tuition reimbursement
+2
Senior Data Architect & Lead Databricks Lakehouse Strategy
Senior Data Architect & Lead Databricks Lakehouse Strategy

Guidehouse • Tysons (VA)

On-site
USD 150,000 - 210,000
Medical Insurance
Rx Coverage
Dental & Vision Insurance
+4
Lead Data Architect, Databricks & DataOps
Lead Data Architect, Databricks & DataOps

Guidehouse • San Antonio (TX)

On-site
USD 150,000 - 190,000
Medical Insurance
Dental Insurance
Vision Insurance
+8
Lead Data Architect: Databricks Lakehouse & DataOps
Lead Data Architect: Databricks Lakehouse & DataOps

Jobtome • Universal City (TX)

On-site
USD 149,000 - 248,000
Medical, Rx, Dental & Vision Insurance
Parental Leave and Adoption Assistance
401(k) Retirement Plan
+2
Lead Data Architect & Deputy Data Workstream Lead
Lead Data Architect & Deputy Data Workstream Lead

Guidehouse Inc. • McLean (VA)

On-site
USD 140,000 - 200,000
Medical Insurance
401(k) Plan
Paid Holidays
+3
Senior Healthcare Data Engineering Lead
Senior Healthcare Data Engineering Lead

Guidehouse • Atlanta (GA)

On-site
USD 130,000 - 216,000
Medical Insurance
Dental Insurance
Vision Insurance
+7
Data Engineering Manager: Lakehouse & Databricks Lead
Data Engineering Manager: Lakehouse & Databricks Lead

Ledgent Technology • San Diego (CA)

On-site
USD 162,000 - 198,000
Databricks Solutions Director: Data Platform & Delivery
Databricks Solutions Director: Data Platform & Delivery

Dentsu International Americas • United States

On-site
USD 190,000 - 240,000