Active-Secret Data Engineer: Databricks & Pipelines

Guidehouse Inc.

Arlington (VA)

On-site

USD 85,000 - 141,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical Insurance
Dental & Vision
Holidays
401(k) Plan
Tuition Reimbursement
Learning Opportunities
Mobility Stipend
Student Loan PayDown
Life Insurance

Job summary

Guidehouse is seeking a Data Engineer to design, develop, and manage scalable data pipelines in Databricks using Python, PySpark, and SQL. You will transform raw data into analysis-ready datasets to support cybersecurity metrics, dashboards, and executive reporting.

You will implement automated data-quality controls and governance, and collaborate with cross-functional teams to meet client needs. The role requires an Active Secret clearance, a Bachelor's degree and at least six years in data or

Qualifications

  • Active Secret clearance required.
  • Six years of total experience in data or programming, including three years in data engineering.
  • Proficiency in Python, PySpark, and SQL.
  • Experience with scheduling, monitoring, troubleshooting, and optimizing production ETL/ELT pipelines.

Responsibilities

  • Design, build, test, deploy, and maintain scalable batch and streaming data pipelines in Databricks using Python, PySpark, and SQL.
  • Transform raw data into curated, analysis-ready datasets and models supporting cybersecurity metrics, dashboards, and executive reporting.
  • Develop automated data-quality controls for ETL jobs, including schema, completeness, validity, and timeliness checks.
  • Define handling for failed quality checks and coordinate corrective action with data owners.
  • Schedule and monitor production workflows; troubleshoot, optimize performance and cost.
  • Maintain data governance standards, including metadata, lineage, and access controls.
  • Document data mappings, transformation rules, and runbooks for audit readiness.
  • Collaborate with analysts, data owners, admins, dashboard developers, and stakeholders to define data needs.
  • Organize client meetings, prepare agendas, record minutes, and track action items.

Skills

Python
PySpark
SQL
Data Engineering
Problem Solving

Education

Bachelor's degree
Master's degree

Tools

Databricks

Job description

Guidehouse is seeking a Data Engineer to design, develop, and manage scalable data pipelines in Databricks using Python, PySpark, and SQL. You will transform raw data into analysis-ready datasets to support cybersecurity metrics, dashboards, and executive reporting.

You will implement automated data-quality controls and governance, and collaborate with cross-functional teams to meet client needs. The role requires an Active Secret clearance, a Bachelor's degree and at least six years in data or

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Cleared Data Engineer — Databricks & Cybersec Analytics
Cleared Data Engineer — Databricks & Cybersec Analytics

US101 Guidehouse Inc. • Arlington (VA)

On-site
USD 85,000 - 141,000
Medical, Rx, Dental & Vision Insurance
401(k) Retirement Plan
Parental Leave and Adoption Assistance
+1
Data Engineer - Secure Pipelines (Secret)
Data Engineer - Secure Pipelines (Secret)

Agile Defense, Inc. • Bellevue Second IV Precinct (NE)

On-site
USD 110,000 - 150,000
Data Engineer (Secret Clearance) – Databricks & Power Apps
Data Engineer (Secret Clearance) – Databricks & Power Apps

Guidehouse • Arlington (VA)

Hybrid
USD 85,000 - 141,000
Medical, Rx, Dental & Vision Insurance
Personal and Family Sick Time & Paid H
401(k) Retirement Plan
+2
Databricks Data Engineering Lead — Senior Consultant
Databricks Data Engineering Lead — Senior Consultant

Guidehouse • Arlington (VA)

On-site
USD 130,000 - 216,000
Medical insurance
401(k) retirement plan
Parental leave
Data Engineer: Secure Pipelines & Analytics (Secret)
Data Engineer: Secure Pipelines & Analytics (Secret)

Agile Defense, Inc. • Illinois

On-site
USD 135,000 - 165,000
Senior Data Engineer (Secret) Databricks, ETL & Pipelines
Senior Data Engineer (Secret) Databricks, ETL & Pipelines

Rubix Solutions LLC • Arlington (VA)

Hybrid
USD 100,000 - 130,000
Databricks Data Engineering Lead - Delivery & DataOps
Databricks Data Engineering Lead - Delivery & DataOps

Guidehouse • McLean (VA)

On-site
USD 130,000 - 216,000
Medical Insurance
Dental & Vision Insurance
401(k) Retirement Plan
+2
Data Engineer - Secure Data Pipelines (Secret)
Data Engineer - Secure Data Pipelines (Secret)

Agile Defense, Inc. • O'Fallon (IL)

On-site
USD 135,000 - 165,000
Data Engineer - Secure Data Pipelines (Secret Clearance)
Data Engineer - Secure Data Pipelines (Secret Clearance)

Agile Defense, Inc. • Arlington (VA)

On-site
USD 110,000 - 160,000
Data Engineer — Secure Data Pipelines (Secret Clearance)
Data Engineer — Secure Data Pipelines (Secret Clearance)

Agile Defense, Inc. • Falls Church (VA)

On-site
USD 135,000 - 165,000