Lead Data Engineer - Databricks, PySpark & Dashboards

BigBear

Norfolk (VA)

On-site

USD 130,000 - 170,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

BigBear.ai seeks a Lead Data Engineer to design and operate data pipelines, analytics datasets, and reporting apps supporting defense and intelligence missions. You will leverage Databricks, Python, Spark, and SQL to prepare reliable data for analyses used in operational planning and senior-leader decisions.

You will collaborate with engineers, analysts, and mission stakeholders to translate requirements into data products, maintain dashboards, and ensure secure data access across platforms.

Qualifications

  • Active TS/SCI clearance is required.
  • Bachelor's degree in Information Systems, Information Technology, Computer Science, Engineering, or a related field.
  • Five+ years of experience supporting data or information systems through data engineering, analytics, database administration, or technical systems support.
  • Hands-on experience with Databricks, Python, Spark or PySpark, and SQL to transform, validate, or analyze data.
  • Experience maintaining data pipelines and developing reports, dashboards, or analytics applications.
  • Experience troubleshooting production data or application issues and administering user access in a secure environment.
  • Ability to work on-site full time in Norfolk, Virginia.

Responsibilities

  • Develop, maintain, and troubleshoot data pipelines and transformations in Databricks using Python, PySpark, and SQL.
  • Clean and validate source data, investigate discrepancies, and deliver datasets for operational reporting and analytics.
  • Build and maintain Qlik Sense applications, dashboards, and SQL reports for mission users.
  • Manage user access and permissions across data and analytics platforms in accordance with security and data governance requirements.
  • Support database upgrades, system migrations, and production troubleshooting in coordination with engineering and operations teams.
  • Document data workflows, reporting logic, and support procedures so other team members can maintain them.
  • Work with mission stakeholders to translate operational reporting needs into data products and application improvements.

Skills

Data engineering

Education

Bachelor's degree in Information Systems/IT/CS/Engineering

Tools

Databricks
Python
Spark/PySpark
SQL
Qlik Sense

Job description

BigBear.ai seeks a Lead Data Engineer to design and operate data pipelines, analytics datasets, and reporting apps supporting defense and intelligence missions. You will leverage Databricks, Python, Spark, and SQL to prepare reliable data for analyses used in operational planning and senior-leader decisions.

You will collaborate with engineers, analysts, and mission stakeholders to translate requirements into data products, maintain dashboards, and ensure secure data access across platforms.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer – Defense Analytics & Databricks
Lead Data Engineer – Defense Analytics & Databricks

BigBear.ai • Norfolk (VA)

On-site
USD 140,000 - 190,000
Lead Data Engineer (TS/SCI) - Databricks & DoD Ops
Lead Data Engineer (TS/SCI) - Databricks & DoD Ops

Bigbear.ai • Norfolk (VA)

On-site
USD 120,000 - 170,000
Lead Data Engineer — Databricks, PySpark, ETL (Secret)
Lead Data Engineer — Databricks, PySpark, ETL (Secret)

Redhorse Corporation • United States

Remote
USD 140,000 - 190,000
Lead Data Architect & Engineer - Databricks Lakehouse
Lead Data Architect & Engineer - Databricks Lakehouse

CloudBigD • Atlanta (GA)

On-site
USD 89,000 - 126,000
Lead Data Engineer & Databricks Architect for AI-Ready Data
Lead Data Engineer & Databricks Architect for AI-Ready Data

Celebal Technologies • Dallas (TX)

On-site
USD 180,000 - 240,000
Senior Data Engineer — Databricks, PySpark & Data Lakehouse
Senior Data Engineer — Databricks, PySpark & Data Lakehouse

Resolve Tech Solutions • Irving (TX)

On-site
USD 110,000 - 160,000
Lead Data Engineer — Databricks Lakehouse & AI Platform
Lead Data Engineer — Databricks Lakehouse & AI Platform

Brown & Brown, Inc. • Plano (TX)

On-site
USD 150,000 - 210,000
Health benefits
401(k) plan
Paid time off
+1
Junior Data Engineer - Cloud ETL & Data Pipelines
Junior Data Engineer - Cloud ETL & Data Pipelines

Bigbear.ai • Norfolk (VA)

On-site
USD 65,000 - 90,000
Lead Data Engineer - Databricks & Spark Expert
Lead Data Engineer - Databricks & Spark Expert

Link Group • United States

On-site
USD 140,000 - 200,000
Senior Data Engineer – Databricks & Pipelines Lead
Senior Data Engineer – Databricks & Pipelines Lead

Redhorse Corporation • United States

Remote
USD 170,000 - 190,000