Remote Databricks Data Engineer: Governance & Pipelines

Embedded Shishya; United States; Remote; USD 112,000 - 179,000; Full time

Job summary

Peraton is seeking a Databricks Data Engineer to build and operate a government-owned Databricks workspace and to consolidate agency data via robust ingestion pipelines. The role emphasizes Unity Catalog governance, MLflow integration, and end-to-end documentation for sustainment.

The position is 100% remote within the United States and requires US citizenship and a DoD Secret clearance. Experience with PySpark, Python, and Databricks workflows is essential for success.

Qualifications

• Minimum 8 years with BS/BA, minimum 6 years with MS/MA, minimum 3 years with PhD.
• US Citizenship required.
• Active DoD Secret clearance required.

Responsibilities

• Build out a dedicated Government Databricks workspace and data ingestion pipelines from core agency systems.
• Implement Unity Catalog for centralized governance, metadata management, and data lineage.
• Configure MLflow for ML workflows and maintain documentation to support sustainment.
• Review security controls for CUI/PII/PHI/financial data and implement improvements.
• Develop comprehensive as-built documentation including architecture diagrams, data dictionaries, and SOPs.
• Document data sources, integration methods, and architecture decisions to support sustainment.

Skills

PySpark

Spark SQL

Python

Unity Catalog

Data lineage

Airflow

Databricks Workflows

Delta Lake

Cloud platforms

IaC

CUI/PII handling

DoD 8570

US Citizenship

DoD Secret clearance

Databricks platform

Git/CI-CD

Education

BS/BA in CS or related field

MS/MA

PhD

Tools

Airflow

Databricks Workflows

Unity Catalog

Delta Lake

Metastore management

CI/CD tooling

Job description

Peraton is seeking a Databricks Data Engineer to build and operate a government-owned Databricks workspace and to consolidate agency data via robust ingestion pipelines. The role emphasizes Unity Catalog governance, MLflow integration, and end-to-end documentation for sustainment.

The position is 100% remote within the United States and requires US citizenship and a DoD Secret clearance. Experience with PySpark, Python, and Databricks workflows is essential for success.