Senior Databricks Lakehouse Engineer for Government Systems

SiloSmashers; Arlington (VA); On-site; USD 120,000 - 180,000; Full time

Job summary

SiloSmashers, Inc. is seeking a Senior Data Engineer to architect and optimize Databricks Lakehouse pipelines for a government client.

You will design, implement, and maintain high-performance ingestion, transformation, and streaming workflows using PySpark, Delta Lake, and Unity Catalog governance. The role requires senior-level expertise with Databricks, cloud data engineering across AWS/Azure, and a focus on cost efficiency, performance tuning, and secure data platforms.

Qualifications

• 8+ years in data engineering or ETL within enterprise cloud environments.
• Experience with Databricks Lakehouse and Delta Lake.
• Proficient in Python and PySpark for distributed data pipelines.
• Ability to obtain/maintain a Public Trust clearance.

Responsibilities

• Act as principal technical authority for Databricks data engineering, pipelines, and Lakehouse standards across programs.
• Lead design reviews, roadmaps, and architectural decisions with federal SMEs and platform teams.
• Build and optimize batch and streaming pipelines using PySpark, Delta Lake, and related tools.
• Enforce medallion-architecture ingestion and governance, data lifecycle management, and data storage policies.
• Provide root-cause analysis and production support for Databricks jobs and pipelines.

Skills

Databricks Lakehouse

Delta Lake

PySpark

Python

Databricks Workflows

Structured Streaming

Cloud (AWS/Azure)

CI/CD pipelines

Terraform / IaC

DevSecOps

AI DevSecOps

Education

Bachelor's or Master's degree

Databricks Data Engineer Professional Certification

Cloud certification (AWS/Azure)

Tools

Databricks

PostgreSQL

Aurora

DynamoDB

GitHub Actions

Azure DevOps

Terraform

Job description

SiloSmashers, Inc. is seeking a Senior Data Engineer to architect and optimize Databricks Lakehouse pipelines for a government client.

You will design, implement, and maintain high-performance ingestion, transformation, and streaming workflows using PySpark, Delta Lake, and Unity Catalog governance. The role requires senior-level expertise with Databricks, cloud data engineering across AWS/Azure, and a focus on cost efficiency, performance tuning, and secure data platforms.