Databricks Data Engineer — Modernize Federal Data Analytics

i4DM; Millersville (MD); On-site; USD 120,000 - 160,000; Full time

Job summary

i4DM is seeking a hands-on Databricks Engineer to design, build, and operate scalable data and analytics solutions on the Databricks Lakehouse platform for federal mission needs. You will work with Spark, Delta Lake, Unity Catalog, and medallion architecture to deliver secure, analytics-ready data and enable ML workflows.

The role requires strong Python/SQL skills, experience with ETL/ELT, and collaboration with cross-functional teams in a compliance-driven environment.

Qualifications

• Bachelor's degree in CS/IT/Engineering or equivalent experience.
• 4+ years in data or analytics engineering.
• 2+ years with the Databricks platform.
• Proficiency in Spark, PySpark, and Spark SQL.
• Experience with Delta Lake and Unity Catalog in production.
• Strong Python and SQL for data engineering.
• Experience with ETL/ELT and data pipeline orchestration.
• Familiar with cloud platforms (AWS, Azure, or Google Cloud).
• Experience integrating data solutions with CI/CD and Git.
• Ability to obtain and maintain a Public Trust.
• Excellent analytical, problem-solving, and communication skills.

Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines.
• Develop Delta Lake tables using medallion architecture for analytics-ready data.
• Implement real-time data ingestion with Spark Structured Streaming and Kafka.
• Configure Databricks clusters, jobs, and workflows in production.
• Apply data governance and Unity Catalog security controls.
• Collaborate with data scientists and analysts on data models and ML use cases.
• Support MLflow model lifecycle management and AI use cases.
• Optimize data workflows for performance, reliability, and cost.

Skills

Apache Spark

PySpark

Spark SQL

Python

SQL

Cloud platforms

Agile

Data governance

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks Platform

Delta Lake

Unity Catalog

Kafka

CI/CD

Git

Spark Structured Streaming

Job description

i4DM is seeking a hands-on Databricks Engineer to design, build, and operate scalable data and analytics solutions on the Databricks Lakehouse platform for federal mission needs. You will work with Spark, Delta Lake, Unity Catalog, and medallion architecture to deliver secure, analytics-ready data and enable ML workflows.

The role requires strong Python/SQL skills, experience with ETL/ELT, and collaboration with cross-functional teams in a compliance-driven environment.