Databricks Data Engineer - TS/SCI Cleared, Lakehouse Expert

Gridiron IT; Washington (District of Columbia); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k)

HSA/FSA

Disability & ADD insurance

Life insurance

Pet insurance

Job summary

Gridiron IT is seeking an experienced Databricks Data Engineer to design and maintain scalable data pipelines in a federal program. You will work with Databricks, Spark, Python, and SQL to enable analytics and operations across large, secure datasets.

Collaboration with data engineers, scientists, and stakeholders is essential, with emphasis on data quality and governance. You will build Delta Lake architectures, use CI/CD for pipelines, and apply best practices in security and information

Qualifications

• Bachelor's degree or equivalent professional experience in a technical field.
• Hands-on experience with Databricks and Spark.
• Proficiency in Python and SQL for data processing.
• Experience with large-scale distributed datasets and data pipelines.
• Knowledge of data modeling, warehousing, and lakehouse architectures.
• Familiarity with version control and CI/CD.
• Security and compliance awareness for government programs.
• DoD or Intelligence Community experience is a plus.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and Spark.
• Build ETL/ELT workflows for ingesting transforming data.
• Develop processing solutions using Python, PySpark, and SQL.
• Build and optimize Delta Lake / Lakehouse architectures.
• Integrate data from APIs, databases, and cloud storage.
• Implement data quality, governance, and monitoring.
• Optimize Spark workloads and Databricks jobs for performance.
• Enable CI/CD deployment of data pipelines.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Data modeling

Data warehousing

CI/CD practices

Git

Collaboration

Security/compliance

Education

Bachelor's degree in Computer Science, Engineering, IT, Data Science, or related field, or equivalent experience

Tools

Databricks Delta Lake

Unity Catalog

Databricks Workflows/Jobs

AWS (S3, Glue, Lambda, Redshift, EMR, IAM)

Azure (Data Lake Storage, Data Factory, Synapse, DevOps)

Airflow

Terraform

Docker

Kubernetes

Job description

Gridiron IT is seeking an experienced Databricks Data Engineer to design and maintain scalable data pipelines in a federal program. You will work with Databricks, Spark, Python, and SQL to enable analytics and operations across large, secure datasets.

Collaboration with data engineers, scientists, and stakeholders is essential, with emphasis on data quality and governance. You will build Delta Lake architectures, use CI/CD for pipelines, and apply best practices in security and information