Data Engineer - Databricks & Cloud for Federal Programs

Data Surge; Arlington (VA); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote-first work environment

PTO – Holidays + Your Birthday

401(k) with 4% Match

Growth and development opportunities

Job summary

Data Surge is seeking a seasoned Data Engineer with deep Databricks and cloud experience to design and implement scalable data solutions for federal services. In this role, you will build and optimize pipelines, automate infrastructure with Terraform, and ensure secure, reliable data workflows in a hybrid on-site environment in Arlington, VA.

Ideal candidates have 5+ years in data engineering, strong PySpark/Python skills, and hands-on experience with Databricks, Spark, Airflow, and Terraform

Qualifications

• 5+ years of data engineering or backend data infra experience.
• Deep Databricks, Spark, or distributed data processing experience.
• Proficiency in PySpark and Python for data tasks.

Responsibilities

• Design, develop, and optimize data pipelines in Databricks.
• Collaborate with data scientists and cloud architects to build secure data systems.
• Automate infrastructure provisioning using Terraform and manage resources on Azure/AWS/GCP.
• Monitor, troubleshoot, and support production data pipelines.
• Develop CI/CD workflows for data processing code.
• Ensure security, performance, and reliability across data workflows.
• Utilize Docker containers for deploying data services.
• Participate in Agile ceremonies and iterative delivery planning.
• Support data governance and data lakehouse concepts.

Skills

Data engineering

Python PySpark

Cloud experience

CI/CD

Terraform

Docker

Kubernetes

Spark

SQL/NoSQL

Security/compliance

Education

Databricks Certification

Tools

Databricks

Spark

Airflow

dbt

Terraform

Docker

Kubernetes

Helm

Delta Lake

Azure

AWS

GCP

PostgreSQL

MySQL

Job description

Data Surge is seeking a seasoned Data Engineer with deep Databricks and cloud experience to design and implement scalable data solutions for federal services. In this role, you will build and optimize pipelines, automate infrastructure with Terraform, and ensure secure, reliable data workflows in a hybrid on-site environment in Arlington, VA.

Ideal candidates have 5+ years in data engineering, strong PySpark/Python skills, and hands-on experience with Databricks, Spark, Airflow, and Terraform