Senior Data Engineer – Databricks & Cloud (Remote)

Data Surge; United States; On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Remote-first

PTO including holidays

401(k) with match

Growth opportunities

Job summary

Data Surge is seeking an experienced Data Engineer with Databricks and cloud expertise to design scalable data pipelines and automate infrastructure in secure cloud environments.

You'll work with data scientists, engineers, and cloud architects to build robust, end-to-end data solutions for federal programs and government-related projects, while embracing remote-first collaboration and secure practices.

Qualifications

• 5+ years in data engineering or backend data infrastructure roles.
• 2+ years hands-on with Databricks, Spark, or distributed data processing in production.
• Deep proficiency with PySpark and Python for data tasks.
• Experience with data modeling.
• Experience with Airflow, dbt, or other orchestration/transformations tools.
• Knowledge of data governance, lineage, or security best practices.
• Experience with Azure (preferred), AWS, or Google Cloud.
• Strong CI/CD and version control knowledge.
• Relational and NoSQL databases experience.
• Monitoring, debugging, and scaling cloud data systems.

Responsibilities

• Design, develop, and optimize robust data pipelines in Databricks.
• Collaborate with data scientists, engineers, and cloud architects to build secure data systems.
• Automate infrastructure provisioning using Terraform and manage resources on Azure/AWS/GCP.
• Monitor, troubleshoot, and support production-grade data pipelines and jobs.
• Develop CI/CD workflows for deployment and version control of data processing code.
• Ensure security, performance, and reliability across all data workflow stages.
• Leverage Docker containers to deploy data services as needed.
• Participate in Agile ceremonies and influence iterative data delivery plans.
• Support improvements in data architecture, automation, and governance.

Skills

Databricks

PySpark

Python

Data modeling

Airflow

dbt

CI/CD

Terraform

Docker

Kubernetes

Helm

Azure

AWS

GCP

PostgreSQL

MySQL

NoSQL

Security best practices

Tools

Databricks

Airflow

dbt

Terraform

Docker

Kubernetes

Helm

Job description

Data Surge is seeking an experienced Data Engineer with Databricks and cloud expertise to design scalable data pipelines and automate infrastructure in secure cloud environments.

You'll work with data scientists, engineers, and cloud architects to build robust, end-to-end data solutions for federal programs and government-related projects, while embracing remote-first collaboration and secure practices.