Data Engineer

Data Surge; Arlington (VA); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote-first work environment

PTO – Holidays + Your Birthday

401(k) with 4% Match

Growth and development opportunities

Job summary

Data Surge is seeking a seasoned Data Engineer with deep Databricks and cloud experience to design and implement scalable data solutions for federal services. In this role, you will build and optimize pipelines, automate infrastructure with Terraform, and ensure secure, reliable data workflows in a hybrid on-site environment in Arlington, VA.

Ideal candidates have 5+ years in data engineering, strong PySpark/Python skills, and hands-on experience with Databricks, Spark, Airflow, and Terraform

Qualifications

• 5+ years of data engineering or backend data infra experience.
• Deep Databricks, Spark, or distributed data processing experience.
• Proficiency in PySpark and Python for data tasks.

Responsibilities

• Design, develop, and optimize data pipelines in Databricks.
• Collaborate with data scientists and cloud architects to build secure data systems.
• Automate infrastructure provisioning using Terraform and manage resources on Azure/AWS/GCP.
• Monitor, troubleshoot, and support production data pipelines.
• Develop CI/CD workflows for data processing code.
• Ensure security, performance, and reliability across data workflows.
• Utilize Docker containers for deploying data services.
• Participate in Agile ceremonies and iterative delivery planning.
• Support data governance and data lakehouse concepts.

Skills

Data engineering

Python PySpark

Cloud experience

CI/CD

Terraform

Docker

Kubernetes

Spark

SQL/NoSQL

Security/compliance

Education

Databricks Certification

Tools

Databricks

Spark

Airflow

dbt

Terraform

Docker

Kubernetes

Helm

Delta Lake

Azure

AWS

GCP

PostgreSQL

MySQL

Job description

Due to the nature of the role, this position requires U.S. Citizenship. It also requires being onsite in Arlington VA, 2-3 days/week.

Data Surge is disrupting the services industry with cutting-edge technology that brings together the very best consultants from the big data, machine learning, and software modernization space.

At Data Surge, we value collaboration, curiosity, and being customer centric. Our team thrives on collective efforts, innovation, and a commitment to wellness, ensuring employees are empowered to thrive professionally and personally.

About the Role

We are seeking a skilled and motivated Data Engineer with deep Databricks and cloud experience to support our Federal Services team. In this role, you will help design and implement scalable data solutions, build and optimize pipelines, and automate infrastructure within secure cloud environments.

This role is ideal for experienced developers who enjoy tackling complex data engineering challenges, optimizing performance, and contributing to impactful government programs. You'll help shape robust, secure, and efficient pipelines that serve critical public-sector missions.

Day-to-Day Impact

• Design, develop, and optimize robust data pipelines in Databricks to process and transform large-scale data sets
• Collaborate with data scientists, engineers, and cloud architects to build reliable, secure data systems
• Automate infrastructure provisioning using Terraform and manage resources on Azure (or AWS/GCP)
• Monitor, troubleshoot, and support production-grade data pipelines and jobs
• Develop CI/CD workflows to streamline deployment and version control of data processing code
• Ensure security, performance, and reliability across all stages of data workflow
• Utilize and manage Docker containers where needed to deploy data services
• Participate in Agile team ceremonies and help shape iterative data delivery plans
• Support continuous improvement in data architecture, automation, and governance practices
• Other duties as reasonably required
• 5+ years of experience in data engineering or backend data infrastructure roles
• 2+ years of hands-on experience with Databricks, Spark, or distributed data processing systems in a production environment
• Deep proficiency with PySpark and Python for data engineering tasks
• Experience with data modeling
• Solid experience with Airflow, dbt, or other orchestration and transformation tools
• Proven skills in data governance, lineage, or security best practices
• Experience working with Azure (preferred), AWS, or Google Cloud
• Strong working knowledge of CI/CD tools and version control workflows
• Comfortable working with relational (e.g., PostgreSQL, MySQL) and NoSQL databases
• Proven experience in monitoring, debugging, and scaling cloud-based data systems
• Strong communication skills and ability to work independently in a remote team environment
• Experience with Kubernetes and Helm for deploying data workloads (not required)
• Solid experience with Terraform or other Infrastructure as Code tools
• Familiarity with containerization using Docker
• Knowledge of data lakehouse architectures and Delta Lake
• Experience working on Federal or government projects
• Databricks Certification

Learn More

Data Surge offers a competitive compensation and total rewards package.

• Remote-first work environment
• PTO – Including holidays + your birthday
• 401(k) with 4% Match, immediately vested
• Growth and development opportunities

Find out more about our story at www.datasurge.com.

Visit our Careers page to explore our openings at https://www.datasurge.com/careers/.

Follow us on LinkedIn at https://www.linkedin.com/company/data-surge-llc.

As a participator in E-Verify, Data Surge electronically confirms employment eligibility of employees to work in the United States.