Senior Data Engineer - Databricks

Steampunk, Inc.; McLean (VA); On-site; USD 140,000 - 180,000; Full time

Job summary

Steampunk is seeking a seasoned Senior Data Engineer to architect and implement enterprise-grade data platforms, lakes, and lakehouses using Databricks. You will collaborate with clients to design scalable pipelines and optimize data workflows across cloud environments (AWS, Azure, GCP) and modern data tools.

You should bring 5–7 years of data engineering experience, strong SQL skills, and hands-on mastery of Databricks, Spark, and orchestration tools.

Qualifications

• U.S. government security clearance ability is required.
• Bachelor's degree in a related technical discipline.
• 5–7 years of industry experience with production software and data engineering.
• Strong experience with Databricks, Spark, Delta Lake, and data pipelines.
• Experience with SQL and relational databases; performance optimization.
• Experience with modern cloud platforms (AWS, Azure, GCP) and data lakehouse architectures.
• Experience with SDLC, CI/CD, and Agile environments.

Responsibilities

• Lead design, development, and migration of data lakes and lakehouses with Databricks.
• Assess ETL jobs, data marts, BI reports and data dependencies.
• Develop scalable data pipelines and processing on Databricks.
• Address questions on customization, enterprise architecture and data products.
• Build and support data warehouse, data lake, and lakehouse solutions in cloud environments.
• Apply Databricks, SQL, PySpark/Python and cloud tech to data engineering tasks.
• Support Agile software development lifecycle.
• Contribute to AI & Data Exploitation Practice growth.

Skills

Databricks

SQL

PySpark/Python

Java

Scala

Cloud platforms (AWS/Azure/GCP)

Data governance & SDLC

Agile

Data modeling & ETL design

Communication / customer service

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or related technical discipline

Tools

Databricks

Apache Spark

Delta Lake

Airflow

AWS

Azure

GCP

Python

Java

C++

Shell/Scripting

Job description

We are looking for a seasoned Senior Data Engineer to work with our team and clients to develop enterprise‑grade data platforms, data lakes and lakehouses, services, and pipelines using Databricks. We are looking for more than just a Senior Data Engineer; we are seeking a technologist with excellent communication and customer service skills and a passion for data and problem solving.

Contributions

• Lead and support the design, development, and migration of data lake and lakehouse environments using Databricks, with a focus on performance, reliability, and scalability.
• Assess and understand existing ETL jobs, workflows, data marts, BI tools, reports, and associated data dependencies.
• Design, develop, and optimize scalable data pipelines and data processing solutions using Databricks.
• Address technical inquiries concerning customization, integration, enterprise architecture, and the features and functionality of data products.
• Develop and support database, data warehouse, data mart, data lake, and lakehouse solutions in cloud environments such as AWS, Azure, or GCP.
• Apply hands‑on expertise with Databricks, SQL, PySpark/Python, and cloud technologies to support data engineering solutions.
• Support an Agile software development lifecycle.
• Contribute to the continued growth of our AI & Data Exploitation Practice.

Qualifications

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor's degree in computer science, information systems, engineering, or a related technical discipline.
• 5–7 years of industry experience developing production software and solving complex technical problems.
• 5–7 years of direct experience in Data Engineering, including experience with tools and technologies such as:
• Big data technologies: Databricks, Apache Spark, Delta Lake, or similar.
• Relational SQL: preferably T‑SQL; alternatively PostgreSQL or MySQL.
• Data pipeline and workflow management tools: Databricks Workflows, Airflow, AWS Step Functions, or similar.
• Cloud services: Databricks on AWS, S3, EC2, RDS, or Azure equivalents.
• Object‑oriented or scripting languages: PySpark/Python, Java, C++, Scala, or similar.
• Data lakehouse architecture and technologies such as Delta Lake or Apache Iceberg.
• Advanced SQL knowledge and experience working with relational databases, including query development and optimization, as well as familiarity with a variety of database technologies.
• Experience manipulating, processing, and extracting value from large, disparate datasets.
• Ability to assess existing data pipelines, understand their purpose and functionality, and efficiently reimplement or modernize them using Databricks.
• Experience working with structured and unstructured data.
• Experience architecting data systems, including transactional systems and data warehouses.
• Experience with the software development lifecycle (SDLC), CI/CD practices, and development, test, and production environments.
• Experience with data cataloging tools such as Informatica EDC, Unity Catalog, Collibra, Alation, Microsoft Purview, or Amazon DataZone is a plus.
• Demonstrated commitment to data governance and data management best practices.
• Experience working in an Agile environment.
• Experience supporting project teams of developers and data scientists building web‑based interfaces, dashboards, reports, analytics solutions, and machine learning models.

About Steampunk

Steampunk relies on several factors to determine salary, including but not limited to geographic location, contractual requirements, education, knowledge, skills, competencies, and experience. The projected compensation range for this position is $140,000 to $180,000. The estimate displayed represents a typical annual salary range for this position. Annual salary is just one aspect of Steampunk’s total compensation package for employees. Learn more about additional Steampunk benefits here.

Identity Statement

As part of the application process, you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

Steampunk is a Change Agent in the Federal contracting industry, bringing new thinking to clients in the Homeland, Federal Civilian, Health and DoD sectors. Through our Human-Centered delivery methodology , we are fundamentally changing the expectations our Federal clients have for true shared accountability in solving their toughest mission challenges. If you want to learn more about our story, visit http://www.steampunk.com.

We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability status, protected veteran status, or any other characteristic protected by law. Steampunk participates in the E-Verify program.