Senior Databricks Data Engineer - Lakehouse Pipelines

Steampunk; McLean (VA); Hybrid; USD 140,000 - 180,000; Full time

Job summary

Steampunk is seeking a seasoned Senior Data Engineer to join our McLean, VA team. You will design and implement enterprise-grade data platforms, data lakes and lakehouses using Databricks, focusing on performance and scalability.

Collaborate with clients and cross-functional teams to migrate ETL jobs, build scalable data pipelines, and apply data governance practices in cloud environments (AWS/Azure/GCP). Strong communication is required.

Qualifications

• Bachelor's degree in computer science, information systems, engineering, or a related technical discipline.
• 5–7 years of industry experience developing production software and solving complex technical problems.
• 5–7 years of direct experience in Data Engineering with Databricks, Spark, Delta Lake, and related tools.
• Advanced SQL knowledge; familiarity with PostgreSQL/MySQL.
• Experience with data pipelines and orchestration tools (Databricks Workflows, Airflow, AWS Step Functions).
• Experience with cloud services (Databricks on AWS, S3, EC2, RDS; Azure equivalents).
• Proficiency in PySpark/Python, Java, or Scala; familiarity with data lakehouse architectures.

Responsibilities

• Lead design, development, and migration of data lake and lakehouse environments using Databricks.
• Assess and optimize ETL jobs, workflows, data marts, BI tools and data dependencies.
• Develop and optimize scalable data pipelines and processing solutions in the cloud.
• Address inquiries on customization, integration, and enterprise architecture of data products.
• Support data warehouse, data mart, data lake and lakehouse solutions in cloud environments.
• Apply hands-on Databricks, SQL, PySpark/Python and cloud tech to data engineering solutions.
• Support an Agile software development lifecycle.
• Contribute to growth of AI & Data Exploitation Practice.

Skills

Databricks

Apache Spark

Delta Lake

SQL

PySpark/Python

Cloud platforms

Airflow

CI/CD

Data governance

Agile

Education

Bachelor's degree in computer science or related

Tools

Databricks Workflows

Airflow

Informatica EDC

Unity Catalog

Collibra

Alation

Microsoft Purview

Amazon DataZone

Job description

Steampunk is seeking a seasoned Senior Data Engineer to join our McLean, VA team. You will design and implement enterprise-grade data platforms, data lakes and lakehouses using Databricks, focusing on performance and scalability.

Collaborate with clients and cross-functional teams to migrate ETL jobs, build scalable data pipelines, and apply data governance practices in cloud environments (AWS/Azure/GCP). Strong communication is required.