Hybrid Data Engineer - Databricks/Spark ETL & Governance

D2 Technologies; Herndon (VA); Híbrido; USD 100.000 - 150.000; Jornada completa

Descripción de la vacante

D2 Technologies in Herndon, VA is seeking an experienced Data Engineer to design and optimize enterprise-grade data platforms on the Databricks Lakehouse Platform. You will build scalable ETL/ELT pipelines, tune Spark and SQL performance, and support analytics and data transformation.

You will collaborate with data architects, software engineers, analysts, and stakeholders to ensure reliable data operations, governance, and security across enterprise platforms.

Formación

• Bachelor’s degree or higher in CS/IS/Engineering or related field.
• Strong programming experience with Spark (PySpark) and Spark SQL.
• Expert SQL skills; ability to write, optimize, and troubleshoot complex queries.
• Experience designing, developing, and supporting ETL/ELT pipelines.
• Hands-on Databricks experience and familiarity with Delta Lake and Unity Catalog.
• Experience with Git/version control and DevOps practices.

Responsabilidades

• Design, develop, and maintain scalable data engineering solutions using Databricks and Apache Spark.
• Build and optimize high-performance ETL/ELT pipelines for large data volumes.
• Develop, tune, and optimize complex SQL queries for analytics.
• Analyze requirements and translate them into scalable technical solutions.
• Troubleshoot production issues and optimize data pipeline performance.
• Ensure data quality, integrity, security, and governance across platforms.
• Collaborate with cross-functional teams and participate in code reviews.
• Create documentation for solutions, deployments, and operations.
• Support production releases and ongoing operations.

Conocimientos

Spark programming

SQL proficiency

Data modeling

ETL/ELT design

Git/version control

Communication

Problem solving

Security & governance

Educación

Bachelor’s degree in CS/IS/Engineering

Herramientas

Databricks

Databricks Lakehouse Platform

Apache Spark

PySpark

Spark SQL

SQL

ETL/ELT

Delta Lake

Git

Azure

AWS

Google Cloud Platform

Databricks Delta Live Tables

Unity Catalog

PL/SQL

CI/CD

Descripción del empleo

D2 Technologies in Herndon, VA is seeking an experienced Data Engineer to design and optimize enterprise-grade data platforms on the Databricks Lakehouse Platform. You will build scalable ETL/ELT pipelines, tune Spark and SQL performance, and support analytics and data transformation.

You will collaborate with data architects, software engineers, analysts, and stakeholders to ensure reliable data operations, governance, and security across enterprise platforms.