Databricks Data Engineering Lead: Lakehouse & Genie Expert

Quantum Technologies. LLC; City of Albany (NY); Presencial; USD 102.000 - 138.000; Jornada completa

Descripción de la vacante

Quantum Technologies. LLC seeks an experienced Databricks Data Engineering Lead in Albany, NY for an 8-month engagement. The role focuses on designing and delivering scalable data solutions with Databricks, Python, PySpark, and Delta Lake, with Genie implementations as needed.

Strong leadership and AWS data services exposure are highly desirable. The candidate will guide architecture, mentor engineers, and ensure secure, high-performance data pipelines across the enterprise.

Formación

• 8+ years of data engineering experience.
• Hands-on Databricks with Python/PySpark.
• Experience with Delta Lake and Lakehouse architectures.
• Strong SQL and data modeling knowledge.
• Experience leading technical teams and driving standards.
• Proficiency with Databricks Genie and AWS data services.

Responsabilidades

• Lead the design, development, and implementation of enterprise-scale data engineering solutions on Databricks.
• Provide hands-on technical leadership and mentorship to the engineering team.
• Develop scalable ETL/ELT pipelines using Databricks, Python, and PySpark.
• Design and optimize Delta Lake-based architectures and Lakehouse patterns.
• Ensure data quality, governance, security, and production reliability across pipelines.
• Support production deployments and performance tuning.
• Contribute to architecture reviews, design reviews, and engineering standards.

Conocimientos

Databricks

Python

PySpark

Delta Lake

Lakehouse

SQL

Data modeling

ETL/ELT pipelines

Data quality

Data governance

Production support

Mentoring

Herramientas

Databricks Genie

AWS

CI/CD

Descripción del empleo

Quantum Technologies. LLC seeks an experienced Databricks Data Engineering Lead in Albany, NY for an 8-month engagement. The role focuses on designing and delivering scalable data solutions with Databricks, Python, PySpark, and Delta Lake, with Genie implementations as needed.

Strong leadership and AWS data services exposure are highly desirable. The candidate will guide architecture, mentor engineers, and ensure secure, high-performance data pipelines across the enterprise.