Databricks Data Engineering Lead — Lakehouse & Genie Expert

Robotic Technologies Automation; City of Albany (NY); Vor Ort; USD 99.000 - 141.000; Vollzeit

Zusammenfassung

Robotic Technologies Automation in Albany, NY seeks an experienced Databricks Data Engineering Lead to guide the design and deployment of scalable data pipelines. The role requires hands-on expertise with Databricks, Python, PySpark, Delta Lake, and Lakehouse architectures, including governance and security considerations.

AWS experience is highly desirable for Databricks deployments. The candidate will mentor engineers, drive engineering best practices, and collaborate with architects and

Qualifikationen

• 8+ years in Data Engineering / Data Platforms.
• Hands-on Databricks experience.
• Proficient in Python.
• PySpark expertise and data pipelines.
• Delta Lake and Lakehouse architecture experience.

Aufgaben

• Lead design and implementation of enterprise-scale data engineering on Databricks.
• Design and develop scalable ETL/ELT pipelines using Databricks, Python and PySpark.
• Provide hands-on technical leadership for Databricks Genie implementations.
• Design scalable data architectures leveraging Delta Lake and Lakehouse patterns.
• Mentor data engineers and enforce engineering standards.
• Support production deployments and performance tuning.

Kenntnisse

Databricks

Python

PySpark

Delta Lake

Lakehouse

SQL

Data Modeling

Data Pipelines

Tools

Databricks Genie

AWS

Jobbeschreibung

Robotic Technologies Automation in Albany, NY seeks an experienced Databricks Data Engineering Lead to guide the design and deployment of scalable data pipelines. The role requires hands-on expertise with Databricks, Python, PySpark, Delta Lake, and Lakehouse architectures, including governance and security considerations.

AWS experience is highly desirable for Databricks deployments. The candidate will mentor engineers, drive engineering best practices, and collaborate with architects and