Databricks Big Data Architect - Enterprise Lakehouse Lead

Hyrhub; New York (NY); On-site; USD 150,000 - 180,000; Full time

Job summary

Hyrhub is seeking a highly skilled Databricks SME to lead the architecture and implementation of enterprise-scale Big Data solutions on the Databricks Lakehouse Platform. The ideal candidate will have 12+ years of architecture experience and proven expertise in migrating data from legacy platforms like Alteryx.

This critical role is focused on driving data modernization initiatives and ensuring robust data governance across multiple systems. Proficiency in Python, SQL, PySpark, and strong knowledge of CI/CD practices is essential.

Qualifications

• 12+ years of experience in architecting Big Data solutions with Databricks.
• Proven track record in migrating pipelines from ETL tools like Alteryx.
• Expertise in building streaming data pipelines.

Responsibilities

• Architect enterprise Big Data solutions on the Databricks platform.
• Migrate data from legacy systems to Databricks.
• Implement CI/CD practices for data pipelines.

Skills

Databricks Lakehouse architecture

Data migration from Alteryx

Streaming pipeline design

CI/CD practices

Python

SQL

PySpark

Tools

Azure DevOps

GitHub Actions

Data quality tools

Job description

Hyrhub is seeking a highly skilled Databricks SME to lead the architecture and implementation of enterprise-scale Big Data solutions on the Databricks Lakehouse Platform. The ideal candidate will have 12+ years of architecture experience and proven expertise in migrating data from legacy platforms like Alteryx.

This critical role is focused on driving data modernization initiatives and ensuring robust data governance across multiple systems. Proficiency in Python, SQL, PySpark, and strong knowledge of CI/CD practices is essential.