Databricks Lakehouse Architect - Lead Data Platform

Arkhya Tech Inc.; New York (NY); On-site; USD 180,000 - 230,000; Full time

Job summary

Arkhya Tech Inc. in New York seeks a Databricks Lakehouse Architect with hands-on data engineering experience designing and implementing production-grade Databricks solutions on AWS.

The candidate will lead enterprise data platform modernization and migration from legacy ETL to Databricks. You will architect scalable Lakehouse solutions (Bronze–Silver–Gold) with Delta Lake, Unity Catalog, and Workflows; design batch, streaming, and real-time pipelines using PySpark, Spark SQL, Auto Loader, and

Qualifications

• Hands-on experience with Databricks Lakehouse platforms in production environments.
• Proficiency in PySpark, Spark SQL, Delta Lake, and Unity Catalog.
• Experience migrating from EMR/Redshift/NiFi to Databricks on AWS.
• Knowledge of data governance, security, and lineage concepts.

Responsibilities

• Lead enterprise data platform modernization and Databricks migration strategy.
• Architect Lakehouse solutions using Bronze/Silver/Gold layers with Delta Lake.
• Design batch, streaming, CDC, and real-time pipelines with PySpark and Auto Loader.
• Define standards for governance, security, metadata, and observability.
• Drive CI/CD, DevOps, and IaC practices; mentor engineers and architects.
• Collaborate with business, analytics, and data science teams on the target state.

Skills

Databricks Lakehouse

Delta Lake

DLT

Unity Catalog

Databricks Workflows

Auto Loader

PySpark

Spark SQL

Python

AWS

CDC & Data Migration

Data Governance

CI/CD

Lakehouse Architecture

Streaming

Tools

Airflow

GitHub

Jenkins

Job description

Arkhya Tech Inc. in New York seeks a Databricks Lakehouse Architect with hands-on data engineering experience designing and implementing production-grade Databricks solutions on AWS.

The candidate will lead enterprise data platform modernization and migration from legacy ETL to Databricks. You will architect scalable Lakehouse solutions (Bronze–Silver–Gold) with Delta Lake, Unity Catalog, and Workflows; design batch, streaming, and real-time pipelines using PySpark, Spark SQL, Auto Loader, and