Databricks architect - HCLTech

OpenTalent

Toronto

On-site

CAD 120,000 - 160,000

Full time

3 days ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

OpenTalent is seeking an experienced data architect to design and own a Databricks Lakehouse on Azure, implementing Delta Lake, Unity Catalog, and medallion architecture. Hands-on role includes building PySpark pipelines, Delta Live Tables, notebooks, and orchestration to set the engineering bar.

You will lead the migration of complex legacy ETL workloads (DataStage) to Databricks/PySpark and Azure Data Factory, aligning with enterprise standards and best practices.

Qualifications

Experience architecting Databricks Lakehouse on Azure
Track record of designing scalable, secure data platforms

Responsibilities

Architect scalable, secure Databricks Lakehouse on Azure with Delta Lake, Unity Catalog, and medallion architecture
Lead migration of legacy ETL (DataStage) workloads to Databricks/PySpark and ADF

Skills

Databricks

Delta Lake

Unity Catalog

PySpark

Spark SQL

Delta Live Tables

Azure Data Factory

ADLS Gen2

ETL Migration

DataStage

Tools

Azure Databricks

ADF

Job description

Architect the lakehouse: Design and own scalable, secure Databricks Lakehouse architecture on Azure (Delta Lake, Unity Catalog, medallion bronze/silver/gold, ADLS Gen2) aligned to enterprise standards. Stay hands-on: Personally build and review PySpark / Spark SQL pipelines, Delta Live Tables, notebooks, and orchestration — setting the engineering bar, not just directing it.

Lead legacy migration: Drive conversion of complex legacy ETL (DataStage) workloads to Databricks/PySpark and ADF, incl

Get your free, confidential resume review.

or drag and drop your file here.