Databricks Data Architect

HCL Technologies Limited; Mississauga; On-site; CAD 160,000 - 210,000; Full time

Job summary

HCL Technologies Limited is seeking a Databricks Architect to design, build, and optimize scalable cloud-based data platforms and Lakehouse solutions. You will lead end-to-end architecture, migration, modernization, governance, and performance optimization across analytics, AI/ML, and BI initiatives.

The role requires deep hands-on experience with Databricks, PySpark, Delta Lake, Unity Catalog, and cloud platforms (AWS/GCP/Azure).

Qualifications

• 10+ years in Data Engineering / Data Architecture.
• 5+ years working with Databricks.
• Experience leading enterprise cloud modernization projects.
• Strong stakeholder management and client-facing experience.

Responsibilities

• Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Workflows
• Implement Delta Lake, Medallion Architecture (Bronze/Silver/Gold), and Unity Catalog
• Lead migration of legacy data warehouses and ETL platforms to Databricks
• Design batch and real-time streaming data pipelines
• Optimize Spark jobs for performance, scalability, and cost efficiency
• Implement CI/CD pipelines and DevOps automation
• Define data governance, security, lineage, and access control standards
• Collaborate with business stakeholders, data engineers, analysts, and leadership teams
• Provide technical leadership, code reviews, architecture reviews, and best practices
• Support AI/ML enablement using MLflow and Databricks capabilities
• Establish monitoring, observability, and operational support frameworks

Skills

Databricks

PySpark

SQL

Python

Delta Lake

Unity Catalog

AWS

GCP

Airflow

Git/DevOps

Tools

Databricks Workflows

Airflow

Terraform

Job description

We are seeking an experienced Databricks Architect to design, build, and optimize scalable cloud-based data platforms and Lakehouse solutions. The ideal candidate will have strong expertise in Databricks, Apache Spark, cloud platforms (AWS/Azure/GCP), data engineering, and modern analytics architecture.

The role involves leading end-to-end architecture, migration, modernization, governance, performance optimization, and implementation of enterprise data solutions supporting analytics, AI/ML, and business intelligence initiatives.

Key Responsibilities

• Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Workflows
• Implement Delta Lake, Medallion Architecture (Bronze/Silver/Gold), and Unity Catalog
• Lead migration of legacy data warehouses and ETL platforms to Databricks
• Design batch and real-time streaming data pipelines
• Optimize Spark jobs for performance, scalability, and cost efficiency
• Implement CI/CD pipelines and DevOps automation
• Define data governance, security, lineage, and access control standards
• Collaborate with business stakeholders, data engineers, analysts, and leadership teams
• Provide technical leadership, code reviews, architecture reviews, and best practices
• Support AI/ML enablement using MLflow and Databricks capabilities
• Establish monitoring, observability, and operational support frameworks

Skill Requirements

Technical Skills

• Strong hands-on experience with:
• Databricks
• PySpark
• SQL
• Python
• Experience with Delta Lake and Unity Catalog
• Expertise in data modeling and Lakehouse architecture
• Experience with cloud platforms:
• AWS
• GCP
• Knowledge of:
• Data Lakes
• Data Warehousing
• Streaming frameworks
• Git/DevOps
• Familiarity with orchestration tools:
• Airflow
• Databricks Workflows

Preferred Skills

• MLflow / AI & ML integration
• Kafka / Structured Streaming
• Snowflake integration
• Infrastructure as Code (Terraform)

Data governance and security frameworks

Other Requirements

• 10+ years in Data Engineering / Data Architecture
• 5+ years working with Databricks
• Experience leading enterprise cloud modernization projects
• Strong stakeholder management and client-facing experience

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 223,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending June 2026totaled $14.8billion.