Senior Databricks Engineer: Data Lakehouse & Pipelines

CIBC; Toronto; Hybrid; CAD 110,000 - 160,000; Full time

Benefits offered by this job

Competitive salary

Defined benefit pension plan

Employee share purchase plan

Job summary

CIBC is seeking a Senior Consultant, Databricks Developer, to lead design, development, and optimization of data engineering solutions using Azure Databricks, Apache Spark, and cloud services. You will translate analytics needs into reliable technical solutions, support batch and near real-time processing, and collaborate with architects, analysts, DevOps partners, and stakeholders.

You will mentor junior engineers, contribute to delivery across the full development lifecycle, and thrive in a

Qualifications

• Strong hands-on Databricks and Spark expertise in cloud environments.
• Ability to translate business needs into scalable data solutions with secure practices.
• Experience building and optimizing data pipelines (ETL/ELT) and data models.
• Knowledge of governance, lineage, and security controls in data platforms.
• Experience mentoring developers and contributing to delivery leadership and agile planning.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, Spark, PySpark, Python, and SQL.
• Translate business requirements into practical data solutions and contribute to architecture decisions.
• Tune and optimize Spark jobs, Databricks clusters, notebooks, and workflows for performance and cost efficiency.
• Implement data quality, governance, and security controls across pipelines.
• Lead delivery with architects, analysts, and stakeholder teams, and drive production readiness.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Tools

Azure Data Factory

GitHub

Airflow

PowerShell

Databricks tooling

Job description

CIBC is seeking a Senior Consultant, Databricks Developer, to lead design, development, and optimization of data engineering solutions using Azure Databricks, Apache Spark, and cloud services. You will translate analytics needs into reliable technical solutions, support batch and near real-time processing, and collaborate with architects, analysts, DevOps partners, and stakeholders.

You will mentor junior engineers, contribute to delivery across the full development lifecycle, and thrive in a