Security Data Engineering Lead | Azure & Spark

ODAIA; Toronto; On-site; CAD 100,000 - 125,000; Full time

Job summary

A leading financial institution in Toronto is seeking a skilled data engineer to design and optimize data pipelines using Azure Databricks. You will leverage your 8+ years of experience in the field, leading a team and implementing best practices in data governance and cloud services. The ideal candidate will have strong knowledge in Databricks and Azure, and excel in collaboration and mentorship within a fast-paced environment. This position offers competitive compensation and opportunities for professional growth.

Benefits

Comprehensive Total Rewards Program

Competitive compensation

Flexible work/life balance options

Opportunities for professional development

Qualifications

• 8+ years of experience in data engineering delivering business-critical software solutions.
• Strong expertise in Databricks and Delta Lake.
• Proficiency in Azure Cloud Services.
• Solid understanding of Spark and PySpark for big data processing.

Responsibilities

• Design end-to-end data pipelines in Azure Databricks.
• Develop and optimize ELT/ELT workflows ensuring data integrity.
• Collaborate with cross-functional teams for data requirements.
• Provide mentorship and technical guidance to junior engineers.
• Implement infrastructure-as-code solutions using Terraform.

Skills

Big Data Management

Cloud Computing

Database Development

Data Mining

Data Warehousing (DW)

ETL Processing

Group Problem Solving

Quality Management

Requirements Analysis

Education

Bachelor’s or Master’s degree in Computer Science or Data Engineering

Tools

Databricks

Azure Cloud Services

Spark

PySpark

Terraform

Kubernetes

Docker

Job description

A leading financial institution in Toronto is seeking a skilled data engineer to design and optimize data pipelines using Azure Databricks. You will leverage your 8+ years of experience in the field, leading a team and implementing best practices in data governance and cloud services. The ideal candidate will have strong knowledge in Databricks and Azure, and excel in collaboration and mentorship within a fast-paced environment. This position offers competitive compensation and opportunities for professional growth.