Databricks Engineer - Azure Data Pipelines & Lakehouse

Compunnel Inc.

Mississauga

In loco

CAD 120.000 - 150.000

Tempo pieno

14 giorni+

Aumenta le possibilità di ricevere una risposta da questo recruiter

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Descrizione del lavoro

A leading technology company in Canada seeks a Databricks Engineer. The role involves designing and optimizing large-scale data pipelines using Databricks and Azure, requiring over 10 years of experience in data engineering and strong expertise in Spark, PySpark, and Databricks. You'll collaborate with teams to provide scalable data solutions and ensure compliance with data governance. This position provides an opportunity to work on impactful projects in the financial sector.

Competenze

  • 10+ years of experience in data engineering.
  • Experience in financial services, banking, or capital markets.
  • Strong expertise in Databricks and Azure cloud services.

Mansioni

  • Design and develop large-scale data pipelines.
  • Optimize Databricks workloads for performance.
  • Implement data governance and compliance frameworks.

Conoscenze

Databricks
Spark/PySpark
Azure cloud services
Data governance
Python programming

Strumenti

Databricks Asset Bundles
GitLab

Descrizione del lavoro

The Databricks Engineer will design, develop, and optimize large-scale data pipelines using Databricks and Azure cloud services.

This role requires advanced expertise in Spark/PySpark, Delta Lake technologies, data governance, and cloud-based data engineering.

The engineer will collaborate with cross-functional teams to deliver scalable, reliable, and high-performance data solutions that support analytics, BI, and business initiatives.

Key Responsibilities

  • Build and maintain scalable ETL/ELT data pipelines using Databricks.
  • Use PySpark/Spark and SQL to process, cleanse, and transform large datasets.
  • Integrate data from diverse sources such as Azure Blob Storage, ADLS, and relational/non-relational databases.
  • Work with teams across the organization to prepare data for BI dashboards and reporting tools.
  • Partner with business stakeholders to understand requirements and deliver tailored data solutions.
Performance & Optimization
  • Optimize Databricks workloads for performance, scalability, and cost efficiency.
  • Monitor, troubleshoot, and resolve issues in data pipelines to ensure accuracy and reliability.
Governance & Security
  • Implement data governance, access controls, and security frameworks using Unity Catalog.
  • Ensure compliance with enterprise data policies and regulatory standards.
Deployment
  • Use Databricks Asset Bundles for deploying jobs, notebooks, and configurations across environments.
  • Maintain version control for Databricks artifacts and collaborate with the team on development best practices.
Required Qualifications
  • 10+ years of overall experience in data engineering.
  • Recent experience in financial services, banking, or capital markets.
  • Strong expertise in Databricks, including:
  • Unity Catalog
  • Lakehouse Architecture
  • Table Triggers
  • Databricks Runtime
  • Proficiency with Azure cloud services.
  • Deep understanding of Spark and PySpark for big data processing.
  • Strong programming skills in Python.
  • Experience with relational databases.
  • Knowledge of Databricks Asset Bundles and GitLab for version control.
Preferred Qualifications
  • Familiarity with advanced Databricks Runtime configurations.
  • Experience with streaming frameworks such as Spark Streaming.
Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Databricks Engineer
Databricks Engineer

Compunnel Inc. • Mississauga

In loco
CAD 120.000 - 150.000
Senior Databricks Engineer
Senior Databricks Engineer

KData AI • Toronto

In loco
CAD 130.000 - 180.000
Databricks Architect
Databricks Architect

Celebal Technologies • Canada

In loco
CAD 150.000 - 190.000
Senior Databricks Engineer
Senior Databricks Engineer

KData Inc. • Toronto

In loco
CAD 170.000 - 210.000
Databricks Architect
Databricks Architect

Acunor • Canada

In loco
CAD 100.000 - 130.000
Databricks Architect
Databricks Architect

TechDoQuest • Toronto

In loco
CAD 100.000 - 130.000
Azure Data Bricks Consultant
Azure Data Bricks Consultant

Compunnel Inc. • Toronto

In loco
CAD 110.000 - 170.000
Azure Databricks Data Architect
Azure Databricks Data Architect

N2P Systems • Toronto

In loco
CAD 180.000 - 230.000
Azure Cloud Data Engineer
Azure Cloud Data Engineer

Compunnel Inc. • Mississauga

In loco
CAD 100.000 - 130.000
Databricks Data Engineer
Databricks Data Engineer

Rubicon Path • Toronto

In loco
CAD 80.000 - 110.000