Azure Data Platform Engineer – Databricks & Delta Lake

Compunnel Inc.; Mississauga; On-site; CAD 100,000 - 130,000; Full time

Job summary

Compunnel, Inc. is seeking an Azure Cloud Data Engineer responsible for designing and maintaining scalable cloud-based data platforms. Your role includes developing high-performance ETL pipelines and optimizing large-scale data processing solutions within Azure environments.

The ideal candidate will possess strong Python and SQL skills, along with a solid understanding of cloud data engineering principles and enterprise data management. Candidates with expertise in Databricks and experience in handling large datasets will be preferred.

Qualifications

• 9+ years of overall data engineering experience.
• 5+ years of hands-on Python development experience.
• 5+ years of SQL Server development experience working with large datasets.
• 5+ years of experience developing ETL pipelines using Databricks and PySpark.
• Experience designing and implementing cloud data architectures.

Responsibilities

• Design, develop, and maintain scalable data pipelines.
• Build and deploy Spark applications using Azure Databricks.
• Optimize Databricks jobs for performance.
• Monitor and resolve issues impacting data pipelines.
• Monitor and resolve issues impacting data pipelines.

Skills

Python

SQL

Databricks

ETL pipelines

Data modeling

Azure

Cloud architecture

Data governance

Education

Bachelor's degree in Computer Science or related field

Master's degree in Computer Science or related field

Tools

Azure Synapse

Snowflake

Amazon Redshift

Google BigQuery

Apache Airflow

Job description

Compunnel, Inc. is seeking an Azure Cloud Data Engineer responsible for designing and maintaining scalable cloud-based data platforms. Your role includes developing high-performance ETL pipelines and optimizing large-scale data processing solutions within Azure environments.

The ideal candidate will possess strong Python and SQL skills, along with a solid understanding of cloud data engineering principles and enterprise data management. Candidates with expertise in Databricks and experience in handling large datasets will be preferred.