Tech Lead - Azure Databricks Developer

United Software Group Inc. - Canada; Canada; Remote; CAD 120,000 - 150,000; Full time

Job summary

A leading software development company is seeking an Azure Databricks technical lead for remote work based in Halifax, NS. The ideal candidate will have over 10 years of experience and expertise in designing data ingestion pipelines and developing scalable frameworks. This role requires a hands-on approach as a developer alongside technical leadership duties.

Qualifications

• 10+ years of experience in data engineering or related fields.
• Strong expertise in designing data ingestion pipelines.
• Hands-on development experience with Azure Databricks.

Responsibilities

• Design and implement data ingestion pipelines.
• Develop scalable frameworks for varied data sets.

Skills

Azure Databricks

Apache Pyspark

Tools

Azure Data Factory

ADLS/Key Vaults

Job description

Overview

Work location : Halifax, NS (Permanent remote)

Note : This position would serve the team as Azure Databricks technical lead rather than a team lead so more than a people mentor, this position is for a technical expert. They would also be expected to contribute as a hands on Azure Databricks developer. Please look for 10+ years experienced candidates.

Job Description

• Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure Databricks.
• Developing scalable and re-usable frameworks for ingesting of varied data sets and data formats.

Skills

• Necessary: Strong expertise with Azure Databricks, Apache Pyspark
• Nice to Have: Azure Data Factory, ADLS/Key Vaults