Tech Lead - Azure Databricks Developer

United Software Group Inc. - Canada; Canada; Remote; CAD 100,000 - 130,000; Full time

Job summary

A technology company based in Halifax, NS, is seeking an Azure Databricks technical expert. The role focuses on designing and implementing high-performance data ingestion pipelines and developing scalable frameworks. Candidates should have over 10 years of experience and strong expertise in Azure Databricks and Apache Pyspark, with additional knowledge in Azure Data Factory being a plus. This is a permanent remote position.

Qualifications

• 10+ years of experience in related technical roles.
• Strong problem-solving skills.
• Ability to work independently and as part of a team.

Responsibilities

• Design and implement data ingestion pipelines using Azure Databricks.
• Develop frameworks for various data sets and formats.

Skills

Strong expertise with Azure Databricks

Apache Pyspark

Azure Data Factory

ADLS/Key Vaults

Job description

Overview

Work location : Halifax, NS (Permanent remote)

Note : This position would serve the team as Azure Databricks technical lead rather than a team lead so more than a people mentor, this position is for a technical expert. They would also be expected to contribute as a hands on Azure Databricks developer. Please look for 10+ years experienced candidates.

Job Description

• Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure Databricks.
• Developing scalable and re-usable frameworks for ingesting of varied data sets and data formats.

Skills

• Necessary: Strong expertise with Azure Databricks, Apache Pyspark
• Nice to Have: Azure Data Factory, ADLS/Key Vaults