Data Engineer — Azure Databricks Pipelines & Streaming

McGregor Recruitment; Southwestern Ontario; On-site; CAD 120,000 - 139,000; Full time

Job summary

A global logistics firm is seeking a Data Engineer to build and deploy data solutions using Azure technologies. You will optimize data pipelines, collaborate with a multi-skilled team, and help implement a new data strategy. Strong skills in Azure Databricks, Python, and data engineering best practices are essential. This role offers a competitive salary and the chance to make significant impacts within the organization.

Qualifications

• Commercial experience as a data engineer or analytics engineer.
• Experience in logistics or heavy industry sectors.
• Hands-on experience with Azure Databricks and Data Factory.
• Strong understanding of Lakehouse architecture.
• Proficient in Python, PySpark, and SQL.

Responsibilities

• Design end-to-end data pipelines using Azure Databricks.
• Implement Medallion Architecture with Azure Data Factory.
• Drive data consistency and accuracy across pipelines.
• Collaborate with analysts for dataset validation.
• Advocate for best practices in data engineering.

Skills

Data pipeline development

Azure Databricks

Python

SQL

ETL processes

Data governance

CI/CD best practices

Data quality frameworks

Agile methodologies

Tools

Azure Data Factory

Delta Lake

Power BI

Git

Job description

A global logistics firm is seeking a Data Engineer to build and deploy data solutions using Azure technologies. You will optimize data pipelines, collaborate with a multi-skilled team, and help implement a new data strategy. Strong skills in Azure Databricks, Python, and data engineering best practices are essential. This role offers a competitive salary and the chance to make significant impacts within the organization.