Azure Databricks Data Engineer

Cognizant; Halifax Regional Municipality; Remote; CAD 90,000 - 105,000; Full time

Job summary

Cognizant is seeking an Azure Databricks Data Engineer to design, build, and optimize scalable data pipelines in a cloud-first environment. You will work with a cross-functional team to deliver robust data solutions using PySpark, Python, and Azure Databricks.

The role emphasizes optimizing ingestion, processing, and transformation frameworks for large datasets, with focus on performance, reliability, and collaboration with business stakeholders.

Qualifications

• 8–12 years of experience in data engineering or data warehousing.
• Hands-on experience with Azure Databricks in enterprise-scale environments.
• Advanced Python and PySpark for distributed data processing.
• Experience designing, developing, and optimizing ETL/ELT pipelines and large-scale data platforms.
• Strong understanding of Azure cloud services, data engineering concepts, data modeling, and SQL.
• Proven ability to troubleshoot complex data processing and pipeline issues in production environments.

Responsibilities

• Design, build, and maintain scalable data pipelines on Azure Databricks.
• Develop ingestion, transformation, and processing frameworks for large datasets.
• Implement cloud-native data engineering solutions using Azure data services.
• Monitor, troubleshoot, and improve data platform reliability and performance.
• Collaborate with stakeholders to translate requirements into robust data solutions.

Skills

Azure Databricks

Python

PySpark

SQL

ETL/ELT pipelines

Azure Cloud

Tools

CI/CD

DevOps

Job description

About the role

As an Azure Databricks Data Engineer, you will make an impact by designing, developing, and optimizing scalable cloud-based data engineering solutions that enable efficient data processing and analytics. You will be a valued member of the Data Engineering team and work collaboratively with architects, business analysts, data scientists, and cross-functional technology teams to deliver high-quality data solutions.

In this role, you will:

• Design, build, and maintain scalable data pipelines using Azure Databricks, Python, and PySpark.
• Develop and optimize data ingestion, transformation, and processing frameworks for large-scale datasets.
• Implement cloud-native data engineering solutions leveraging Azure data services and best practices.
• Monitor, troubleshoot, and enhance the performance, reliability, and scalability of data platforms and pipelines.
• Collaborate with business and technical stakeholders to translate requirements into robust data solutions and support production deployments.

Work model

We strive to provide flexibility wherever possible. Based on this role's business requirements, this is a remote position open to qualified applicants in Mississauga, ON or Halifax, NS. Regardless of your working arrangement, we are here to support a healthy work-life balance through our various wellbeing programs.

The working arrangements for this role are accurate as of the date of posting. This may change based on the project you're engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.

What you need to have to be considered

• 8-12 years of experience in Data Engineering, Data Warehousing, or related areas.
• Strong hands-on experience with Azure Databricks in enterprise-scale environments.
• Advanced programming skills in Python and extensive experience developing distributed data processing solutions using PySpark.
• Experience designing, developing, and optimizing ETL/ELT pipelines and large-scale data platforms.
• Strong understanding of Azure cloud services, data engineering concepts, data modeling, and SQL.
• Proven ability to troubleshoot complex data processing and pipeline issues in production environments.

These will help you stand out

• Experience within the Healthcare domain.
• Knowledge of Databricks performance tuning, optimization techniques, and production support.
• Experience supporting cloud modernization and enterprise data transformation initiatives.
• Familiarity with CI/CD, DevOps practices, and automated deployment strategies for data engineering solutions.
• Experience working with large, complex datasets in highly regulated environments.

We regularly assess market data to ensure we offer a competitive compensation package for our associates. The base salary for this position ranges between $90,000 – 105,000 per year. Where the successful candidate may fall within the range depends on relevant education, work and/or management experience and other business-related and job-necessary qualifications. This position is also eligible for Cognizant’s discretionary annual performance-based bonus, as well as benefits that support your physical, mental and financial wellbeing.

Cognizant will only consider applicants for this position who are legally authorized to work in Canada without requiring employer sponsorship, now or at any time in the future.

Applications will be accepted until November 01, 2026.