MS Fabric Data Engineer

Cognizant; Sheffield; Hybrid; GBP 65,000 - 90,000; Full time

Job summary

Cognizant is seeking an experienced Azure Data Engineer to support the design, development and delivery of modern data platform solutions, with a focus on Microsoft Fabric migration and implementation.

You will deliver high-quality, scalable data solutions and provide technical guidance while collaborating with engineering teams and stakeholders across Azure data technologies.

Qualifications

• Strong experience designing and delivering Azure-based data platform solutions.
• Hands-on expertise in PySpark, SQL and Python.
• Knowledge of real-time streaming technologies (Kafka, Azure Functions, Azure Service Bus).
• Spark processing and performance tuning experience.
• Data modelling knowledge (3NF, dimensional modelling, Data Vault 2.0).
• Experience with Agile delivery environments and technical documentation.

Responsibilities

• Configure data ingestion processes and solutions.
• Lead the execution of Microsoft Fabric migration activities.
• Lead and guide migration engineers and QA resources.
• Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks.
• Create data integration test cases and support testing activities.
• Implement and amend data processing and worker pipelines.
• Implement data validation and curation rules.
• Translate data models and mappings into detailed technical data mappings.
• Design and implement data integrations.
• Support the implementation and execution of data migration activities.
• Perform data analysis and profiling.
• Configure Azure DevOps / GitHub environments and CI/CD processes for Azure Data Factory and Databricks code.

Skills

Azure data platform

PySpark

Spark SQL

Python

Real-time streaming

Spark tuning

Data modelling

Agile delivery

Technical documentation

Tools

Azure Data Factory

Databricks

GitHub

Azure DevOps

CI/CD pipelines

Parquet

JSON

XML

CSV

Job description

Work Pattern: Hybrid, client office 3 days p/w

About the Role

We are looking for an experienced Azure Data Engineer to support the design, development and delivery of modern data platform solutions, with a particular focus on Microsoft Fabric migration and implementation.

You will play a key role in delivering data engineering and migration activities, working closely with engineering teams and stakeholders to deliver high-quality, scalable and well-tested data solutions.

The role requires strong technical expertise across Azure data technologies, alongside the ability to provide technical guidance, communicate effectively and contribute to successful delivery.

Key Responsibilities

• Configure data ingestion processes and solutions.
• Lead the execution of Microsoft Fabric migration activities.
• Lead and guide migration engineers and QA resources.
• Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks.
• Create data integration test cases and support testing activities.
• Implement and amend data processing and worker pipelines.
• Implement data validation and curation rules.
• Translate data models and mappings into detailed technical data mappings.
• Design and implement data integrations.
• Support the implementation and execution of data migration activities.
• Perform data analysis and profiling.
• Configure Azure DevOps / GitHub environments and CI/CD processes for Azure Data Factory and Databricks code.

We are looking for candidates with:

• Strong experience designing and delivering Azure-based data platform solutions.
• Experience with technologies including: Microsoft Fabric o Power Apps o Azure Synapse
• Hands‑on expertise in PySpark, SQL and Python.
• Good knowledge of real‑time streaming applications, preferably including Kafka, Azure Functions or Azure Service Bus.
• Strong understanding of Spark processing and performance tuning.
• Experience working with file formats and partitioning, including Parquet, JSON, XML and CSV.
• Experience with Azure DevOps / GitHub and CI/CD.
• Strong programming experience in Python, with knowledge of other programming languages beneficial.
• Experience with both synchronous and asynchronous interface approaches.
• Knowledge of data modelling, including 3NF, dimensional modelling and Data Vault 2.0.
• Experience working within Agile delivery environments.
• Ability to produce comprehensive technical documentation.
• Ability to set and manage realistic expectations around timescales, costs, benefits and measures of success.