Senior Data Engineer - West Yorkshire - £80k

Exalto Consulting; West Yorkshire; Hybrid; GBP 70,000 - 80,000; Full time

Job summary

Exalto Consulting is seeking a Senior Data Engineer highly skilled in Microsoft Fabric to design and build end-to-end ETL/ELT pipelines in a Lakehouse environment. You will implement Dataflow Gen2, Fabric Notebooks, and Delta Lake, and work across SQL, Python and Spark SQL to integrate data from multiple sources.

The role is hybrid in West Yorkshire, with 2 days onsite per week. You will collaborate with Data Architects, BI teams and business stakeholders to deliver scalable data solutions,

Qualifications

• Strong commercial experience with Microsoft Fabric.
• Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture.
• Strong SQL and Python skills.
• Experience with Spark and Spark SQL.
• Proven experience designing scalable ETL and ELT pipelines.
• Data Lake and Delta Lake experience.
• Experience integrating data from multiple sources.
• Good understanding of CI/CD, DevOps and modern data engineering practices.

Responsibilities

• Design and build end to end ETL and ELT solutions using Microsoft Fabric
• Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks
• Develop and maintain Lakehouse solutions using Medallion architecture
• Work with SQL, Python, Spark SQL and Delta Lake
• Integrate data from APIs, databases, CRM, ERP and third party platforms
• Support both batch and real time data processing
• Develop semantic models and work with Star Schema and Kimball modelling
• Implement CI/CD and automated deployment of data solutions
• Build monitoring, data quality, lineage and observability into data pipelines
• Work closely with Data Architects, BI teams and wider business stakeholders

Skills

Microsoft Fabric

Dataflow Gen2

Fabric Notebooks

SQL

Python

Spark SQL

Delta Lake

CI/CD

DevOps

Data pipelines

Job description

Senior Data Engineer Microsoft Fabric

Location: West Yorkshire

Working pattern: Hybrid, 2 days per week onsite

Salary: £70-80k

Key responsibilities

• Design and build end to end ETL and ELT solutions using Microsoft Fabric
• Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks
• Develop and maintain Lakehouse solutions using Medallion architecture
• Work with SQL, Python, Spark SQL and Delta Lake
• Integrate data from APIs, databases, CRM, ERP and third party platforms
• Support both batch and real time data processing
• Develop semantic models and work with Star Schema and Kimball modelling
• Implement CI/CD and automated deployment of data solutions
• Build monitoring, data quality, lineage and observability into data pipelines
• Work closely with Data Architects, BI teams and wider business stakeholders

Essential experience

• Strong commercial experience with Microsoft Fabric
• Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture
• Strong SQL and Python skills
• Experience with Spark and Spark SQL
• Proven experience designing scalable ETL and ELT pipelines
• Data Lake and Delta Lake experience
• Experience integrating data from multiple structured and unstructured sources
• Good understanding of CI/CD, DevOps and modern data engineering practices
• Able to work onsite in West Yorkshire 2 days per week

Experience with Semantic Models, DAX, Kimball modelling, Microsoft Fabric certifications such as DP 700, and data governance would also be beneficial