Data Engineer

Devonshire Hayes Recruitment Specialists Ltd; City Of London; Hybrid; GBP 60,000 - 80,000; Part time

Job summary

A recruitment agency seeks an experienced Data Engineer to work on large-scale data enablement projects in London. The role involves designing and developing Azure Pipelines and data transformation frameworks. Candidates should have strong skills in Azure Synapse, SQL, Pyspark, and Python. This is a contract position requiring onsite attendance once a week and has been deemed inside IR35.

Qualifications

• Experience in Azure Synapse and ETL processes.
• Proficiency in data modelling and Data Bricks.
• Ability to develop Azure Pipelines and document source-to-target mappings.

Responsibilities

• Design and develop Azure Pipelines for data transformation and cleansing.
• Re-engineer manual data flows for scalability.
• Build accessible datasets for analysis.

Skills

Azure Synapse

ETL

Pyspark

SQL

Data Modelling

Data Bricks

T-SQL

Python

Job description

If you are an experienced Data Engineer with excellent communication skills and a proven track record of working on large scale data enablement projects, we have a new contract we would like to discuss with you.

Please note this role requires onsite attendance once a week and has been deemed inside IR35.

Requirement:

• Experience in Azure Synapse, ETL, Pyspark, SQL , data Modelling and Data Bricks.
• Design and develop Azure Pipelines including data transformation and data cleansing
• Document source-to-target mappings
• Re-engineer manual data flows to enable scaling and repeatable use
• Build accessible datasets for distribution and analysis
• Development of Azure Pipelines for transforming data
• A scalable meta-data driven ingestion and transformation framework
• Aligning transformation pipelines and datasets with Purview
• Experience of working with Azure Data Lakes, Data warehousing, and pipelines and Storage accounts
• Experience in building and managing Pipelines including: building data interfaces to source systems, combining and transforming data into appropriate storage formats
• Experience identifying and resolving issues in databases, data processes, data products and services.
• Proficiency in T-SQL and Python to develop automation