Data Engineer

Devonshire Hayes Recruitment Specialists Ltd.; Glasgow, Cardiff, City Of London; Hybrid; GBP 50,000 - 80,000; Full time

Job summary

A recruitment agency is seeking an experienced Data Engineer to work on large-scale data enablement projects in Glasgow. The role requires excellent communication skills and involves designing and developing Azure Pipelines for data transformation. Candidates must have experience with Azure Synapse, ETL, and data modeling, and be able to work onsite once a week. This contract is inside IR35.

Qualifications

• Proven experience in large scale data enablement projects.
• Excellent communication skills.
• Ability to work onsite once a week.

Responsibilities

• Design and develop Azure Pipelines including data transformation and cleansing.
• Document source-to-target mappings.
• Re-engineer manual data flows for scalability.
• Build accessible datasets for analysis.
• Develop frameworks for transformation.

Skills

Azure Synapse

ETL

Pyspark

SQL

Data modelling

Data Bricks

T-SQL

Python

Job description

If you are an experienced Data Engineer with excellent communication skills and a proven track record of working on large scale data enablement projects, we have a new contract we would like to discuss with you. Please note this role requires onsite attendance once a week and has been deemed inside IR35.

Requirements

• Experience in Azure Synapse, ETL, Pyspark, SQL, data modelling and Data Bricks.
• Design and develop Azure Pipelines including data transformation and data cleansing.
• Document source-to-target mappings.
• Re-engineer manual data flows to enable scaling and repeatable use.
• Build accessible datasets for distribution and analysis.
• Development of Azure Pipelines for transforming data.
• A scalable meta-data driven ingestion and transformation framework.
• Aligning transformation pipelines and datasets with Purview.
• Experience of working with Azure Data Lakes, Data warehousing, and pipelines and Storage accounts.
• Experience in building and managing Pipelines including: building data interfaces to source systems, combining and transforming data into appropriate storage formats.
• Experience identifying and resolving issues in databases, data processes, data products and services.
• Proficiency in T‑SQL and Python to develop automation.