Data Engineer

Devonshire Hayes Recruitment Specialists Ltd.; Glasgow, Cardiff, Greater London; On-site; GBP 45,000 - 65,000; Full time

Job summary

A leading recruitment agency in the UK is seeking an experienced Data Engineer for a contract role focusing on large-scale data enablement projects. Candidates must have strong expertise in Azure Synapse, Pyspark, and SQL, along with excellent communication skills. The role requires a weekly onsite presence and is deemed inside IR35. Responsibilities include designing Azure Pipelines, data transformation, and building accessible datasets for analysis.

Qualifications

• Experience in building and managing Azure Pipelines for data transformation.
• Proficiency in developing source-to-target mappings and documentation.
• Experience with Azure Data Lakes and data warehousing.

Responsibilities

• Design and develop Azure Pipelines including data transformation and cleansing.
• Re-engineer manual data flows for scaling and repeatable use.
• Identify and resolve issues in databases and data processes.

Skills

Azure Synapse

ETL

Pyspark

SQL

data Modelling

Data Bricks

Automation with T-SQL

Python

Job description

If you are an experienced Data Engineer with excellent communication skills and a proven track record of working on large scale data enablement projects, we have a new contract we would like to discuss with you. Please note this role requires onsite attendance once a week and has been deemed inside IR35.

Requirement:

• Experience in Azure Synapse, ETL, Pyspark, SQL , data Modelling and Data Bricks.
• Design and develop Azure Pipelines including data transformation and data cleansing
• Document source-to-target mappings
• Re-engineer manual data flows to enable scaling and repeatable use
• Build accessible datasets for distribution and analysis
• Development of Azure Pipelines for transforming data
• A scalable meta-data driven ingestion and transformation framework
• Aligning transformation pipelines and datasets with Purview
• Experience of working with Azure Data Lakes, Data warehousing, and pipelines and Storage accounts
• Experience in building and managing Pipelines including: building data interfaces to source systems, combining and transforming data into appropriate storage formats
• Experience identifying and resolving issues in databases, data processes, data products and services.
• Proficiency in T-SQL and Python to develop automation