Azure Data Engineer

Scrumconnect Limited; Newcastle upon Tyne; On-site; GBP 55,000 - 75,000; Full time

Job summary

Scrumconnect Limited in Newcastle upon Tyne is seeking a proactive Azure Data Engineer to design, develop, and support scalable data solutions on Microsoft Azure. You will collaborate with engineering, data, and delivery teams to build reliable cloud data platforms and pipelines, while applying best practices for automation and source control.

The role emphasizes hands-on work with ADF, Synapse, Data Lake, and IaC with Terraform, alongside GitLab for CI/CD, and requires strong data engineering

Qualifications

• Experience designing and implementing scalable data pipelines in Azure.
• Hands-on experience with Azure Data Factory and Synapse Analytics.
• Strong knowledge of data lake storage and IaC with Terraform.
• Proficient in GitLab for CI/CD and source control.
• Ability to work with cross-functional teams and improve engineering practices.

Responsibilities

• Design, develop, and maintain scalable Azure data solutions and pipelines.
• Build data pipelines using Azure Data Factory.
• Develop data solutions with Azure Synapse Analytics.
• Manage data storage with Azure Data Lake.
• Implement cloud infrastructure using Terraform.
• Collaborate with developers, data analysts, architects, and delivery teams.
• Monitor and optimize data pipelines and Azure services.
• Ensure security, reliability, and maintainability of data solutions.
• Follow best practices around automation and source control.
• Use GitLab for CI/CD processes.

Skills

Data engineering

Data pipelines

Problem solving

Communication

Team collaboration

Automation mindset

Tools

Azure Data Factory

Azure Synapse Analytics

Azure Data Lake

Terraform

GitLab

GitLab CI/CD

Job description

Newcastle, United Kingdom

Role Overview

We are looking for a proactive and hands-on Azure Data Engineer to join our engineering team. The successful candidate will be responsible for designing, developing, and supporting scalable data solutions using Microsoft Azure.

You will work closely with engineering, data, and delivery teams to build reliable data pipelines and cloud-based data platforms, while following best practices around infrastructure, automation, and source control.

Key Responsibilities

• Design, develop, and maintain scalable Azure data solutions and pipelines .
• Build and manage data pipelines using Azure Data Factory (ADF) .
• Develop and optimise data solutions using Azure Synapse Analytics .
• Work with Azure Data Lake for data storage and processing.
• Implement and maintain cloud infrastructure using Terraform .
• Use GitLab for source control, version management, and CI/CD processes.
• Monitor, troubleshoot, and optimise data pipelines and Azure services.
• Ensure data solutions are secure, scalable, reliable, and maintainable.
• Collaborate with developers, data analysts, architects, and delivery teams.
• Proactively identify technical issues and opportunities for improvement.
• Contribute to automation and continuous improvement of engineering practices.
• Strong experience working with Microsoft Azure cloud services.
• Hands-on experience with Azure Data Factory (ADF) .
• Strong experience with Azure Synapse Analytics .
• Experience working with Azure Data Lake .
• Hands-on experience with Terraform and Infrastructure as Code (IaC).
• Experience using GitLab , including GitLab CI/CD.
• Strong data engineering and data pipeline development experience.
• Good understanding of cloud-based data architecture and integration.
• Strong troubleshooting and problem-solving skills.
• Proactive approach with the ability to work independently and as part of a team.
• Good communication and collaboration skills.

Nice to Have

• Experience with Databricks .
• Experience with Apache Kafka .
• Experience with Kafka Connect .
• Knowledge of event-driven data architectures and real-time data processing.

Key Technologies

Diversity & Inclusion

At Scrumconnect Consulting, we believe that diversity drives innovation and better outcomes. We are committed to fostering an inclusive environment where every individual is valued, respected, and supported.

We welcome applications from candidates of all backgrounds and actively encourage individuals from underrepresented groups, including women, people with disabilities, and diverse communities, as well as those seeking flexible working arrangements.

As a Disability Confident Level 1 employer, we are committed to providing reasonable adjustments throughout the recruitment process and during employment to ensure equal opportunities for all.