Data Engineer

Dwr Cymru Cyf; Cardiff; On-site; GBP 45,000 - 65,000; Full time

Job summary

Dwr Cymru Cyf is seeking a Data Engineer to evolve our cloud-based enterprise data platform. You will design robust data pipelines, maintain SQL servers feeding analytics products, and support data-driven decision making for public services.

You will work across on‑premises and cloud environments, ensuring data quality, security, and scalable data delivery. Strong Python/SQL skills and experience with Microsoft data tools are essential.

Qualifications

• Substantial experience designing and managing data pipelines.
• Ability to add customer value while improving data engineering effectiveness.
• Proactive problem solving and clear communication to data teams and stakeholders.
• Good understanding of database administration principles.
• Knowledge of data transformation approaches.
• Strong Python and SQL programming capabilities.
• Knowledge of data security practices and permission models (AD/RBAC) and encryption/masking.
• Familiarity with agile project management in data/analytics contexts.
• Understanding ingestion patterns ETL/ELT in on-prem/cloud environments.
• Knowledge of data modelling approaches (Kimball).
• Experience with Microsoft data stack: Azure Data Lake, Synapse, Data Factory, Databricks.
• Experience across full software lifecycle.
• Knowledge of Microsoft BI stack (SSAS/SSIS/SSRS, Power BI).
• Understanding of DevOps, CI/CD and version control.

Responsibilities

• Designing efficient and robust data engineering projects within on-premises and cloud-based data stacks that add business value and comply with architectural and data security requirements.
• Building, maintaining and supporting data engineering pipelines, with resolution of incidents.
• End-to-end delivery of data engineering solutions, including data quality, testing, peer reviewing, documenting and demonstrating of completed solutions to the team.
• Working closely with teams within data and other technical colleagues across the business.
• Remaining aware of new data engineering approaches, and be able to suggest how the latest research, techniques and approaches could be implemented to achieve business benefit.
• Undertaking any other duties as required to meet the needs of the business.

Skills

Python

SQL

Data pipelines

Data security

Communication

DB administration

Data transformation

Tools

Azure Data Lake

Azure Synapse

Azure Data Factory

Databricks

Job description

This is an exciting role that will allow you to be integral to the evolution of our cloud-based enterprise data platform. As a Data Engineer, you will be working on technical solutions, whilst supporting our existing set of SQL Servers that are predominantly used to feed data into a number of analytics products. This will help the business with data-led decision making to provide essential public services.

Responsibilities

• 1. Designing efficient and robust data engineering projects within both on-premises and cloud-based data stacks that add business value and comply with architectural and data security requirements.
• 2. Building, maintaining and supporting data engineering pipelines, with resolution of incidents.
• 3. End-to-end delivery of data engineering solutions, including data quality, testing, peer reviewing, documenting and demonstrating of completed solutions to the team.
• 4. Working closely with teams within data and other technical colleagues across the business.
• 5. Remaining aware of new data engineering approaches, and be able to suggest how the latest research, techniques and approaches could be implemented to achieve business benefit
• 6. Undertaking any other duties as required to meet the needs of the business.

Requirements

• 1. Substantial experience of data pipeline design and management
• 2. Enthusiasm to add customer value whilst improving the effectiveness of the data engineering function
• 3. Proactive problem solving and being able to effectively communicate solutions to both the data team, and to stakeholders
• 4. Good understanding of database administration activities and principles
• 5. Data transformation approaches
• 6. Strong Python and SQL programming capabilities
• 7. Data security approaches - e.g. permission models (AD, role-based etc.) and techniques to protect data in certain circumstances (e.g. encryption, masking)

Desirable Qualifications, Knowledge and Experience

• 1. Familiarity with common agile project management principles, particularly in the context of data and analytical project delivery
• 1. Good understanding of typical ingestion patterns (e.g. ETL, ELT) and their effective implementation with on-premises and cloud-based environments
• 2. Knowledge of typical data modelling approaches (e.g. Kimball)
• 3. Experience using the Microsoft data stack (especially Azure Data Lake, Azure Synapse, Azure Data Factory, Databricks)
• 4. Experience of the full software/reporting lifecycle from planning and design through to deployment and maintenance
• 5. Knowledge of the Microsoft BI Stack, including Analysis Services, Integration Services and Reporting Services, and Power BI
• 6. Understanding of DevOps (or other source control technologies) - particularly code repositories and version control, containerisation methodologies, and application deployment practices (CI/CD pipelines)