Lead Data Engineer

Amber Labs; United Kingdom; Remote; GBP 90,000 - 130,000; Full time

Job summary

Amber Labs is seeking an experienced Lead Data Engineer with strong Semarchy xDM expertise to lead enterprise MDM initiatives for UK government-related programs. You will own data modelling, Golden Records and survivorship rules, and guide data integration across Azure Databricks, APIM and Redis.

You will collaborate with data architects and service teams, promoting governance and high-quality data across complex, regulated environments. Active UK SC Clearance is required.

Qualifications

• Hands-on Semarchy xDM development and configuration.
• Strong SQL and data management fundamentals.
• Experience with master/reference data modelling and data quality.
• Experience with enterprise data integration and bidirectional data exchange.
• Ability to lead within agile teams and collaborate with architects.

Responsibilities

• Lead design, development and configuration of Semarchy xDM solutions.
• Design master/reference data models, entities and hierarchies.
• Develop Golden Records using Match & Merge and survivorship rules.
• Implement data quality, validation, enrichment and deduplication.
• Configure workflows, approvals and data certification.
• Design data ingestion and bidirectional integration with enterprise systems.
• Support integrations with Azure Databricks, APIM and Redis.
• Provide technical leadership across delivery teams.

Skills

Semarchy xDM

SQL

Data governance

Data modelling

Data integration

Azure Databricks

Azure API Management

Redis

Agile delivery

Golden Records

Match & Merge

Survivorship rules

Tools

Azure Databricks

APIM

Redis

Job description

Location: UK Remote - Very occasional travel to client site for workshops

Employment Type: Permanent or Contract (Inside IR35)

Security Clearance: Active UK SC Clearance Required

About Amber Labs

Amber Labs is a fast-growing UK technology consultancy delivering digital transformation across enterprise, government and regulated sectors. We specialise in cloud, data, software engineering, user-centred design and AI, helping organisations deliver better services through modern technology and agile ways of working.

We are currently supporting a major UK Government digital transformation programme for the Animal and Plant Health Agency (APHA) and are looking for an experienced Lead Data Engineer with strong Semarchy xDM expertise.

The Role

You will lead the development, configuration and implementation of enterprise Master Data Management (MDM) solutions using Semarchy xDM.

Working alongside data architects, engineers and service teams, you will help shape the MDM strategy and deliver scalable solutions that bring together data from multiple systems, creating accurate, consistent and trusted master data.

You will take ownership of data modelling, Golden Records, Match & Merge, survivorship rules, workflows and data quality processes. You will also support integrations with Azure Databricks, Azure API Management (APIM) and Redis.

This is a hands-on technical leadership role, combining specialist Semarchy development expertise with architectural collaboration and enterprise data integration.

Key Responsibilities

• Lead the design, development and configuration of Semarchy xDM solutions.
• Design and maintain master and reference data models, entities, relationships and hierarchies.
• Develop Golden Records using Match & Merge and survivorship rules.
• Implement data quality, validation, enrichment and deduplication processes.
• Configure workflows, approvals, issue resolution and data certification.
• Design data ingestion and bidirectional integration between Semarchy and enterprise systems.
• Support integrations with Azure Databricks, APIM and Redis.
• Work with architects and engineering teams to define scalable and secure solutions.
• Promote data governance, stewardship and consistent data management practices.
• Provide technical leadership and guidance across multidisciplinary delivery teams.
• Active UK SC Security Clearance.
• Strong hands-on experience with Semarchy xDM.
• Strong understanding of master and reference data modelling.
• Experience with Golden Records, Match & Merge and survivorship rules.
• Knowledge of data quality, validation, enrichment and deduplication.
• Experience with workflow management and data certification.
• Understanding of data governance, stewardship and custodianship.
• Experience with enterprise data integration and bidirectional data exchange.
• Strong SQL and data management fundamentals.
• Experience working with architects and technical stakeholders.
• Ability to provide technical leadership within agile delivery teams.
• Redis caching and performance optimisation.
• Experience delivering solutions within UK Government or the wider public sector.
• Experience working within complex, regulated enterprise environments.

Amber Labs is an equal opportunities employer. We are committed to creating an inclusive environment where everyone can thrive, regardless of background, identity or experience.