Lead Data Engineer

CreateFuture; Glasgow; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Flexible working

Hybrid/remote options

Office hubs: Edinburgh, Leeds, London,

Office hub in Bulgaria

Job summary

CreateFuture in Glasgow is seeking an experienced Lead Data Engineer to own the discovery and shaping of a large SQL Server estate for a financial services client. You will map data lineage, identify live vs. dead structures, and mentor a small team through remediation and future-state planning.

The role blends hands-on analysis with strategic guidance, requiring collaboration with DBAs and platform owners, while offering hybrid work with two days per week on client sites in Glasgow.

Qualifications

• Extensive experience with large SQL Server estates (OLTP).
• Ability to map data lineage, identify live vs dead schemas, and resolve duplication.
• Understanding of financial services data retention, audit, and regulatory needs.
• Comfort with ambiguity and methodical, evidence-based analysis.

Responsibilities

• Lead the as-is analysis of a 1,100-table SQL Server RDS estate.
• Map integration points and trace data lineage using Claude-driven tooling.
• Mentor a small team of data engineers on the engagement.
• Advise on remediation, consolidation, archiving and migration options.
• Produce clear, defensible documentation of the database estate.
• Collaborate with client stakeholders (DBAs, platform owners, risk teams).
• Define and validate the engineering approach for estate discovery.

Skills

SQL Server expertise

Data estate analysis

Mentoring/leadership

AI tooling (Claude)

Tools

AWS RDS for SQL Server

Python

PowerShell

Job description

Working at CreateFutureCreateFuture is an AI-native consulting partner where people do work that matters and are supported to do it well. We work alongside organisations such as PayPal, adidas, NatWest, FanDuel and Money Saving Expert, building digital products and services that make a difference while always putting people first.

We’re a team of creators. We write code, shape delivery, build go-to-market strategies, develop AI solutions and create the practices that support our people. We work side by side with our clients, challenging what’s not working and helping them to build the

About the role

We're looking for an experienced Lead Data Engineer to join our growing Data Practice, working on a FinancialServices engagement centred on a large, long-lived SQL Server estate. You'll work as part of a multidisciplinaryteam helping a major FS client understand and de-risk a database landscape that has grown, unchecked, over 15years of trading activity.

This is a role for someone who is comfortable going in without a map: a single SQL Server RDS instance holding around 1,100 tables and fifteen years of trade data, with no archiving strategy, no inventory of stored procedures, and tables duplicated across multiple schemas with no reliable way to tell what's actually in use.Part of the engagement's as-is analysis uses Claude to scan the codebase, map integration points, and flag likelydead code — you'll sit at the centre of that process, validating, interrogating, and making sense of what thetooling surfaces.

As a Lead, you'll shape the technical approach to the estate discovery, mentor engineers supporting theanalysis, and act as a trusted, credible partner to the client's data and platform teams.

This role is Hybrid - requires 2 days per week on our clients site in Glasgow.

Key Responsibilities

• Leading the as-is analysis of a ~1,100-table SQL Server RDS estate — establishing what exists, how it'sstructured, and where the risk sits, in the absence of any existing documentation or stored procedureinventory.
• Working alongside Claude-driven codebase scanning to map integration points, trace data lineage, andidentify likely dead code, then applying engineering judgement to validate and prioritise the findings.
• Investigating duplicated tables across schemas to determine which copies are live, which are redundant,and what depends on each — building the tooling and queries needed where none currently exist.
• Designing an approach to 15 years of unarchived trade data — assessing volume, growth, and usagepatterns to inform archiving, retention, and future-state options.
• Producing clear, defensible documentation of the database estate — schemas, dependencies, storedprocedures, and data flows — that the client can rely on long after the engagement ends.
• Advising on remediation and modernisation options once the estate is understood, from consolidationand archiving through to longer-term platform or cloud migration.
• Leading and mentoring a small team of data engineers on the engagement, setting standards for howfindings are verified, tested, and documented.
• Acting as a credible, hands-on technical partner to client stakeholders — including DBAs, platformowners, and risk/compliance contacts — who will be understandably cautious about changes to a livetrading data estate.

Skills & Experience

You know your way around legacy SQL Server estates

• You're genuinely strong in SQL Server — schema design, query performance, stored procedures,indexing — and comfortable working in large, poorly documented production estates.
• You've reverse-engineered a database landscape before: worked out what's live vs. dead, resolvedduplicated or conflicting schemas, and built confidence in an estate nobody fully understood.
• You understand the particular pressures of financial services data — long retention requirements, auditand regulatory expectations, and the need to prove data integrity for trade and transaction records.
• You're comfortable with ambiguity and incomplete information, and know how to build a reliablepicture of a system methodically rather than guessing.

You're comfortable using AI tooling as part of the analysis

• You're open to and effective at using AI-assisted tools (we use Claude) to scan codebases, mapintegration points, and surface candidate dead code at a scale manual review can't match.
• You know how to sanity-check and validate AI-generated findings against the real system — you treatthe output as a strong starting hypothesis, not ground truth.
• You can translate what the tooling surfaces into a clear, prioritised narrative that both engineers andnon-technical stakeholders can act on.

You know your tools

• Deep, hands-on SQL Server experience — T-SQL, SSMS, execution plans, and diagnosing performance orintegrity issues in large OLTP schemas.
• Comfortable with AWS RDS for SQL Server, including monitoring, performance insights, and operationalmanagement.
• Solid scripting ability (Python and/or PowerShell) for building the discovery, cataloguing, anddependency-mapping tooling this engagement needs from scratch.
• Experience with data cataloguing, lineage, or metadata management tooling is a strong plus, even if theclient doesn't currently have any in place.
• Familiarity with data archiving and retention strategies for large historical datasets is beneficial.
• Primary platform Microsoft SQL Server (AWS RDS) — T-SQL, SSMS, stored procedures,execution plans
• Cloud AWS — RDS, S3, and supporting services for tooling and discovery outputs
• Estate ~1,100 tables, 15 years of trade data, no archiving strategy, no storedprocedure inventory, duplicated schemas
• AI-assisted analysis Claude — codebase scanning, integration point mapping, dead codeidentification
• Languages SQL (T-SQL), Python and/or PowerShell for tooling
• Documentation & lineage - Data cataloguing / lineage tooling (implementation as part of the
  engagement)
• Ways of working Hybrid — client site in Glasgow, 2 days per week

What we’ll offer you:

We trust people to do their best work. That means flexibility over rigid rules, impact over activity, and real investment in your growth both professionally and personally. You’ll be part of a supportive, and friendly culture, surrounded by smart, curious people who care deeply about what they do.

We offer flexible working, including hybrid and remote options. Our office hubs are located in Edinburgh, Leeds, Manchester, London and Bulgaria, with occasional travel to client sites or CreateFuture offices when needed.

We trust you to manage your time balancing collaboration with client time and focused work. What matters is the impact you have, not how busy you look.

Our hiring process

We try to keep our hiring process clear, fair and respectful of your time. We aim to get back to everyone who applies and we will be upfront about where you are in the process.

It usually looks like this:

• Call with our Talent Acquisition Team
• Role specific capability interview

Depending on the role, we might also ask you to do a short presentation, a practical or technical task or have a values focused conversation. We will explain what is involved before anything happens.

Inclusion at CreateFuture

We believe diverse teams build better workplaces and better products. We want CreateFuture to be a place where people feel able to be themselves and do their best work.

If you need any adjustments or support during the application process, just. We will do what we can to help.

We look forward to your application!