Lead Data Engineer

Inara; Greater Manchester; Hybrid; GBP 80,000 - 85,000; Full time

Job summary

Inara is seeking a hands-on Lead Data Engineer in Greater Manchester. You will own the data platform, oversee a small technical team, and integrate modern batch and live data processing to serve BI, Data Science and AI use cases. You will push technology boundaries by using Claude Code and cloud-based data tooling.

The role demands excellence in data governance, robust SQL/Python, and strong stakeholder communication to align with business goals. This is a hybrid role with on-site days.

Qualifications

• Experience delivering batch processing, ELT, and streaming pipelines.
• Hands-on data engineering for BI, data science and AI use cases.
• Strong SQL and Python with testing and engineering discipline.
• Proven leadership of a small technical team including hiring.
• Experience with data governance, retention, lineage and PCI-DSS controls.
• Comfort using Claude Code or similar tools to accelerate development.

Responsibilities

• Improve and evolve the data estate for modern batch and live data processing.
• Automate manual, duplicated, or fragile data processes.
• Support BI, Data Science, and AI teams from a single platform.
• Build a semantic layer and expose via MCP interfaces for analytics tools.
• Deliver governed self-serve data access for safe querying across the business.
• Lead the design of a Customer Data Platform for batch and streaming data.
• Line-manage the data engineers, including hiring and development.
• Own governance, security and running costs of the data platform.

Skills

Batch processing

Streaming data

SQL & Python

Leadership

Data governance

Tools

Claude Code

Terraform

AWS basics

Kinesis

Power BI

Iceberg

Job description

Location: Greater Manchester (Hybrid, 1 - 2 days a week on-site)

Salary: £80,000 - £85,000 (potentially some wiggle room for the right person)

We're working with an established fintech business in Greater Manchester that is doing something genuinely interesting with AI right now. Not pilots. Not proof of concepts gathering dust. Real, production-grade AI embedded into customer journeys and internal operations, and they're only just getting started.

They've recently brought in a Head of Data and AI who has both the vision and the organisational backing to push this forward hard. This role sits right at the centre of that push, owning the data platform that everything else gets built on.

The Role

This is a hands-on Lead Data Engineer role reporting into the Head of Data & AI. You'll take on the existing data engineering estate and bring state-of-the-art batch and live data processing to it, so it can properly serve BI, Data Science and Agentic AI use cases going forward.

It's a lead role with line management, but a genuinely hands-on one. You'll be expected to use tools like Claude Code as a core part of how you engineer and build, push the technology envelope where it's warranted, and bring a small technical team with you as you do it.

The aim isn't to rebuild for the sake of it. It's to evolve what works, automate what shouldn't be manual, and drive towards one platform that serves the whole business rather than a fragmented, function-based estate.

What You'll Be Doing

• Improving and evolving the existing data estate so it's ready for modern batch and live-streaming data processing
• Simplifying and automating data processes that are manual, duplicated or fragile today
• Supporting the BI, Data Science and Agentic (Applied AI) teams from a single platform
• Building a semantic layer over the data and exposing it through MCP and similar interfaces, giving the business the ability to analyse its data through tools like Claude
• Delivering a governed self-serve capability that lets anyone in the business query data safely
• Leading the design of a Customer Data Platform for batch and live data processing, working closely with ML and Data Science on shared ingestion, storage and feature needs
• Line-managing the data engineers, including hiring and developing the team needed to deliver
• Owning the governance, security and running cost of the data platform alongside Data & AI leadership

What You'll Need

• Proven delivery in both batch processing (warehouse loads, ELT, orchestration) and modern streaming, with Flink or comparable engines
• Production AWS data engineering experience: S3, Lambda, Glue, Athena/Iceberg, Kinesis or equivalent streaming services, RDS, IAM, CloudWatch, plus infrastructure as code with Terraform and CI/CD
• Strong lakehouse and data modelling depth: Iceberg-style open table formats, medallion-style layering, dimensional modelling, and the judgement to know when a warehouse should consume from a lakehouse rather than replicate it
• Strong SQL and Python with proper testing and engineering discipline. Willingness to work with SQL Server, SSIS/SSDT and Power BI at the start, even if you've moved beyond them elsewhere
• Experience designing semantic layers, conformed metric definitions and glossaries, and exposing data to LLM tooling through MCP servers or similar
• Daily hands-on use of Claude Code or an equivalent frontier coding assistant as part of how you engineer, with a clear story of how it's made you materially faster without compromising quality or security
• Solid data governance experience in a regulated business: retention, lineage, classification, PII minimisation, auditability and PCI-DSS boundaries built into the platform, not bolted on
• Business-area fluency, learning what functions like FinCrime, Regulatory Reporting, Product, CX, Finance and Marketing actually do with data, and prioritising from their outcomes
• Experience line-managing a small technical team in a delivery setting, including hiring and development
• Clear written and spoken communication with senior stakeholders, including architecture position papers and candid risk statements

Bonus Points For

• Experience in financial services or another regulated environment
• PySpark for large-scale batch or streaming transformations
• Migrating a SQL Server/SSIS estate to a metadata-driven or lakehouse architecture, and a clear view on what you'd do differently
• Designing or operating a Customer Data Platform or event platform (Kinesis, Flink, StarRocks or comparable)
• Building data-quality and data-contract tooling (dbt tests, Great Expectations or equivalent) and synthetic monitoring around feeds
• Authoring Claude skills or MCP servers for a team, or running an LLM natural-language layer over BI data
• FinCrime, fraud or regulatory-reporting data domains
• Power BI at the platform level (semantic models, gateway, administration) for the transition period