Lead Data Engineer

Mirai Talent; Wilmslow; Hybrid; GBP 68,000 - 92,000; Full time

Benefits offered by this job

35 days holiday + bank holidays

Private medical & dental

Up to 20% bonus

Flexible working

Defined progression towards Head of I

Competitive salary (£80,000)

Job summary

Mirai Talent is partnering with a fast-growing data business to recruit a Lead Data Engineer & Solution Architect who will own Databricks-first platforms and drive greenfield architecture.

You’ll design the foundations, evolve reference architectures, and work directly with clients as the company grows, with a clear path toward engineering leadership and a generous bonus structure.

Qualifications

• Hands-on Databricks experience with Delta Lake, Unity Catalog, Workflows and DLT/Lakeflow.
• Advanced SQL and Python/PySpark skills.
• Strong ingestion and integration experience, including APIs, CDC and incremental loading.
• Data modelling and medallion architecture in real production environments.
• Refined architectural thinking with a track record of owning data platforms.

Responsibilities

• Own and evolve a Databricks-first lakehouse architecture.
• Design complex ingestion and integration pipelines.
• Work hands-on across Python, PySpark and SQL.
• Make decisions on batch, incremental and streaming patterns.
• Design dimensional models and data products.
• Set engineering standards across CI/CD, testing, IaC and deployment.
• Translate technical discovery into scalable solution architecture.
• Build engineering capability and standards as the team grows.
• Collaborate directly with clients on greenfield challenges.

Skills

Databricks / Delta Lake

SQL

Python / PySpark

Data ingestion / CDC

Data modelling (medallion)

Ownership / leadership

Tools

Databricks

Terraform

dbt

Fabric

Snowflake

MLflow

Azure

Job description

• £80,000, with flexibility for an exceptional candidate
• Up to 20% bonus
• Wilmslow / flexible working
• 35 days holiday + bank holidays
• Private medical & dental
• Defined progression towards Head of Engineering

We’re partnering with a brilliant, growing data business to find a Lead Data Engineer & Solution Architect who wants genuine technical ownership.

This is a company we absolutely love working with. We’ve introduced a number of people into the team over the years who are still there, thriving and progressing. That says a lot about the culture, the leadership and how seriously they take developing their people.

The opportunity itself is a rare mix of architecture and genuinely hands-on engineering.

You’ll become the technical owner of the Databricks-first platforms they build for clients, working predominantly on greenfield projects where you’re designing the foundations rather than inheriting years of legacy.

You’ll own and evolve the reference architecture, shape technical solutions and roadmaps, tackle the hardest engineering challenges, and help build the engineering function as the company enters its next stage of growth.

There’s also a defined progression route towards Head of Engineering.

What you’ll be doing

• Owning and evolving a Databricks-first lakehouse architecture
• Designing and building complex ingestion and integration pipelines
• Working hands-on across Python, PySpark and SQL
• Making decisions around batch, incremental and streaming patterns
• Designing dimensional models and data products
• Setting engineering standards across CI/CD, testing, IaC and deployment
• Translating technical discovery into scalable solution architecture
• Building the engineering capability and standards as the team grows
• Working directly with clients on interesting, predominantly greenfield challenges

What we’re looking for

There are four core things that matter:

• Deep, hands-on Databricks experience, including Delta Lake, Unity Catalog, Workflows and DLT/Lakeflow
• Advanced SQL and Python/PySpark
• Strong ingestion and integration experience, including APIs, CDC and incremental loading
• Data modelling and medallion architecture experience in real production environments

You do not need to come from consultancy.

In fact, if you’re currently leading or owning a data platform in-house and want broader technical ownership, greenfield work and the opportunity to develop your architecture and client-facing experience, we’d particularly like to hear from you.

Experience with Terraform, dbt, Fabric, Snowflake, MLflow, Azure infrastructure or Power BI is useful, but it isn’t the reason we’ll hire you.

This is ultimately about finding someone who still loves engineering, has strong technical judgement and gets excited by the idea of building something rather than simply inheriting it.

And the culture matters just as much. It’s a human-centred, B Corp certified business that invests heavily in development, including funded Databricks certification, while also doing meaningful work around social mobility and technology education in Greater Manchester.

If you’re a Lead / Principal Data Engineer or Data Platform Engineer looking for more ownership and a genuine path towards engineering leadership, we’d love to chat.

Diversity & Inclusion

At Mirai, we believe diversity drives innovation. We’re committed to creating an inclusive recruitment process and encouraging applications from people of all backgrounds, experiences and perspectives.

We know that no candidate will tick every single box. If you’re excited about this opportunity but don’t meet every requirement, we’d still encourage you to apply. If you need any reasonable adjustments throughout the recruitment process, please let us know.