Data Engineer

Mirai Talent; Manchester; Hybrid; GBP 55,000 - 75,000; Full time

Job summary

Mirai Talent is seeking a Data Engineer in Manchester to join a hybrid team. You will help build the data platform, focusing on production data pipelines, Databricks native environments and automated data workflows.

The role emphasises hands-on Databricks and web scraping, with Python, SQL/Spark and ETL/ELT expertise. You’ll work across ingestion, APIs and deployment in a technically demanding setting.

Qualifications

• Hands-on Databricks and web scraping experience essential.
• Experience with Python, SQL/Spark and ETL/ELT.
• Understanding of databases, data warehousing and CI/CD basics.

Responsibilities

• Build and maintain production data pipelines and ETL/ELT processes.
• Support move to a Databricks-native environment and scalable platforms.
• Collaborate across teams on data ingestion, APIs and automation.
• Ensure platform reliability, monitoring and governance.

Skills

Databricks

Python

SQL

Spark

Web scraping

ETL/ELT

CI/CD

Azure

Platform reliability

Education

STEM background

Tools

Databricks

Azure

Terraform/Bicep

Job description

Manchester | Hybrid, ideally 2 days per week in office

We're working with a truly disruptive FinTech that is continuing to invest heavily in its data and technology capability.

With a growing Data Science function and around 3-5TB of new data being processed each week, they're looking for a Data Engineer to join the team and help build and develop the platform behind it.

This isn't necessarily a traditional Data Engineering profile. They're open to people from Data Engineering, Platform, DevOps or Systems Engineering backgrounds, but hands-on Databricks and web scraping experience are essential.

The role

You'll work within a cross-functional team, contributing to the move towards a Databricks-native environment and helping build reliable, scalable data and platform capabilities.

You'll be working across:

• Databricks, Python, SQL and Spark
• Web scraping and automated data collection
• High-volume data ingestion and transformation
• Production ETL/ELT pipelines
• APIs, SFTP/FTPS and automated data retrieval
• CI/CD, automated testing and deployment
• Platform reliability, monitoring and troubleshooting
• Azure infrastructure, security and governance
• Data Science, ML and increasingly agentic workflows

The focus is production engineering rather than building AI models. You'll be helping create the foundations that allow Data Science and automated workflows to operate effectively.

What we're looking for

Databricks and web scraping are the two big ones.

Alongside these, we're looking for hands-on experience with Python, SQL/Spark and ETL/ELT, an understanding of databases and data warehousing, and some exposure to platform, DevOps or systems engineering principles.

Just as important is mentality. They want someone who is technically curious, excited by what's happening across AI and constantly looking at what's next. Someone who experiments, keeps up with new tools and approaches, and naturally wants to stay ahead of the curve rather than waiting for new technology to become mainstream.

You don't need to be an expert in everything. They're looking for strong foundations, the right mindset and someone who wants to keep developing within a highly technical environment.

A STEM background is beneficial, but not essential.

Nice to have

• Playwright/Selenium or similar web scraping/automation tooling
• Databricks Asset Bundles, Lakeflow Jobs/Pipelines, Unity Catalog, Volumes and/or Lakebase
• Agentic or AI-enabled engineering workflows, LLM integrations or AI coding tools
• Terraform/Bicep
• Scala, PowerShell and YAML

The team

You'll join a highly technical, collaborative team working closely with a growing Data Science function. It's a great environment for someone who wants to deepen their Data Engineering experience while being exposed to some of the newest developments across data and AI.

Ideally you'll spend around two days per week in the Manchester office, but it's an output-driven environment with plenty of autonomy.

Diversity & Inclusion

We welcome applications from people of all backgrounds and experiences. If the role interests you but you don't tick every box, we'd still encourage you to apply. Different experiences, perspectives and routes into technology are valued.