PERM - Staff Data Engineer

WeDoTech; City Of London; Hybrid; GBP 84,000 - 140,000; Full time

Job summary

WeDoTech in London is partnering with a major UK financial services organisation to modernise its data estate and AI-driven engineering capabilities.

As Staff Data Engineer you will lead hands-on delivery, design scalable data architectures and mentor multiple teams through large-scale transformations.

This role combines technical leadership with coding, requiring deep Python, PySpark and Spark expertise and strong collaboration with senior stakeholders.

Qualifications

• Extensive hands-on Python, PySpark and Apache Spark expertise.
• Deep experience with AWS data engineering, particularly AWS Glue.
• Strong experience with Apache Airflow and complex data pipeline orchestration.
• Proven experience delivering large-scale data transformation, migration or modernisation programmes.
• Strong software engineering fundamentals and scalable, maintainable solutions.
• Experience operating at Staff, Principal or equivalent leadership level.
• Ability to independently shape technical direction, challenge existing approaches and influence multiple engineering teams.
• A genuine desire to remain deeply hands-on with engineering and code.
• Experience adopting modern AI-assisted development tools, such as Claude Code, within engineering workflows.
• Strong communication skills and the ability to engage with senior technical stakeholders.
• Experience with Snowflake and delivering data solutions within financial services or another highly regulated environment would be particularly beneficial.

Responsibilities

• Providing Staff-level technical leadership across major data engineering and transformation initiatives.
• Remaining deeply hands-on with Python, PySpark and Apache Spark.
• Leading the modernisation and migration of large-scale data pipelines across AWS.
• Designing and challenging technical approaches across AWS Glue, Airflow and associated technologies.
• Shaping scalable data architectures, reusable engineering frameworks and development standards.
• Supporting the design and delivery of new data products.
• Championing AI-assisted development tools to improve engineering productivity and accelerate delivery.
• Collaborating with senior technical stakeholders and engineering teams to challenge existing thinking and drive technical improvements.

Skills

Python

PySpark

Apache Spark

AWS

AWS Glue

Airflow

Data architecture

Leadership

Hands-on coding

AI-assisted development

Claude Code

Tools

AWS Glue

Apache Airflow

Snowflake

AWS

Claude Code

Job description

Staff Data Engineer | Permanent | Up to £140,000 + Bonus

Location: Hybrid – London (3 days per week on-site)

Salary: Up to £140,000 + c.20% bonus + additional benefits

Employment: Permanent

We're supporting a major UK financial services organisation as they undertake a significant programme of data engineering modernisation and transformation, alongside substantial investment in AI-driven engineering capabilities.

We're looking for an exceptional Staff Data Engineer who combines deep hands‑on engineering expertise with the technical leadership to challenge existing approaches, shape solutions and raise engineering standards across multiple teams.

This is a high-impact opportunity to help modernise a large-scale data estate, develop new data products and influence how modern engineering practices are adopted across the organisation.

Crucially, this is a hands‑on technical leadership role, not a purely strategic or architectural position.

The opportunityYou'll play a central role in transforming the organisation's data engineering capabilities, including the modernisation and migration of thousands of existing data pipelines.

Working across multiple engineering teams, you'll help establish scalable data engineering frameworks, accelerate delivery through AI-assisted development and support the organisation's ambition to build greater internal data engineering capability.

Key responsibilities:

• Providing Staff-level technical leadership across major data engineering and transformation initiatives.
• Remaining deeply hands‑on with Python, PySpark and Apache Spark.
• Leading the modernisation and migration of large-scale data pipelines across AWS.
• Designing and challenging technical approaches across AWS Glue, Airflow and associated technologies.
• Shaping scalable data architectures, reusable engineering frameworks and development standards.
• Supporting the design and delivery of new data products.
• Championing AI-assisted development tools to improve engineering productivity and accelerate delivery.
• Collaborating with senior technical stakeholders and engineering teams to challenge existing thinking and drive technical improvements.

What we're looking for

• Extensive hands‑on Python, PySpark and Apache Spark expertise.
• Deep experience with AWS data engineering, particularly AWS Glue.
• Strong experience with Apache Airflow and complex data pipeline orchestration.
• Proven experience delivering large-scale data transformation, migration or modernisation programmes.
• Strong software engineering fundamentals and the ability to design scalable, maintainable engineering solutions.
• Experience operating at Staff, Principal or equivalent technical leadership level.
• Ability to independently shape technical direction, challenge existing approaches and influence multiple engineering teams.
• A genuine desire to remain deeply hands‑on with engineering and code.
• Experience adopting modern AI-assisted development tools, such as Claude Code, within engineering workflows.
• Strong communication skills and the ability to engage with senior technical stakeholders.
• Experience with Snowflake and delivering data solutions within financial services or another highly regulated environment would be particularly beneficial.

Why consider this opportunity?

This is an opportunity to take a leading technical role in an organisation making a significant investment in modernising its data estate and transforming its engineering capabilities.

You'll have the opportunity to influence technical direction across multiple teams, tackle complex data engineering challenges and help establish modern engineering practices, including the adoption of AI-assisted development.

We're particularly interested in engineers who can operate across technical strategy and hands‑on delivery — individuals who can challenge the thinking, define the approach and then get into the code to prove and deliver the solution.

The permanent nature of the position also offers the opportunity to take long‑term ownership of technical decisions, develop internal engineering capability and play a meaningful role in the organisation's wider transformation.

Salary: Up to £140,000 + c.20% bonus and additional benefits