Staff Data Engineer & Tech Lead - Data Modernisation (Hybrid)

WeDoTech; City Of London; Hybrid; GBP 84,000 - 140,000; Full time

Job summary

WeDoTech in London is partnering with a major UK financial services organisation to modernise its data estate and AI-driven engineering capabilities.

As Staff Data Engineer you will lead hands-on delivery, design scalable data architectures and mentor multiple teams through large-scale transformations.

This role combines technical leadership with coding, requiring deep Python, PySpark and Spark expertise and strong collaboration with senior stakeholders.

Qualifications

• Extensive hands-on Python, PySpark and Apache Spark expertise.
• Deep experience with AWS data engineering, particularly AWS Glue.
• Strong experience with Apache Airflow and complex data pipeline orchestration.
• Proven experience delivering large-scale data transformation, migration or modernisation programmes.
• Strong software engineering fundamentals and scalable, maintainable solutions.
• Experience operating at Staff, Principal or equivalent leadership level.
• Ability to independently shape technical direction, challenge existing approaches and influence multiple engineering teams.
• A genuine desire to remain deeply hands-on with engineering and code.
• Experience adopting modern AI-assisted development tools, such as Claude Code, within engineering workflows.
• Strong communication skills and the ability to engage with senior technical stakeholders.
• Experience with Snowflake and delivering data solutions within financial services or another highly regulated environment would be particularly beneficial.

Responsibilities

• Providing Staff-level technical leadership across major data engineering and transformation initiatives.
• Remaining deeply hands-on with Python, PySpark and Apache Spark.
• Leading the modernisation and migration of large-scale data pipelines across AWS.
• Designing and challenging technical approaches across AWS Glue, Airflow and associated technologies.
• Shaping scalable data architectures, reusable engineering frameworks and development standards.
• Supporting the design and delivery of new data products.
• Championing AI-assisted development tools to improve engineering productivity and accelerate delivery.
• Collaborating with senior technical stakeholders and engineering teams to challenge existing thinking and drive technical improvements.

Skills

Python

PySpark

Apache Spark

AWS

AWS Glue

Airflow

Data architecture

Leadership

Hands-on coding

AI-assisted development

Claude Code

Tools

AWS Glue

Apache Airflow

Snowflake

AWS

Claude Code

Job description

WeDoTech in London is partnering with a major UK financial services organisation to modernise its data estate and AI-driven engineering capabilities.

As Staff Data Engineer you will lead hands-on delivery, design scalable data architectures and mentor multiple teams through large-scale transformations.

This role combines technical leadership with coding, requiring deep Python, PySpark and Spark expertise and strong collaboration with senior stakeholders.