Data Engineer - London/Hybrid - TWE41666

ZipRecruiter; London; On-site; GBP 50,000 - 80,000; Full time

Job summary

A leading global law firm is initiating a major digital transformation and seeking a skilled Data Engineer to design efficient data pipelines. This role involves working with Azure technologies and collaborating within a dynamic team dedicated to advancing data capabilities in a prestigious environment.

Benefits

Collaborative team environment

Opportunity to work with latest Azure tools

Qualifications

• Experience with Microsoft Azure data tools, especially Data Factory and Synapse.
• Proficiency in Infrastructure as Code, preferably Terraform.
• Comfortable in a complex environment evolving from legacy systems.

Responsibilities

• Design and develop end-to-end data pipelines for analytics and machine learning workflows.
• Implement validation, cleansing, and monitoring frameworks to improve data quality.
• Build scalable data workflows within the Azure ecosystem.

Skills

Microsoft Azure data tools

Distributed processing frameworks

Problem-solving

Infrastructure as Code

Collaboration

Tools

Terraform

Data Factory

Synapse

Apache Spark

Job description

Job Description

twentyAI are partnering with a globally renowned law firm currently undergoing a major digital and data transformation. With a deep-rooted legacy in legal excellence and a strong global footprint, the firm is currently modernising their data capability and building a new data platform.

About the Role

You will be part of a diverse group of engineers and machine learning experts, working with cutting-edge Azure cloud technologies, including Microsoft Fabric and related services. Your mission is to design reliable, efficient data pipelines that enable the business to access trusted, well-structured data. You will also focus on building and scaling the core data infrastructure that supports advanced analytics and machine learning efforts across the business.

Responsibilities

• Design and develop end-to-end data pipelines that ingest, transform, and prepare data for analytics and machine learning workflows.
• Work with Infrastructure as Code, primarily Terraform, to automate and manage cloud infrastructure, enabling repeatable and reliable deployment processes.
• Collaborate closely with data scientists, MLEs, and business teams in an agile environment to deliver data solutions that support key firm initiatives.
• Build scalable and efficient batch and streaming data workflows within the Azure ecosystem.
• Apply distributed processing techniques using Apache Spark to handle large datasets effectively.
• Help drive improvements in data quality, implementing validation, cleansing, and monitoring frameworks.
• Contribute to the firm’s efforts around data security, governance, and compliance by adopting best practices and integrating security controls in pipelines.
• Identify bottlenecks and optimise performance across data pipelines and cloud infrastructure.
• Participate in the ongoing migration from legacy systems to modern data platforms.

Your Background

• Experience with Microsoft Azure data tools — especially Data Factory and Synapse.
• Familiarity with Microsoft Fabric will be beneficial. Otherwise, experience with platforms like Databricks or Snowflake is also valued.
• Proficiency in Infrastructure as Code, preferably with Terraform, and understanding of CI/CD pipelines in a data engineering context.
• Practical knowledge of distributed processing frameworks, particularly Spark.
• Comfortable working in a complex environment that is evolving from legacy systems toward a modern data architecture.
• Strong problem-solving skills and the ability to work collaboratively in a cross-functional agile team.
• Exposure to data governance, security, and compliance principles is desirable.
• Background in industries where data security and governance are paramount is a plus, such as financial services, professional services, or legal.

Why join?

• Work at the forefront of a major digital transformation in a prestigious global legal organisation.
• Be part of a collaborative team that values innovation, continuous learning, and practical engineering approaches.
• Opportunity to work with the latest Azure tools and technologies - including Microsoft Fabric.

If you’re passionate about building scalable, secure data platforms and enjoy working in a dynamic, supportive environment, we want to hear from you. Click the Apply button or send your CV to [e-mail] directly.