Lead Data Engineer: Pipelines, Governance & Cloud

AND Digital; Greater London; Hybrid; GBP 75,000 - 105,000; Full time

Benefits offered by this job

Blended Working

Dedicated career scrum team

Training & upskilling budget

Team socials

Autonomy to drive data architecture

Job summary

AND Digital is seeking a Lead Data Engineer for a 12 Month FTC to shape and evolve data capabilities underpinning the Accounts platform. You will provide technical leadership across data pipelines, governance and architecture, while mentoring engineers and promoting best practices.

You will work across data engineering, architecture and platform delivery, ensuring data is accurate, accessible and trusted. Strong SQL, Python and IaC experience are essential in this role.

Qualifications

• Strong SQL skills and extensive experience working with relational and/or modern database technologies.
• Proven commercial experience developing data solutions using Python.
• Strong understanding of modern data engineering principles, practices and design patterns.
• Experience designing and implementing ETL/ELT pipelines and data processing frameworks.
• Experience working with data quality, reconciliation, monitoring and operational data management.
• Experience contributing to data architecture and governance standards.
• Experience with Infrastructure as Code, ideally using Terraform.
• Experience working with cloud-based data platforms and modern engineering tooling.
• Experience leading technical delivery and making pragmatic architectural decisions in a complex technology environment.

Responsibilities

• Lead the design, development and ongoing evolution of robust, scalable data ingestion and processing pipelines.
• Provide technical leadership across ETL/ELT processes, data transformation and data integration services.
• Design and implement operational data stores and reference data services that support the Accounts platform and its consumers.
• Establish and maintain effective data quality, validation, monitoring, reconciliation and observability controls.
• Contribute to the evolution of data architecture, ensuring solutions are scalable, resilient, secure and aligned with wider technology strategy.
• Define and promote data engineering standards, patterns and best practices across the team.
• Support the development and implementation of data governance, lineage and data management practices.
• Work closely with software engineers, architects, product teams and other stakeholders to understand data requirements and translate them into effective technical solutions.
• Provide technical guidance and mentoring to other data engineers, helping to raise engineering capability and consistency across the team.
• Champion automation, Infrastructure as Code and modern engineering practices across the data landscape.
• Identify opportunities to improve the reliability, performance, maintainability and cost-effectiveness of data services and pipelines.

Job description

AND Digital is seeking a Lead Data Engineer for a 12 Month FTC to shape and evolve data capabilities underpinning the Accounts platform. You will provide technical leadership across data pipelines, governance and architecture, while mentoring engineers and promoting best practices.

You will work across data engineering, architecture and platform delivery, ensuring data is accurate, accessible and trusted. Strong SQL, Python and IaC experience are essential in this role.