Senior Data Engineer

develop; Metropolitan Borough of Solihull; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Performance bonus

Healthcare

Employer pension

25 days holiday

Flexible benefits

Employee discounts

Enhanced family leave

Onsite gym

Job summary

develop is seeking an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a large retail brand. You will lead the design and delivery of scalable data platforms, set engineering standards, and mentor engineers.

You will work with data, product and technology stakeholders to solve complex problems and deliver high-quality data solutions, guiding architecture choices and aiding in technical hiring and onboarding.

Qualifications

• Extensive experience leading data engineering teams and architectures.
• Proven ability to design scalable batch and streaming pipelines.
• Strong Python and SQL with a software engineering mindset.

Responsibilities

• Lead the technical design and architecture of data pipelines and platform components.
• Establish standards across code quality, testing, security, observability, and documentation.
• Drive technical discovery, design sessions, code reviews and debt resolution.
• Design and deliver scalable batch and streaming data solutions.
• Mentor and coach engineers, supporting technical development and knowledge sharing.
• Make informed recommendations on tools, technologies and architectural approaches.
• Champion data quality, governance, security and compliance by design.
• Collaborate with data, product and technology teams to deliver joined-up solutions.
• Contribute to technical hiring, onboarding and team capability development.

Skills

Python

SQL

Data engineering

Cloud data platforms

Communication

Tools

GCP BigQuery

Dataflow

Pub/Sub

Looker

Airflow/Cloud Composer

Terraform

Dataform

Job description

The Role

We’re looking for an experienced Lead Data Engineer to provide hands‑on technical leadership within a data engineering team for a large retail brand. You’ll lead the design and delivery of scalable, resilient data platforms and pipelines, set engineering standards, shape technical direction, and mentor engineers. You’ll work closely with engineering, data and business stakeholders to solve complex problems and deliver high‑quality data solutions.

Key Responsibilities

• Lead the technical design and architecture of data pipelines, models and platform components.
• Establish standards across code quality, testing, security, observability and documentation.
• Drive technical discovery, design sessions, code reviews and resolution of technical debt.
• Design and deliver scalable batch and streaming data solutions.
• Mentor and coach engineers, supporting technical development and knowledge sharing.
• Make informed recommendations on tools, technologies and architectural approaches.
• Champion data quality, governance, security and compliance by design.
• Collaborate with data, product and technology teams to deliver joined‑up solutions.
• Contribute to technical hiring, onboarding and team capability development.

What You’ll Bring

• Deep experience with GCP, particularly BigQuery, Dataflow, Pub/Sub, Cloud Storage and Cloud Composer/Airflow.
• Advanced Python and SQL skills with a strong software engineering mindset.
• Proven experience building complex, scalable batch and event‑driven pipelines.
• Experience with Dataform or similar SQL‑based transformation tooling.
• Understanding of CI/CD, Git, automated testing and infrastructure as code, particularly Terraform.
• Experience implementing data quality, monitoring, observability and alerting.
• Ability to lead architecture decisions and communicate technical trade‑offs clearly.

Desirable: Spark/DataProc, Looker or similar BI tools, analytics engineering, data mesh/platform architecture, GCP cost optimisation, and experience within retail or e‑commerce environments.

• Performance‑related bonus
• Healthcare and employer pension contributions
• 25 days’ holiday plus birthday and bank holidays
• Flexible benefits, including EV, dental, cycle‑to‑work and holiday trading schemes
Employee discounts and wellbeing support
• Enhanced family leave and life assurance
• Onsite gym membership, lunch and coffee facilities