Lead Data Engineer - Architect Data Pipelines & Platforms

develop; Metropolitan Borough of Solihull; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Performance bonus

Healthcare

Employer pension

25 days holiday

Flexible benefits

Employee discounts

Enhanced family leave

Onsite gym

Job summary

develop is seeking an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a large retail brand. You will lead the design and delivery of scalable data platforms, set engineering standards, and mentor engineers.

You will work with data, product and technology stakeholders to solve complex problems and deliver high-quality data solutions, guiding architecture choices and aiding in technical hiring and onboarding.

Qualifications

• Extensive experience leading data engineering teams and architectures.
• Proven ability to design scalable batch and streaming pipelines.
• Strong Python and SQL with a software engineering mindset.

Responsibilities

• Lead the technical design and architecture of data pipelines and platform components.
• Establish standards across code quality, testing, security, observability, and documentation.
• Drive technical discovery, design sessions, code reviews and debt resolution.
• Design and deliver scalable batch and streaming data solutions.
• Mentor and coach engineers, supporting technical development and knowledge sharing.
• Make informed recommendations on tools, technologies and architectural approaches.
• Champion data quality, governance, security and compliance by design.
• Collaborate with data, product and technology teams to deliver joined-up solutions.
• Contribute to technical hiring, onboarding and team capability development.

Skills

Python

SQL

Data engineering

Cloud data platforms

Communication

Tools

GCP BigQuery

Dataflow

Pub/Sub

Looker

Airflow/Cloud Composer

Terraform

Dataform

Job description

develop is seeking an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a large retail brand. You will lead the design and delivery of scalable data platforms, set engineering standards, and mentor engineers.

You will work with data, product and technology stakeholders to solve complex problems and deliver high-quality data solutions, guiding architecture choices and aiding in technical hiring and onboarding.