Lead Data Engineer

Onyx-Conseil; Leeds; Hybrid; GBP 63,000 - 75,000; Full time

Job summary

A technology solutions company seeks a Data Engineering Manager in Leeds. You will lead a remote team focusing on scalable cloud data solutions using Databricks, Python, and SQL. Responsibilities include overseeing data pipeline development, ensuring efficient processes, and mentoring the team. Proven leadership and strong technical skills are essential for this role, offering a salary of up to £75,000 per annum.

Qualifications

• Hands-on experience with Databricks or similar data engineering platforms.
• Strong Python and SQL skills in data engineering contexts.
• Expertise in data modelling and building analytics-ready datasets.
• Experience with AWS or Azure cloud data services.
• Proven leadership or management experience in data engineering teams.
• Excellent communication and stakeholder management skills.
• Knowledge of agile delivery methodologies and DevOps tooling.

Responsibilities

• Manage the development of data pipelines and solutions using Databricks, Python, and SQL.
• Oversee the design and maintenance of data models that support analytics and business intelligence needs.
• Ensure efficient ELT/ETL processes on AWS or Azure cloud platforms.
• Collaborate with analysts, architects, and stakeholders to deliver high-quality data products.
• Promote best practices in testing, CI/CD, version control, and infrastructure as code.
• Mentor and support the data engineering team, taking accountability for technical delivery and decision-making.
• Drive continuous improvements in platform performance, cost-efficiency, and reliability.

Skills

Databricks

Python

SQL

Data Modelling

AWS

Azure

Agile methodology

DevOps tooling

Leadership

Stakeholder management

Tools

Terraform

CloudFormation

Job description

Data Engineering Manager


Location: Leeds (Mostly Remote - 1 Day On-Site)


Salary: Up to £75,000 per annum

About the Role

Are you an experienced Senior Data Engineer ready to step into a managerial role? We are seeking a Data Engineering Manager to lead our Leeds-based team, working mostly remotely with one day on-site per week.

In this role, you will manage the design and delivery of scalable, cloud-based data solutions using Databricks, Python, and SQL. You will lead and mentor a team of data engineers, drive engineering best practices, and own the data architecture strategy in an agile environment.

Key Responsibilities

• Manage the development of data pipelines and solutions using Databricks, Python, and SQL





• Oversee the design and maintenance of data models that support analytics and business intelligence needs





• Ensure efficient ELT/ETL processes on AWS or Azure cloud platforms





• Collaborate with analysts, architects, and stakeholders to deliver high-quality data products





• Promote best practices in testing, CI/CD, version control, and infrastructure as code





• Mentor and support the data engineering team, taking accountability for technical delivery and decision-making





• Drive continuous improvements in platform performance, cost-efficiency, and reliability

Key Requirements

• Hands-on experience with Databricks or similar data engineering platforms





• Strong Python and SQL skills in data engineering contexts





• Expertise in data modelling and building analytics-ready datasets





• Experience with AWS or Azure cloud data services





• Proven leadership or management experience in data engineering teams





• Excellent communication and stakeholder management skills





• Knowledge of agile delivery methodologies and DevOps tooling

Desirable Skills

• Experience with infrastructure-as-code tools such as Terraform or CloudFormation





• Familiarity with CI/CD pipelines and orchestration tools





• Understanding of data governance and quality controls





• Experience working in regulated or large-scale environments