Data Engineer Manager

TN United Kingdom; London; Hybrid; GBP 60,000 - 100,000; Full time

Job summary

An established industry player is seeking a Data Engineer Manager to lead the design and optimization of data solutions within a dynamic team. This role focuses on enhancing data infrastructure and ensuring alignment with organizational goals. You'll mentor engineers, develop innovative data applications, and oversee data governance frameworks. Join a forward-thinking company that is committed to supporting the UK's Net Zero target by 2050, while enjoying a variety of employee benefits and flexible working arrangements. If you're passionate about data and leadership, this opportunity is perfect for you.

Benefits

25 days' annual leave plus bank holidays

Company pension contribution

Enhanced maternity/paternity leave

Employee assistance programme

Special leave options (study leave, sabbatical)

Three days paid leave for volunteering

Season ticket loan scheme

Access to discounts through Work Perks

Family savings on days out

Gym discounts

Qualifications

• 3+ years of experience in Data Engineering and Governance frameworks.
• Strong expertise in Azure services and data security standards.
• Proficiency in managing teams and delivering technical solutions.

Responsibilities

• Mentor the data engineering team to implement tailored data solutions.
• Design and optimize end-to-end data pipelines on Azure.
• Collaborate with stakeholders to translate requirements into technical solutions.

Skills

Data Engineering

Data Architecture

Python

Azure

ETL/ELT Processes

DevOps Practices

Communication Skills

Stakeholder Management

Creative Problem-Solving

Analytical Skills

Education

Bachelor's Degree in Computer Science or related field

Relevant Certifications (e.g., Azure, Data Engineering)

Tools

Azure Databricks

SQL Data Warehouse

Power BI

Azure DevOps

Job description

Social network you want to login/join with:

Contract type: Permanent
Hours: Full time, 37.5 hours per week

Location: Canary Wharf

WFH policy: Employees are required to attend the office 2 days/week

Flexible working: Variety of flexible work patterns subject to line manager discretion, e.g., compressed 9-day fortnight.

Reports to: Lead Data Engineer
Deadline Note: We reserve the right to close the advert before the advertised deadline if there is a high volume of applications.

Role Summary:

The Data Engineer Manager is responsible for driving the design, development, and optimization of data solutions in LCCC’s data infrastructure. In addition to fostering the growth of a skilled team, you will play a pivotal role in delivering LCCC’s data applications, infrastructure, and services, ensuring they align with organizational goals and industry best practices.

As part of the Technology Hub within LCCC, the Data Engineer Manager will work closely with all teams across LCCC. The role is instrumental in defining and upholding a clear vision for data lifecycle management in line with LCCC’s strategic goal of becoming a center of expertise. Additionally, it ensures stewardship of LCCC’s data and technical architecture, fostering innovation and reliability across all data initiatives.

Key Responsibilities

• Mentor the data engineering team to design and implement complex, tailored data solutions supporting high-volume data processing across schemes and applications.
• Establish and support the technical vision and strategy for a robust data architecture aligned with LCCC’s strategy, with a focus on data security.
• Maintain operational support and monitoring systems to ensure reliable performance of critical systems in live environments.
• Facilitate continuous delivery practices and advocate for cloud solutions.
• Design, implement, and optimize end-to-end data pipelines on Azure, ensuring data quality, reliability, and security, integrating structured and unstructured data sources.
• Oversee project timelines, scope, and deliverables, actively monitoring progress and proactively addressing risks.
• Implement best practices for process improvements, cost optimization, and platform performance enhancement.
• Collaborate with stakeholders to understand requirements and translate them into technical solutions.
• Develop and implement a data governance framework to ensure data quality, security, and compliance.
• Evaluate impacts, perform design reviews, and approve technical designs as part of the design authority process.
• Maintain comprehensive documentation for all data engineering processes, pipelines, and systems, ensuring version control and traceability.
• Foster continuous learning and technological adoption, mentoring the team accordingly.

Skills, Knowledge, and Expertise

• Experience leading small teams of engineers.
• At least 3 years’ experience in Data Engineering, Data Architecture, and Governance frameworks.
• Minimum 3 years' experience with Python, preferably PySpark.
• Excellent communication and stakeholder management skills.
• Strong expertise in Azure (AWS considered for strong candidates): ADLS, Databricks, Stream Analytics, SQL DW, Synapse, Azure Functions, Serverless Architecture, ARM Templates, DevOps.
• Hands-on experience with ETL/ELT processes and data warehousing.
• Understanding of data security and compliance standards.
• Experience with DevOps practices and tools (e.g., CI/CD pipelines, Azure DevOps).
• Ability to simplify complex technical issues for non-technical stakeholders.
• Understanding business needs and providing insightful recommendations.
• Proficiency in delivering presentations and technical reports.
• Creative problem-solving skills for complex data challenges.
• Openness to new ideas, experimentation, and iteration.
• Ability to manage multiple priorities and adapt to change.
• Encourages open communication and cross-functional collaboration.
• Proficiency in Azure and AWS cloud platforms.
• Experience with Power BI for reporting is a plus.
• Strong analytical, decision-making, and organizational skills; effective influencing abilities.

Employee Benefits

We offer a full range of benefits, including:

• 25 days' annual leave plus bank holidays
• Recognition schemes for colleagues
• Company pension contribution
• Family-friendly policies, including enhanced maternity/paternity leave and shared parental benefits
• Employee assistance programme for confidential support
• Special leave options such as study leave, sabbatical, or public duties
• Three days paid leave annually for volunteering
• Season ticket loan scheme
• Access to discounts and cashback through “Work Perks”
• Family savings on days out, English Heritage, and gym discounts

The Low Carbon Contracts Company (LCCC) aims to support decarbonizing electricity generation and making energy more affordable, contributing to the UK’s Net Zero target by 2050.

Learn more about LCCC and our work by watching this video:
https://vimeo.com/1071699533

Please consider answering the optional diversity questions. At LCCC, we are committed to fostering a diverse and inclusive workplace where everyone can thrive. Your diversity data remains anonymous and helps us assess our outreach and inclusivity efforts.