Lead Data Engineer

TN United Kingdom; Clayton-le-Moors; On-site; GBP 60,000 - 100,000; Full time

Job summary

An established industry player is seeking a Lead Data Engineer to spearhead the design and maintenance of robust data infrastructures. This pivotal role involves overseeing data collection, storage, and processing to empower data-driven decision-making. The ideal candidate will lead a team of data engineers, fostering collaboration across departments to implement innovative data solutions and optimize data pipelines. With a focus on data integrity and security, you'll play a crucial role in aligning data initiatives with business objectives. If you thrive in dynamic environments and are passionate about data engineering, this opportunity is perfect for you.

Qualifications

• Strong knowledge of data warehousing and integration, with significant SQL experience.
• Experience in leading and mentoring junior team members in a fast-paced environment.

Responsibilities

• Lead the design and maintenance of scalable data pipelines for data extraction and transformation.
• Manage and mentor a team of data engineers, ensuring alignment with organizational goals.

Skills

Data Governance

SQL

Data Integration

Critical Thinking

Solution Creation

Team Leadership

Cloud Data Warehousing

Business Intelligence

Tools

Snowflake

DBT

Stitch

Fivetran

Snowpipe

Microsoft PowerBI

Python

Job description

The role of the Lead Data Engineer is to lead the design, development, and maintenance of the organisation's data infrastructure. The role involves overseeing the efficient, reliable, and secure collection, storage, and processing of data, which is critical for enabling data-driven decision-making across FGFS.

The Team Lead will help manage and mentor a team of data engineers, working collaboratively with cross-functional teams to implement data solutions, optimise data pipelines, and drive the overall data strategy, ensuring data integrity, security, and accessibility across FGFS. The Team Lead will also play a key role in strategic planning, process improvement, and aligning data initiatives with business goals.

Responsibilities

• Lead the design, development, and maintenance of scalable and efficient data pipelines for extracting, transforming, and loading data from multiple sources.
• Oversee the team's efforts in building and optimising data pipelines, ensuring alignment with organisational goals and performance standards.
• Manage and mentor a team of data engineers, providing guidance, support, and professional development opportunities.
• Oversee the diagnosis and resolution of complex data-related issues within the platform, ensuring prompt and effective support and maintenance of data systems.
• Implement best practices for monitoring and alerting to proactively address potential issues.
• Act as a primary liaison between the data engineering team, spoke analysts, and other key stakeholders to align data solutions with business needs.
• Ensure the team adheres to data governance policies, regulatory requirements, and data security best practices.
• Oversee the creation and maintenance of comprehensive documentation of data engineering processes.
• Regularly report on system performance, data quality, and pipeline health to senior management and other stakeholders.
• Drive the continuous improvement of data engineering practices by staying current with industry trends, emerging technologies, and best practices.
• Lead initiatives to implement innovative solutions and improvements, ensuring the data engineering team's approach remains efficient, scalable, and effective.

Qualifications

Required

• Good knowledge of enterprise data warehousing, data integration and Business Intelligence (BI) reporting.
• Well versed with cloud-based data warehouse solutions including Snowflake.
• Knowledge of DBT and different ingest tools such as Stitch, Fivetran and Snowpipe.
• Significant SQL experience.
• Experience with leading and mentoring junior members of a team.
• Comfortable with working in a high pressure environment and working to tight deadlines.
• Strong understanding of Data Governance best practice.

Desirable

• Working knowledge of Python.
• Experience with modelling data for Microsoft PowerBI.
• Knowledge of best practice for data modelling and architecture.
• Prior Financial Services experience.

Non-technical required:

• Excellent critical thinking and solution creation skills.
• Ability to deliver in a fast-paced and dynamic environment.

As FGFS is within a regulated environment any offer is subject to satisfactory background checks including criminal record check, credit check, and employment references.