Senior Data Engineer

First Central Insurance & Technology Group; United Kingdom; On-site; GBP 55,000 - 75,000; Full time

Job summary

A leading insurance company in the UK is seeking a Senior Data Engineer to build technical data solutions and mentor fellow engineers. This role involves designing complex data pipelines and ensuring data quality. Ideal candidates will have extensive experience with Azure and big data technologies like Databricks. The company offers a supportive work environment and career growth opportunities.

Qualifications

• Extensive experience developing and implementing end-to-end data solutions in the cloud, preferably Azure.
• Experience with big data technologies such as Databricks and/or Synapse Analytics, using PySpark.
• Proactive, optimistic and collaborative with strong time management and stakeholder management skills.

Responsibilities

• Contribute to low-level design of data solutions ensuring they align with standards.
• Develop high-complexity, secure data pipelines from diverse data sources.
• Coach and mentor Data Engineers & Associate Engineers.

Skills

Problem solving

Collaboration

Time management

Stakeholder management

Tools

Azure

Databricks

PySpark

Python

Job description

Overview

We\'re 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an excellent service. And that\'s the same for our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too!

Ready to take the next step in your data career within a fast-paced, dynamic environment? We\'re looking for a passionate and experienced Senior Data Engineer to join our forward-thinking technology and data teams.

Responsibilities

In this hands-on technical role, you\'ll be responsible for building technical data solutions for various projects and persistent data products. You\'ll design and implement complex data pipelines and manage database population, ensuring the data solutions align with the technical design and overall data platform patterns.

While the role does not include direct people management responsibilities, you\'ll provide invaluable support, coaching and mentoring to data engineers and associate engineers, helping them develop and grow their skills.

Core to the role is contributing to low-level design from solution architecture artefacts, assuring the quality of data pipelines and database solutions, developing high-complexity, secure, governed data pipelines, and building data observability and quality by design into pipelines. You\'ll also create BI solutions including data marts and visualisation in recognised BI tools.

You\'ll build data models to meet business needs, perform unit testing, contribute to coding standards, maintain appropriate documentation, and coach Data Engineers & Associate Data Engineers. You\'ll work across data lake, data warehouse, and analytics applications, ensuring data is cleansed, mapped, transformed and optimised for storage.

What\'s involved

• Contribute to the low-level design of data solutions by translating high-level solution architecture artefacts into workable designs and work packages.
• Assure the quality of data pipelines and database solutions, aligning with designs, patterns and standards.
• Contribute to coding standards and design patterns for data engineers to follow, ensuring safe, high-quality, compliant code.
• Develop high-complexity, secure, governed data pipelines from diverse data sources (on-premise and cloud).
• Ensure data is cleansed, mapped, transformed and optimised for storage to meet business and technical use cases.
• Build data observability and data quality by design into pipelines and promote self-testing to identify processing issues.
• Pipe and transform data into data lake storage, physical database models and reporting structures across data lake, data warehouse and analytics applications.
• Design physical data models that meet business needs and optimise storage, enabling re-use.
• Perform unit testing, peer-review code, and ensure testing completeness for data solution deliverables.
• Maintain effective documentation that brings transparency to pipelines and code.
• Coach and mentor Data Engineers & Associate Data Engineers.
• Create high-complexity BI solutions including data marts, semantic layers, and reporting/visualisation in BI tools such as Power BI.

Experience & knowledge

• Extensive experience developing and implementing end-to-end data solutions in the cloud, preferably Azure.
• Experience with big data technologies such as Databricks and/or Synapse Analytics, using PySpark.
• Solution design experience across end-to-end data solutions (sourcing to consumption).
• Experience with Azure services such as Data Factory, Azure Functions, ADLS Gen2, Key Vault, Synapse Dedicated SQL Pool and Azure SQL.
• Experience using programming languages such as Python, C# and PowerShell.
• Understanding of infrastructure as code; experience with data modelling techniques and operational/documentation practices.
• Experience of testing techniques and tools; excellent communicator with the ability to build rapport.
• Keen interest in data with active external networks; drives value from data across the business.
• Extensive experience of the full data lifecycle and participation in CI/CD methods.
• Experience working in an agile environment with self-managing teams; comfortable leading sprint teams.
• Good coaching skills and ability to help others grow and develop.

Skills & qualifications

• An excellent creative problem solver who can manage multiple priorities.
• Able to turn ambiguous business challenges into well-formed data solutions.
• Proactive, optimistic and collaborative with strong time management and stakeholder management skills.
• Supportive team player who helps others deliver shared outcomes and enjoys mentoring less experienced engineers.
• Commitment to standards and high-quality code, with a focus on continuous improvement.

This is just the be beginning. Imagine where you could end up! The journey\'s yours to shape.

What can we do for you?

People first. Always. We\'re passionate about our colleagues and offer an extraordinary working environment, with energetic, inspirational, and supportive workplaces. See our perks in full here.

Intrigued? Our Talent team can tell you everything you need to know about what we want and what we\'re offering, so feel free to get in touch.