Enable job alerts via email!

Lead Data Engineer - MSI

TN United Kingdom

Norwich

On-site

GBP 60,000 - 100,000

Full time

27 days ago

Boost your interview chances

Create a job specific, tailored resume for higher success rate.

Job summary

An established industry player is seeking a Lead Data Engineer to oversee the design and optimization of data orchestration and pipeline architecture. In this hands-on role, you will leverage your extensive experience in data management and engineering to mentor junior engineers while driving innovative solutions for complex data challenges. You will be instrumental in building scalable data systems that support next-generation products and initiatives. If you thrive in a fast-paced environment and are passionate about data architecture, this opportunity is perfect for you.

Qualifications

  • 7-10 years of data-related experience, with 3 years in Azure ecosystem.
  • Experience in building and optimizing big data pipelines and architectures.

Responsibilities

  • Maintain and update Data Engineering Architecture.
  • Design and implement ELT and ETL processes.
  • Collaborate with stakeholders on data-related issues.

Skills

SQL
Data Engineering
Azure Data Factory
Spark/PySpark
Data Architecture
Data Management
Automation
Analytical Skills
Python
Big Data Pipelines

Education

Bachelor’s Degree in a Related Field

Tools

Azure Synapse
Data Lakes
Lake Houses
File Storage Management

Job description

Why MSI? We thrive on solving challenges. As a leading MGA, MSI combines deep underwriting expertise with insurer risk capacity to create specialized insurance solutions that empower distribution partners to meet customers’ unique needs. We have a passion for crafting solutions for the important risks facing individuals and businesses. We offer an expanding suite of products – from fully-digital embedded renters coverage to high-value homeowners insurance to sophisticated commercial coverages, such as cyber liability and habitational property – delivered through agents, brokers, wholesalers and other brand partners.

Our partners and customers count on us to deliver exceptional service through a dedicated team that makes rapid resolutions a priority. We simplify the insurance experience through our advanced technology platform that supports every phase of the policy lifecycle. Bring on your challenges and let us show you how we build insurance better.

Job Description: Lead Data Engineer

The Lead Data Engineer will be responsible for overseeing the design, building, and optimization of our data orchestration and data pipeline architecture, as well as optimizing data collection and flow for cross-functional teams.

The ideal candidate is an experienced, technically-oriented Lead Data Engineer who enjoys data orchestration, optimizing data systems, and building them from the ground up. This role requires a deep understanding of data management and engineering principles as they apply to scalability and the design of dynamic, flexible data orchestration patterns. Additionally, the Lead Data Engineer is responsible for mentoring junior engineers and providing technical guidance and a ‘design thinking’ perspective to team members.

They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The right candidate will be excited about contributing to the design of our company’s data architecture to support next-generation products and data initiatives. You will be hands-on in building data systems and mentoring team members.

Key Responsibilities:
  1. Maintain and update our Data Engineering Architecture.
  2. Design and implement ELT and ETL processes.
  3. Create and maintain optimal data orchestration architecture to support data initiatives.
  4. Provide thought leadership to improve implementation approaches and increase delivery velocity.
  5. Assemble large, complex data sets to meet business requirements.
  6. Identify and implement process improvements, automate manual efforts, and optimize infrastructure for scalability.
  7. Build infrastructure for data extraction, transformation, and ingestion using SQL, Azure Data Factory, Spark/Synapse.
  8. Support analytics tools that utilize the data pipeline for actionable insights.
  9. Collaborate with stakeholders (Executive, Product, Data, Tech teams) on data-related issues and infrastructure needs.
  10. Create data tools for analytics and data science teams.
  11. Work with data experts to enhance system functionality.
  12. Collect, audit, compile, and validate data from multiple sources.
  13. Communicate with internal and external clients regarding data collection and validation.
  14. Apply advanced technical knowledge to manage projects and recommend new practices.
  15. Build automation and efficiencies into manual processes.
Requirements:
  • Bachelor’s degree in a related field preferred; equivalent experience considered.
  • 7-10 years of data-related or analytical experience, with at least 3 years in Azure ecosystem.
  • Experience building and optimizing big data pipelines and architectures.
  • Proficiency in SQL, relational databases, and data platforms.
  • Experience with data lakes, lake houses, and file storage management (parquet, delta formats).
  • Proficiency with Spark (PySpark) and Azure Synapse tools.
  • Knowledge of Python preferred.
  • Strong analytical skills with large, structured and unstructured datasets.
  • Experience in supporting cross-functional teams in a dynamic environment.
  • Insurance industry experience is a plus.
Working Conditions:

Fast-paced, multi-tasking environment.

This description outlines the primary responsibilities and may be modified to accommodate organizational needs or challenged colleagues.

Get your free, confidential resume review.
or drag and drop a PDF, DOC, DOCX, ODT, or PAGES file up to 5MB.

Similar jobs

Lead Data Engineer

JR United Kingdom

West Midlands Combined Authority

Remote

GBP 70,000 - 100,000

Yesterday
Be an early applicant

Lead Data Engineer - Databricks

JR United Kingdom

West Midlands Combined Authority

Remote

GBP 50,000 - 80,000

Yesterday
Be an early applicant

Lead Data Engineer - Databricks

TN United Kingdom

Remote

GBP 60,000 - 80,000

6 days ago
Be an early applicant

Principal Data Engineer

Maxwell Bond

Remote

GBP 80,000 - 100,000

7 days ago
Be an early applicant

Lead Data Engineer - Microsoft Fabric

Columbus UK

Birmingham

Remote

GBP 60,000 - 80,000

7 days ago
Be an early applicant

Lead Data Engineer

Aker Systems Limited

Remote

GBP 70,000 - 90,000

10 days ago

Principal Data Scientist (Manager) - Remote

Datatech Analytics

London

Remote

GBP 58,000 - 72,000

Yesterday
Be an early applicant

Lead Data Engineer

TN United Kingdom

Remote

GBP 60,000 - 100,000

18 days ago

Lead Oracle Data Engineer

JR United Kingdom

Remote

GBP 60,000 - 80,000

11 days ago