Lead Data Engineer - MSI

Baldwin Risk Partners; Norwich; Remote; GBP 60,000 - 80,000; Full time

Job summary

A leading MGA is seeking a Lead Data Engineer to design and optimize data architecture and pipelines. The role requires extensive experience in data engineering principles, particularly within the Azure ecosystem, and emphasizes mentorship of junior staff. This position blends hands-on development and strategic planning to support advanced product initiatives.

Qualifications

• At least seven to ten years of data-related experience.
• Experience within the Azure ecosystem.
• Insurance industry experience preferred.

Responsibilities

• Overseeing design and optimization of data orchestration.
• Building infrastructure for data extraction and ingestion.
• Mentoring junior engineers and providing technical guidance.

Skills

Data orchestration

SQL

Big Data

Data processing techniques

Analytical skills

Python

Education

Bachelor’s degree in related field

Tools

Azure Data Factory

SPARK (PySpark)

Azure Synapse

Job description

Social network you want to login/join with:

Why MSI? We thrive on solving challenges. As a leading MGA, MSI combines deep underwriting expertise with insurer risk capacity to create specialized insurance solutions that empower distribution partners to meet customers’ unique needs. We have a passion for crafting solutions for the important risks facing individuals and businesses. We offer an expanding suite of products – from fully-digital embedded renters coverage to high-value homeowners insurance to sophisticated commercial coverages, such as cyber liability and habitational property – delivered through agents, brokers, wholesalers and other brand partners.

Our partners and customers count on us to deliver exceptional service through a dedicated team that makes rapid resolutions a priority. We simplify the insurance experience through our advanced technology platform that supports every phase of the policy lifecycle. Bring on your challenges and let us show you how we build insurance better.

The Lead Data Engineer will be responsible for, overseeing the design, building and optimization of

our data orchestration and data pipeline architecture, as well as optimizing data collection and flow for cross functional teams.

The ideal candidate is an experienced technical oriented Lead Data Engineer who enjoys data orchestration and optimizing data systems and building them from the ground up. This role requires a deep understanding of data management / data engineering principles as they apply to scalability, and design of dynamic flexible data orchestration patterns. Additionally, the Lead Data Engineer is responsible for mentoring junior engineers and providing technical guidance and a ‘design thinking’ perspective to the team members.

They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of contributing to the design of our company’s data architecture to support our next generation of products and data initiatives. You will be equally comfortable hands on building data systems as you are mentoring more junior team members.

• Maintaining and updating our Data Engineering Architecture.
• Designing and implementing ELT and ETL processes.
• Create and maintain optimal data orchestration architecture to support data initiatives.
• Provide thought leadership that helps the team improve implementation approaches that drive increased velocity of delivery
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and ingestion of data from a wide variety of data sources using SQL, Azure Data Factory, Spark / Synapse technologies.
• Support analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Technology teams to assist with data-related technical issues and support their data infrastructure needs.
• Create data tools for analytics and data scientist team members that assist them in executing and optimizing data projects.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Collaborating with colleagues for the purpose of collecting and structuring data.

Collect, audit, compile, and validate data from multiple sources.

• Communicate internally and with clients externally to collect and validate data as well as answer questions regarding data.
• Apply advanced knowledge and understanding of concepts, principals, and technical capabilities to manage a wide variety of projects.
• Recommends new practices, processes and procedures.
• Provides solutions that may set precedent or have significant impact.
• Build automation and additional efficiencies into manual efforts.

Education, Experience, Skills and Abilities Requirements:

• Bachelor’s degree in related field preferred, equivalent years’ experience considered.
• At least seven to ten years of data related or analytical work experience in a Data Engineer role, preferably three of those within the Azure ecosystem.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of data platforms.
• Advanced understanding of and experience implementing data lakes, lake houses.
• Advanced understanding and experience with file storage layer management in data lake environment, including parquet and delta file formats.
• Solid experience with SPARK (PySpark) language, and data processing techniques.
• Solid Understanding of and experience with AZURE SYNAPSE tools and services.
• Some knowledge of Python preferred.
• Strong analytic skills related to working with structured, semi-structured, and unstructured datasets and blob storage.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Insurance industry experience preferred.

Special Working Conditions:

Fast paced, multi-tasking environment.

This position description is intended to describe the level of work required of the person performing in the role and is not a contract. The essential responsibilities are outlined; other duties may be assigned as needs arise or as required to support the organization. All requirements may be modified to reasonably accommodate physically or mentally challenged colleagues.

#LI-BM1

#LI-REMOTE

Click for some insight into our culture!