Senior Data Engineer

1st Central; Haywards Heath; Hybrid; GBP 55,000 - 75,000; Full time

Job summary

A leading insurance provider in the UK is seeking a talented Data Engineer to design and manage data solutions across cloud platforms. The ideal candidate will have extensive experience in data engineering, proficiency in Python and Azure services, and a passion for mentoring fellow data professionals. This role offers flexibility with remote working arrangements and a collaborative team environment.

Benefits

Flexible working arrangements

Supportive workplace culture

Qualifications

• Extensive experience with end-to-end data solutions in Azure.
• Proficiency in programming languages such as Python and C#.
• Expertise in big data technologies like Databricks and PySpark.

Responsibilities

• Contributing to the design of data solutions.
• Ensuring quality of data pipelines.
• Building data observability and quality by design into pipelines.
• Creating high complexity BI solutions in Power BI.

Skills

Data engineering

Cloud platforms

Python

Power BI

CI/CD

Agile methodologies

Tools

Azure Data Factory

Databricks

Azure Synapse

Azure SQL

Job description

Data & Technology

You'll have a deep passion for data, analytics and technology, along with proven experience in designing and managing metadata-driven pipelines.

Self-Starter

You proactively manage workloads, make informed decisions, and work independently to deliver results.

Mentorship

You'll lead by example, mentoring and inspiring fellow data professionals to grow and succeed.

Data Lifecycle Expertise

You'll appreciate the full data lifecycle and can demonstrate thoughtful upfront design while adhering to established standards and best practices.

Cloud Data Engineering

You'll bring deep expertise in data engineering across modern cloud platforms, delivering scalable and robust solutions.

What's involved:

1. Contributing to the low-level design of data solutions, translating high-level architecture into workable designs and work packages.
2. Assuring the quality of data pipelines and database solutions, ensuring alignment with designs, patterns, and standards.
3. Contributing to coding standards and design patterns for data engineers.
4. Developing high complexity, secure, governed, high-quality data pipelines from various data sources.
5. Ensuring data is cleansed, mapped, transformed, and optimized for storage to meet business and technical needs.
6. Building data observability and quality by design into pipelines, promoting self-testing and proactive issue detection.
7. Building solutions that transform data into data lake storage, physical database models, and reporting structures.
8. Designing physical data models to meet business needs and optimize storage.
9. Carrying out unit testing, peer reviewing, and ensuring testing completeness.
10. Maintaining documentation for transparency and understandability.
11. Coaching and mentoring Data Engineers & Associate Data Engineers.
12. Creating high complexity BI solutions, including data marts, semantic layers, and visualizations in tools like Power BI.

Extensive experience with end-to-end data solutions in Azure, big data technologies like Databricks and PySpark, and Azure services such as Data Factory, Synapse, Azure SQL, etc. is required. Proficiency in Python, C#, PowerShell, data modeling, CI/CD, agile methodologies, and excellent communication skills are also essential. The role involves supporting data value extraction, full data lifecycle management, and fostering a positive, collaborative team environment.

Our company, 1st Central, is a leading insurance provider utilizing smart data and technology, recognized as Insurance Employer of the Year 2024. We offer flexible working arrangements, including remote options within the UK and Guernsey, and a vibrant, supportive workplace culture.

If you're ready to advance your data career in a dynamic environment, apply now to join our innovative team!