Principal Data Engineer

Markerstudy Group; Salford; Hybrid; GBP 65,000 - 90,000; Full time

Benefits offered by this job

Starting with 25 days annual leave

Yearly bonus

Private Medical Insurance

Life Assurance 4x annual salary

Collaborative, fast paced working env

Hybrid Working

Modern, vibrant offices

Job summary

Markerstudy Group is seeking an experienced Data Engineer to join our hybrid team. The role is based in our UK offices in Manchester, Stoke or Peterborough, with two days a week on-site. You will own the Databricks Lakehouse platform, define standards, and lead scalable data product design across the enterprise.

You will mentor engineers, collaborate with stakeholders, and drive resilience, governance and performance improvements. Experience with Databricks, Delta Lake and Python is essential.

Qualifications

• Expert-level Databricks experience.
• Advanced PySpark development.
• Delta Lake architecture and optimisation.
• Databricks SQL expertise.
• Experience with data warehouse and dimensional modelling.
• Experience with Data Lake architectures.
• Familiarity with Databricks Genie.
• Strong Python development skills.

Responsibilities

• Primary technical contact on data for new projects; design data solutions.
• Design scalable Medallion Architecture (Bronze/Silver/Gold).
• Embed data governance, lineage, security and compliance in solutions.
• Provide technical leadership across multiple squads.
• Drive process improvements and automation with clear roadmaps.
• Coach and mentor team members; be escalation point for issues.
• Coordinate with vendors/data sources to align roadmaps.
• Work with stakeholders to meet needs and standards.
• Lead data product design end-to-end; be the technical expert.
• Focus on resilience, availability and recoverability for platforms.
• Manage resources: availability, capacity, performance and operability.
• Deliver robust data products within time and budget.
• Create clear documentation for data solutions.
• Communicate effectively with stakeholders and non-technical teams.
• Ensure high-quality, well-performing code.
• Provide training and develop data engineers.

Skills

Databricks

PySpark

Delta Lake

Databricks SQL

Data Warehouse

Dimensional Modelling

Data Lake

Databricks Genie

Python

Tools

Databricks

Job description

We have an exciting opportunity for an experienced Data Engineer to join our team.

This is a hybrid role, with requirements to work from either our Manchester, Stoke or Peterborough office twice a week.

Role Purpose

The Principal Data Engineer is the senior technical authority for the Databricks Lakehouse platform, responsible for defining engineering standards, architecture, and best practices across the enterprise data ecosystem. The role leads the design and delivery of scalable, secure, high-performing data products using Databricks, Delta Lake, Unity Catalog and Azure Data Services. The role provides technical leadership, ensuring solutions are robust, cost-effective, highly available, and aligned to business strategy and operational resilience requirements.

Key Accountabilities And Responsibilities

• Being the primary technical contact on data for all new projects, designing the detailed data solution and ensuring that requirements are understood and achievable.
• Design scalable Medallion Architecture solutions (Bronze, Silver, Gold).
• Ensure data governance, lineage, security, and compliance requirements are embedded within all solutions.
• Provide technical leadership across multiple squads.
• Driving continuous improvement through defining and redefining processes within the team, increased automation and providing detailed technical roadmaps of how to achieve our target state.
• Coaching and mentoring other members of the team in technical, delivery and behavioural areas, as well as being the primary escalation point for technical issues and decisions.
• Be the technical contact with vendors and data source supplier to ensure data roadmap is aligned.
• Closely working with stakeholders to ensure they are meeting our needs and delivering to required standards.
• Taking Lead on data product design and development and be the technical expert in all data engineering toolset.
• A heavy focus on resilience, availability and recovery to provide a solid platform that meets Operational Resilience needs for all our systems.
• Proactive technical resource management including availability, capacity, performance and operability.
• Be able to deliver robust data products end to end from requirement gathering to release within time and budget.
• Create clear documentation for the data solution.
• Communicate well with stakeholder and non-technical team.
• To ensure all codes are written are in high standard and perform well.
• Provide technical training and mentor data engineers and support their personal technical development.
• To ensure data dictionaries and models are robust and maintained on a regular basis.

Technical Skill

• Expert-level Databricks experience
• Advanced PySpark development
• Delta Lake architecture and optimisation
• Databricks SQL
• Data warehouse
• Dimensional modelling
• Data Lake
• Databricks Genie
• Python development

Key Skills, Experience And Knowledge

• Extensive Data Engineering experience within enterprise-scale data environments.
• Extensive hands-on experience delivering solutions using Databricks.
• Extensive experience designing and implementing Lakehouse architectures.
• Strong commercial experience using PySpark, Python, SQL and Delta Lake.
• Experience optimising Databricks workloads for performance, scalability and cost efficiency
• Previous experience delivering solutions within insurance, financial services or similarly regulated industries.
• Ability to understand and interpret business needs and how they translate to technical and functional requirements.
• Good project management skills; experience managing projects / requests with multiple senior stakeholders.
• Experience managing and mentoring engineering teams.

About Us

Markerstudy Group Services Limited is one of the largest Managing General Agents in the UK. With a strong presence in the UK motor insurance market, we specialise in niche motor cover, where our solid market knowledge and experience enables us to create highly targeted products.

Our success is underpinned by our underwriting strategy to identify and apply special risk factors to the customers’ advantage. That, and our skilled underwriting technicians who are friendly, accessible and empowered to make decisions.

Benefits

• Starting with 25 days annual leave which increases with service
• Opportunity for a yearly bonus
• Private Medical Insurance
• Life Assurance 4x annual salary
• Collaborative, fast paced working environment
• Hybrid Working
• Modern, vibrant offices