Lead Data Engineer
Quilter plc
A complete application in a minute — tailored resume and cover letter, ready to send.
Benefits offered by this job
Job summary
Quilter plc leads data engineering with a strong focus on scalable, secure data platforms. The Lead Data Engineer within the Chief Data Office guides domain solutions, defines engineering standards, and drives data products that generate actionable insights across the organisation.
You will mentor teams, collaborate with product owners and stakeholders, and ensure delivery aligns with regulatory requirements and enterprise data governance.
Qualifications
- Degree in a technical discipline or equivalent practical experience.
- Proven senior/lead data engineer experience in complex enterprise environments.
- Strong hands-on data platforms and pipelines with Databricks on Azure.
- Experience with batch and real-time processing, event-driven architectures.
- Ability to define standards, lead design delivery, and mentor engineers.
- Knowledge of data governance, data lineage, metadata management, and security.
Responsibilities
- Lead the technical delivery of major data initiatives and ensure high-quality outcomes.
- Translate business requirements into detailed technical designs and roadmaps.
- Define and evolve data engineering standards, patterns, and best practices.
- Mentor engineers, conduct code reviews, and promote engineering excellence.
- Collaborate with product, analytics, architecture, and governance teams to align with enterprise standards.
Skills
Education
Tools
Job description
The role sits within the Chief Data Office (CDO) and reports to the Data Engineering Manager. This role accelerates Quilter's data strategy by providing deep, technical leadership across one or more data domains; defining engineering standards, and delivering robust, scalable and secure solutions. The Lead Data Engineer plays a pivotal role in enabling business value through the design, development, and continuous improvement of data products and platforms that support actionable insights across the organisation. Working closely with engineers, architects, analysts, platform teams, and business stakeholders, the Lead Data Engineer is responsible for translating business requirements into robust technical solutions while driving engineering best practices and fostering a culture of technical excellence. The role requires deep technical expertise, strong problem-solving capabilities, and the ability to influence and guide both technical and non-technical stakeholders.
- Provide technical leadership for one or more data domains, acting as the primary engineering authority and subject matter expert.
- Define, champion, and evolve data engineering standards, patterns, and best practices to ensure robust, efficient, secure, and maintainable solutions.
- Lead solution design activities and guide the implementation of scalable data pipelines, data products, and platform capabilities.
- Review code, architecture, and engineering approaches to ensure consistency, quality, and alignment with strategic objectives.
- Remain hands-on when required, supporting the resolution of complex technical challenges and critical delivery issues.
- Lead the technical delivery of key initiatives and ensure solutions are delivered to a high standard, meeting business requirements and engineering principles.
- Translate business and product requirements into detailed technical designs and implementation plans.
- Collaborate with Data Engineering Managers, Product Owners, Architects, and stakeholders to prioritise and deliver work effectively.
- Drive adoption of modern engineering practices including automation, testing, CI/CD, source control, observability, and infrastructure-as-code.
- Contribute to roadmap planning by identifying technical opportunities, risks, dependencies, and improvement initiatives.
- Provide coaching, mentoring, and technical guidance to data engineers across the team.
- Support the development of engineering capability through knowledge sharing, technical reviews, pair programming, and communities of practice.
- Promote a collaborative and inclusive engineering culture focused on quality, learning, and continuous improvement.
- Act as a role model for engineering excellence and professional standards.
- Partner with product managers, analysts, data scientists, platform teams, and business stakeholders to understand requirements and deliver valuable outcomes.
- Work closely with architecture, platform, and governance teams to ensure solutions align with enterprise standards and strategic direction.
- Communicate technical concepts clearly to both technical and non-technical audiences.
- Influence decision-making by providing expert technical advice and recommendations.
- Promote operational stability, reliability, and performance of data products and pipelines through monitoring, alerting, automation, and incident management practices.
- Champion high standards of data quality, security, governance, and regulatory compliance.
- Identify and implement opportunities to improve platform efficiency, engineering productivity, and service resilience.
- Support root cause analysis and continuous improvement activities to enhance operational performance and service reliability.
- Degree in a technical discipline (e.g. computer science, engineering, maths, physics) or evidence of equivalent practical experience.
- Proven experience as a senior or lead data engineer within a complex enterprise environment, delivering large-scale data solutions.
- Strong hands-on experience designing, developing, and supporting data platforms and pipelines using Databricks within Azure.
- Experience implementing both batch and real-time data processing solutions, including event-driven and streaming architectures.
- Demonstrable experience leading the technical design and delivery of data products and platform capabilities across one or more business domains.
- Experience implementing and promoting data quality frameworks, service level objectives (SLOs), data contracts, and observability tooling.
- Strong understanding of data governance principles, including data lineage, metadata management, cataloguing, security, and access controls.
- Experience ensuring compliance with regulatory and organisational requirements, including GDPR, data retention, and information security controls.
- Proven ability to define technical standards, engineering patterns, and best practices and drive their adoption across engineering teams.
- Experience mentoring and coaching engineers, conducting code reviews, and promoting engineering excellence.
- Strong experience designing and implementing data models across multiple architectural layers, including Kimball dimensional modelling and Data Vault methodologies.
- Experience working within Agile delivery teams and collaborating with architects, product owners, analysts, and business stakeholders to deliver business outcomes.
- Strong technical leadership skills with the ability to influence engineering direction and drive best practices across teams.
- Strategic thinker with a curious mindset, strong analytical capability, and a passion for continuous improvement.
- Excellent problem- solving skills with the ability to resolve complex technical challenges and make sound engineering decisions.
- Strong communication skills, with the ability to explain complex technical concepts clearly to both technical and non-technical audiences.
- Databricks: Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables, Spark Declarative Pipelines and platform optimisation.
- Microsoft Fabric: Lakehouse, Data Warehouse, Real-Time Intelligence, Semantic Models, and Fabric administration concepts.
- Azure: ADLS Gen2, Azure Data Factory, Key Vault, Functions, Azure Monitor, and broader Azure data services.
- Spark: Deep understanding of Spark internals, distributed computing principles, optimisation techniques and features, performance tuning.
- Languages: Advanced Python and SQL development skills; Scala experience beneficial.
- CI/CD & DevOps: Azure DevOps, Git-based development workflows, Infrastructure as Code, automated testing, and deployment automation.
- Orchestration: Databricks Workflows, Azure Data Factory, Fabric pipelines, and other enterprise scheduling and orchestration frameworks.
- Data Architecture & Modelling: Strong knowledge of Kimball, Data Vault, medallion architecture, and modern lakehouse design patterns.
- Data Governance & Security: Experience implementing secure-by-design data solutions, platform governance controls, and operational monitoring practices.
- Passionate about engineering excellence and continuous learning.
- Acts as a trusted technical advisor and subject matter expert within their domain.
- Proactively identifies opportunities to improve platform capability, performance, reliability, and security.
- Collaborative and supportive team player who enjoys mentoring others and sharing knowledge.
- Takes ownership and accountability for technical outcomes while maintaining a pragmatic delivery-focused approach.
Quilter plc is a leading wealth management business, helping to enable brighter financial futures for every generation. Quilter oversees £157.4billion in customer investments (as of June 2026). It has an adviser and customer offering spanning financial advice, investment platforms, multi-asset investment solutions, and discretionary fund management. The business is comprised of two segments: Affluent and High Net Worth. Affluent encompasses the financial planning business, Quilter Financial Planning, the Quilter Investment Platform and Quilter Investors, the multi-asset investment solutions business. High Net Worth includes the discretionary fund management business, Quilter Cheviot, together with Quilter Cheviot Financial Planning - offering a highly personalised service to private clients, charities, trustees, and professional partners. Quilter Cheviot has presence throughout the UK, Ireland and Channel Islands. At Quilter we never stand still. Our foundations are rooted in our extraordinary expertise, which is trusted by hundreds of thousands of customers, but we have great ambitions to stay one step ahead and make an even greater difference to the people and communities we serve, including our colleagues. Our business is transforming, continually modernising, and becoming even more customer centric. So, if you want to be bold in the pursuit of your ambitions, bring new ideas, and challenge and evolve what we do, it’s the perfect time to join us.
We value diversity and strive to promote inclusivity in all aspects of our culture. We believe in equal opportunities for all, ensuring that no applicant encounters less favourable treatment based on anything but their skills, qualifications, experience, and potential. We celebrate the unique contributions of a diverse workforce and create a respectful, nurturing environment where every colleague can thrive.
- Do the right thing: We act with integrity and are proudly committed to going above and beyond in service of our clients and the support we provide our communities.
- Always curious: We continuously seek new ideas and knowledge so we’re one step ahead of our clients’ needs.
- We look for inspiration everywhere and encourage experimentation, recognising that this is how we create brilliant solutions for brighter futures.
- Embrace challenge: We aim high to transform our potential into meaningful outcomes. With ambition as our driving force and a steadfast commitment to growth, we succeed for the good of every generation.
- Stronger together: Combining our diverse talents, we accomplish more collectively than we ever could do alone. We speak openly, actively listen, and support each other, and constructively challenge and embrace new ideas.
- We seek empowerment and demonstrate ownership and trust, with the confidence to make impactful decisions.
- Holiday: 182 hours (26 days)
- Quilter Incentive Scheme: All employees are eligible to participate in incentive scheme, to incentivise business performance and their contribution.
- Pension Scheme: A non-contributory company pension scheme that can be boosted through personal contributions.
- Private Medical Insurance: Single cover as standard with options to increase cover to include your partner or children.
- Life Assurance: 4x your salary.
- Income Protection: 75% of salary, less state benefits, payable after 26 weeks of absence.
- Healthcare Cash Plan: Jersey employees only.
- In addition to our core benefits, we offer a range of flexible benefits to UK employees that you can choose from and pay for conveniently via a salary deduction.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
