Head of Data Engineering

Cornwallis Elt Ltd; City Of London; On-site; GBP 80,000 - 120,000; Full time

Job summary

A global Private Equity firm seeks an experienced Head of Data Engineering to lead their data transformation initiative. This role involves managing the development of a modern data platform using Databricks and Azure, focusing on data governance and team leadership to drive business growth through actionable insights.

Qualifications

• Proven experience managing the end-to-end build of an Azure Databricks platform.
• Background in Financial Services or similarly regulated industry is critical.
• Experience in managing high-performing teams in technology.

Responsibilities

• Lead the transformation to a data-first approach and manage the Data Engineering team.
• Progress the Databricks setup from PoC to production, ensuring optimization.
• Develop data pipelines and real-time reporting capabilities.

Skills

Python

Databricks

Azure

Data Governance

ETL

Data Pipelines

Job description

Head of Data Engineering – Databricks, Azure, Data Strategy, Data Governance, Leadership, Management, Azure, Python, ETL, Data Pipelines, Private Markets

A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking line management responsibilities for their Data Engineering team.

You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale.

This will also involve implementing a semantic layer within the platform for effective data management and organisation, as well as the development of associated data pipelines and real-time reporting & visualization capabilities.

Once the platform is in place, the business will then look to apply Data Science techniques with the aim of building a best-in-class data function that works in partnership with the Investment Team and actively provides deep, actionable insights to enable business growth.

The ideal candidate will demonstrate:

• A technical background in data engineering with experience of technologies including Python, Spark, Databricks and Azure cloud.
• Previous experience in managing the end-to-end build of an Azure Databricks platform
• Passionate about all aspects of data (data governance, data quality management etc.) and the positive impact ‘good data’ can bring to a business
• Demonstrable experience in setting up and managing high-performing technology teams, establishing and driving best practises and being able to still engage at a technical level
• Able to credibly and confidently engage with the business at all levels
• A background in Financial Services (Private Markets, Investment Banking, Investment Management) or similarly regulated indsutry such as Gambling/Online Gaming is critical

If you are a data enthusiast with experience in leading high-performing teams to develop modern data platforms, then this is a genuinely exciting time to join a growing business transitioning to a data-first approach.