Databricks Engineer

twentyAI; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

twentyAI is partnering with a leading private equity firm to hire a Databricks Data Engineer for a London-based hybrid role. You will help ingest, transform and provide investment data across the firm’s data platform.

You will design and maintain robust data pipelines in Databricks, collaborate with engineering, analytics and business stakeholders, and ensure data quality and reliability across complex private markets datasets.

Qualifications

• Experience in private equity or private markets investment data environments.
• Hands-on Databricks development and maintenance experience.
• Strong knowledge of data pipelines, transformations, and data integration.
• Understanding of private markets data flows and operations.

Responsibilities

• Design, develop and maintain data pipelines and engineering solutions within Databricks.
• Build and optimise data ingestion, transformation and integration processes across multiple systems.
• Work with complex private markets and investment datasets, ensuring data quality, consistency and reliability.
• Collaborate with data engineering, analytics and business stakeholders to understand requirements and deliver scalable solutions.

Skills

Private equity / private markets

Databricks development

Data pipelines / transformations

Cross-system data integration

Tools

Databricks

Job description

London | Hybrid (1–2 days per week in the office)

The Opportunity

We are partnered with a leading Private Equity firm who are seeking a Databricks Data Engineer with a strong understanding of private markets and investment data. You will work across data integration, engineering and platform development, helping ensure critical investment data is efficiently ingested, transformed and made available to the wider firm.

Key Responsibilities

• Design, develop and maintain data pipelines and engineering solutions within Databricks.
• Build and optimise data ingestion, transformation and integration processes across multiple systems.
• Work with complex private markets and investment datasets, ensuring data quality, consistency and reliability.
• Collaborate with data engineering, analytics and business stakeholders to understand requirements and deliver scalable solutions.

Experience & Skills

• Experience working within private equity, private markets or an alternative investment management environment.
• Hands-on experience developing and maintaining solutions using Databricks.
• Strong understanding of private markets data, investment operations and associated data flows.
• Experience building data pipelines, performing data transformations and integrating information across multiple platforms.