Databricks Data Engineer — Private Markets Data Pipelines

twentyAI; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

twentyAI is partnering with a leading private equity firm to hire a Databricks Data Engineer for a London-based hybrid role. You will help ingest, transform and provide investment data across the firm’s data platform.

You will design and maintain robust data pipelines in Databricks, collaborate with engineering, analytics and business stakeholders, and ensure data quality and reliability across complex private markets datasets.

Qualifications

• Experience in private equity or private markets investment data environments.
• Hands-on Databricks development and maintenance experience.
• Strong knowledge of data pipelines, transformations, and data integration.
• Understanding of private markets data flows and operations.

Responsibilities

• Design, develop and maintain data pipelines and engineering solutions within Databricks.
• Build and optimise data ingestion, transformation and integration processes across multiple systems.
• Work with complex private markets and investment datasets, ensuring data quality, consistency and reliability.
• Collaborate with data engineering, analytics and business stakeholders to understand requirements and deliver scalable solutions.

Skills

Private equity / private markets

Databricks development

Data pipelines / transformations

Cross-system data integration

Tools

Databricks

Job description

twentyAI is partnering with a leading private equity firm to hire a Databricks Data Engineer for a London-based hybrid role. You will help ingest, transform and provide investment data across the firm’s data platform.

You will design and maintain robust data pipelines in Databricks, collaborate with engineering, analytics and business stakeholders, and ensure data quality and reliability across complex private markets datasets.