Lead Data Engineer - Snowflake Platform for Private Markets

twentyAI; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

twentyAI in London is seeking an experienced Lead Data Engineer to own a brand-new greenfield enterprise data platform built on Snowflake. You will design, implement and govern data pipelines, models and governance to unify investment, portfolio and financial data for advanced analytics.

You will work closely with technology and business stakeholders to deliver scalable, secure, cost-efficient cloud-based infrastructure, empowering AI-driven insights and executive reporting.

Qualifications

• Extensive hands-on experience designing, implementing and managing Snowflake data platforms in production environments.
• Strong SQL and Python skills, with experience developing complex data pipelines and transformation frameworks.
• Proven experience in financial services, ideally within investment management, asset management, private markets or fund administration.
• Strong understanding of financial datasets, including positions, transactions, cash flows, valuations and reference data.
• Expertise in data modelling, data warehousing, master data management and modern cloud-based architectures.
• Experience with Azure cloud technologies and associated data engineering services.
• Strong understanding of APIs, data ingestion, third-party integrations and data reconciliation.
• Experience with Git, CI/CD, infrastructure as code, automated testing and platform monitoring.
• Ability to engage confidently with senior stakeholders and independently own technical architecture and delivery.
• Experience within private credit, direct lending, alternative investments or broader private markets.
• Familiarity with loan administration, portfolio accounting or investment management platforms.
• Exposure to market data providers such as ICE, Bloomberg or equivalent.
• Experience with Power BI, dbt and data catalogue or lineage tools.
• Knowledge of Snowflake Cortex, AI/LLM data integration or advanced analytics use cases.
• Understanding of financial services regulatory requirements, including DORA and GDPR.
• Previous experience building a data platform from the ground up or leading a significant data transformation programme.

Responsibilities

• Lead the design, development and ongoing management of a Snowflake-based enterprise data platform.
• Develop scalable data architectures using modern data modelling principles, including medallion architecture (bronze, silver and gold layers).
• Build and maintain robust ingestion pipelines integrating multiple internal applications, external data providers and third-party administrators.
• Establish data governance, quality, security and lineage frameworks to ensure the integrity and reliability of investment data.
• Implement infrastructure as code, automated deployment processes and CI/CD best practices.
• Lead the integration of portfolio management, fund administration, market data and financial reporting systems.
• Design and maintain security master and reference data models covering financial instruments, issuers and counterparties.
• Develop data reconciliation, validation and exception management processes across multiple data sources.
• Build governed datasets supporting investment positions, exposures, cash flows, liquidity, counterparties and fund accounting.
• Work closely with investment, finance, risk and technology teams to translate business requirements into scalable technical solutions.
• Take end-to-end responsibility for platform performance, reliability, security and cost optimisation.
• Develop data models and semantic layers supporting Power BI, management reporting and investment analytics.
• Support the adoption of AI and automation by making high-quality, governed data accessible to emerging applications and intelligent workflows.
• Establish best practices around data engineering, documentation, monitoring and operational resilience.
• Collaborate with external technology partners while maintaining internal ownership of architecture and technical decisions.

Skills

Snowflake

SQL

Python

Data modeling

Azure cloud

APIs & integration

CI/CD

Stakeholder management

Financial services domain

Tools

dbt

Power BI

Git

Terraform

Snowflake Cortex

Job description

twentyAI in London is seeking an experienced Lead Data Engineer to own a brand-new greenfield enterprise data platform built on Snowflake. You will design, implement and govern data pipelines, models and governance to unify investment, portfolio and financial data for advanced analytics.

You will work closely with technology and business stakeholders to deliver scalable, secure, cost-efficient cloud-based infrastructure, empowering AI-driven insights and executive reporting.