AI Data Engineer

ARC IT Recruitment; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

ARC IT Recruitment is helping a major financial services group recruit an AI Data Engineer to join their central Data Platform team in London.

The role blends hands-on data engineering with AI/ML capabilities to support analytics, generative AI and agentic AI under strong governance. You will design reusable platform capabilities, collaborate across engineering and AI teams, and help set data product standards.

Qualifications

• 5+ years in cloud-based data engineering delivering enterprise data solutions.
• Proficient Python and SQL with hands-on DBT experience.
• Deep ETL/ELT, data modelling, warehousing, lakehouse and data lake architectures.
• Experience with Apache Spark and large-scale data processing.
• Hands-on experience with Microsoft Fabric; familiarity with Power BI and semantic models.
• Good grasp of machine learning concepts and the data requirements behind them.
• Understanding of generative AI and agentic AI patterns.
• A track record of setting standards and influencing technical direction across engineering, architecture and stakeholder communities.
• Experience in large-scale, ideally regulated, enterprise environments.

Responsibilities

• Design, build and maintain reusable platform capabilities across the business.
• Develop scalable, resilient data solutions using modern cloud data engineering practices.
• Act as primary SME for AI-ready data and platform AI capabilities.
• Define and maintain standards for data products supporting AI, ML, analytics and automation.
• Advise delivery teams on architecture, governance and engineering considerations for AI-enabled solutions.
• Build reusable design patterns to accelerate delivery while maintaining consistency and control.
• Track developments across Microsoft Fabric, ML, generative AI and agentic AI.
• Work with platform engineers, federated data teams, AI practitioners and architects org-wide.

Skills

Python

SQL

DBT

ETL/ELT

Spark

Microsoft Fabric

Power BI

Data Modeling

AI concepts

Generative AI

Tools

Microsoft Fabric

Power BI

Job description

Our client, one of the world's largest independent financial services groups, is looking for an AI Data Engineer to join their central Data Platform team.

This is a rare chance to combine deep data engineering expertise with emerging AI and machine learning capability, within a platform team that sits at the heart of the business.

It is a hands-on role where you will help the data platform evolve to support advanced analytics, machine learning, generative AI and agentic AI, all within a robust governance and control framework.

What You'll Do

• Design, build and maintain reusable platform capabilities across the business
• Develop scalable, resilient data solutions using modern cloud data engineering practices
• Act as primary SME for AI-ready data and platform AI capabilities
• Define and maintain standards for data products supporting AI, ML, analytics and automation
• Advise delivery teams on architecture, governance and engineering considerations for AI-enabled solutions
• Build reusable design patterns to accelerate delivery while maintaining consistency and control
• Track developments across Microsoft Fabric, ML, generative AI and agentic AI
• Work with platform engineers, federated data teams, AI practitioners and architects org-wide

What You'll Bring

• 5+ years in cloud-based data engineering, delivering complex enterprise data solutions
• Strong Python and advanced SQL skills, including hands‑on DBT experience
• Deep understanding of ETL/ELT patterns, data modelling, warehousing, lakehouse and data lake architectures
• Experience with Apache Spark and large-scale data processing
• Strong hands‑on experience with Microsoft Fabric; familiarity with Power BI and semantic models
• Good grasp of machine learning concepts and the data requirements behind them
• Understanding of generative AI and agentic AI patterns
• A track record of setting standards and influencing technical direction across engineering, architecture and stakeholder communities
• Experience in large-scale, ideally regulated, enterprise environments