Principal AI Data Engineer - Gen AI & RAG (Hybrid London)

Talent; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

Talent is seeking a Principal AI Data Engineer to design and deliver Generative AI solutions for a major enterprise client in London (Canary Wharf). This role bridges frontier AI experimentation with production-grade data platforms, pipelines, and governance.

You will lead agentic AI workflows, RAG pipelines, and cloud lakehouse architectures, guiding agile squads, setting standards, and implementing MLOps practices for reliable, auditable AI systems.

Qualifications

• 8–12+ years of hands-on experience in data engineering, data science, or production ML/AI environments.
• Proven delivery with LangGraph, AutoGen, CrewAI, smolagents, or MCP integrations.
• Strong background with enterprise Databricks environments (Delta Lake, Unity Catalog, Delta Live Tables/DLT, Spark Structured Streaming, Autoloader).
• Broad exposure to Azure cloud environments or Databricks Mosaic AI ecosystem.
• Advanced Python (modular OOP, Pydantic, design patterns) and strong SQL capabilities.

Responsibilities

• Architect, build, and deploy agentic AI workflows and multi-agent systems for production use.
• Design and tune enterprise-grade Retrieval-Augmented Generation (RAG) pipelines.
• Build scalable data pipelines across lakehouse architectures (Delta Lake, streaming, governance).
• Establish model evaluation, reproducibility checks, tracing, and benchmarking frameworks.
• Set architectural standards, blueprints, and lead agile squads; communicate trade-offs to stakeholders.
• Implement CI/CD, containerisation, and cloud gateway governance in production ML/AI environments.

Skills

Advanced Python

OO Python (design patterns)

SQL

MLOps tooling

Education

Degree or higher in Mathematics/Computer Science/Engineering

Tools

LangGraph

LangChain

AutoGen

Databricks Lakehouse

Azure AI Foundry/OpenAI

Job description

Talent is seeking a Principal AI Data Engineer to design and deliver Generative AI solutions for a major enterprise client in London (Canary Wharf). This role bridges frontier AI experimentation with production-grade data platforms, pipelines, and governance.

You will lead agentic AI workflows, RAG pipelines, and cloud lakehouse architectures, guiding agile squads, setting standards, and implementing MLOps practices for reliable, auditable AI systems.