AI Data Engineer

Talent; Greater London; Ibrido; GBP 90.000 - 130.000; Tempo pieno

Descrizione del lavoro

Talent is seeking a Principal AI Data Engineer to design and deliver Generative AI solutions for a major enterprise client in London (Canary Wharf). This role bridges frontier AI experimentation with production-grade data platforms, pipelines, and governance.

You will lead agentic AI workflows, RAG pipelines, and cloud lakehouse architectures, guiding agile squads, setting standards, and implementing MLOps practices for reliable, auditable AI systems.

Competenze

• 8–12+ years of hands-on experience in data engineering, data science, or production ML/AI environments.
• Proven delivery with LangGraph, AutoGen, CrewAI, smolagents, or MCP integrations.
• Strong background with enterprise Databricks environments (Delta Lake, Unity Catalog, Delta Live Tables/DLT, Spark Structured Streaming, Autoloader).
• Broad exposure to Azure cloud environments or Databricks Mosaic AI ecosystem.
• Advanced Python (modular OOP, Pydantic, design patterns) and strong SQL capabilities.

Mansioni

• Architect, build, and deploy agentic AI workflows and multi-agent systems for production use.
• Design and tune enterprise-grade Retrieval-Augmented Generation (RAG) pipelines.
• Build scalable data pipelines across lakehouse architectures (Delta Lake, streaming, governance).
• Establish model evaluation, reproducibility checks, tracing, and benchmarking frameworks.
• Set architectural standards, blueprints, and lead agile squads; communicate trade-offs to stakeholders.
• Implement CI/CD, containerisation, and cloud gateway governance in production ML/AI environments.

Conoscenze

Advanced Python

OO Python (design patterns)

SQL

MLOps tooling

Formazione

Degree or higher in Mathematics/Computer Science/Engineering

Strumenti

LangGraph

LangChain

AutoGen

Databricks Lakehouse

Azure AI Foundry/OpenAI

Descrizione del lavoro

Contract Duration: 6 Months (with view to extend)

Inside IR35

Location: London (Canary Wharf) — Hybrid (3 days onsite / 2 days remote)

A global technology consultancy is seeking a Principal AI Data Engineer to spearhead the design, architecture, and production delivery of advanced Generative and Agentic AI solutions for a major enterprise client.

In this role, you will bridge the gap between rapid frontier AI experimentation and robust, large-scale enterprise data engineering. Working closely with commercial stakeholders, cross-functional squads, and technical leadership, you will translate high-value operational use cases into reliable, production-grade AI pipelines, intelligent agentic workflows, and governed data platforms.

Key Responsibilities

• Agentic AI & Prototype Delivery: Architect, build, and deploy multi-agent workflows and task-driven AI systems using frameworks such as LangGraph, LangChain, or AutoGen, turning early proof-of-concepts into resilient production applications.
• Retrieval-Augmented Generation (RAG): Design and tune enterprise-grade RAG pipelines, including chunking strategy, dense/hybrid retrieval, reranking, and semantic search.
• Data Platform Engineering: Build scalable, streaming, and batch data processing pipelines across modern cloud lakehouse architectures (Delta Lake, streaming architectures, and unified governance layers).
• Systematic Evaluation & Benchmarking: Establish continuous model evaluation, reproducibility checks, tracing, and benchmarking frameworks (MLflow, LLM-as-a-judge, golden test suites) to ensure performance, safety, and auditability.
• Architecture & Backlog Leadership: Set technical standards, define architectural blueprints, shape the product backlog, and lead agile delivery squads while communicating technical trade-offs to commercial business stakeholders.
• Production MLOps & Governance: Implement modern software engineering and CI/CD practices (OOP in Python, Pydantic, containerisation, automated testing, and cloud gateway governance).

Key Technical Stack & Experience

• Experience: 8–12+ years of hands-on experience in data engineering, data science, or production ML/AI environments.
• Agentic Frameworks: Proven delivery with LangGraph, AutoGen, CrewAI, smolagents, or Model Context Protocol (MCP) integrations.
• Lakehouse & Data Engineering: Strong background with enterprise Databricks environments (Delta Lake, Unity Catalog, Delta Live Tables/DLT, Spark Structured Streaming, or Autoloader).
• Cloud & Model Ecosystem: Broad exposure to Azure cloud environments (Azure AI Foundry, Azure OpenAI) or Databricks Mosaic AI ecosystem.
• Code Craftsmanship: Advanced Python (modular OOP, Pydantic, design patterns) and strong SQL capabilities.
• Education: Degree or advanced degree (BSc / MSc) in Mathematics, Statistics, Computer Science, Engineering, Econometrics, or related quantitative discipline.