Senior GenAI Data Engineer — Scale AI Solutions

Eaglecliff Recruitment; Greater London; On-site; GBP 90,000 - 140,000; Full time

Job summary

Eaglecliff Recruitment is seeking a Senior AI Data Engineer to help build and scale GenAI and agentic AI solutions for a Global Energy Company. You will design prototypes, evaluate models and contribute to scalable AI platforms with Azure, Databricks and modern tooling.

You'll write clean Python, work on data pipelines, and implement robust testing and deployment practices. The role requires 4-8 years in data science/AI, strong knowledge of LLMs, Databricks, streaming tech, and experience with

Qualifications

• 4-8 years of experience delivering data science/AI projects.
• Good working knowledge of LLMs (GPT, Llama, Claude, Mistral) and their reasoning capabilities.
• Experience with Databricks (DLT, Delta Lake, Unity Catalog governance).
• Understanding of streaming technologies (Spark Structured Streaming, Autoloader).
• Proficiency in Python, SQL, or Scala.
• Experience with ETL/ELT, data modelling, and data architecture.
• Exposure to ADF and Qlik Replicate.

Responsibilities

• Build and evaluate GenAI/Agentic AI prototypes (Copilot Studio, Azure AI Foundry, Databricks Mosaic AI, Genie, MLflow).
• Design and tune RAG systems.
• Build AI agents using LangChain, LangGraph, AutoGen, and similar frameworks.
• Deploy models via Azure OpenAI, Azure AI Foundry, and Docker.
• Write clean, modular Python (OOP, Pydantic, PyTorch) with solid DevOps/CI-CD practices.
• Manage large-scale text and structured datasets on Databricks.
• Support testing and evaluation frameworks for GenAI systems (benchmarking, reproducibility).

Skills

GenAI prototyping

LLMs knowledge

Databricks

Streaming technologies

Python / SQL / Scala

ETL/ELT & data modelling

Azure / Docker deployment

Tools

Databricks

Azure OpenAI

Docker

Qlik Replicate

ADF

LangChain

Job description

Eaglecliff Recruitment is seeking a Senior AI Data Engineer to help build and scale GenAI and agentic AI solutions for a Global Energy Company. You will design prototypes, evaluate models and contribute to scalable AI platforms with Azure, Databricks and modern tooling.

You'll write clean Python, work on data pipelines, and implement robust testing and deployment practices. The role requires 4-8 years in data science/AI, strong knowledge of LLMs, Databricks, streaming tech, and experience with