Principal AI & Data Engineer

EPAM Systems; Greater London; Hybrid; GBP 120,000 - 180,000; Full time

Benefits offered by this job

EPAM ESPP

Private medical insurance

Dental care

Life assurance

Pension plan

Lunch on Wednesdays in-office

In-office massages

Job summary

EPAM Systems in London is seeking a Principal AI & Data Engineer to design, build and productionize enterprise-grade AI and GenAI solutions on Azure and Databricks.

You will lead the development of scalable AI applications, intelligent agents and data-driven systems powered by LLMs, with hands-on experience across AI data engineering, cloud tech and GenAI adoption to drive innovation, reliability and scalability.

Qualifications

• Bachelor’s or Master’s degree in Statistics, Mathematics, Econometrics, Computer Science or a related field
• 8+ years of experience delivering data engineering, ML or AI solutions in enterprise environments
• Strong AI Data Engineering expertise with production-grade AI/GenAI systems
• Hands-on experience with Azure AI Foundry, Azure cloud services and Databricks AI technologies
• Deep understanding of LLMs and Retrieval-Augmented Generation (RAG) solutions
• Strong experience with MLflow, GenAI evaluation frameworks and model benchmarking methodologies
• Advanced programming in Python and SQL; solid data modeling and ETL/ELT skills
• Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processing

Responsibilities

• Develop and evaluate AI, GenAI and Agentic AI solutions using enterprise AI platforms and frameworks
• Build and optimize Retrieval-Augmented Generation (RAG) applications with embeddings, prompts and evaluation
• Design, develop and deploy AI agents using LangChain, LangGraph, AutoGen and similar frameworks
• Create scalable AI workflows using Databricks, MLflow, Azure AI Foundry and cloud services
• Build and maintain robust data pipelines using modern data engineering practices and streaming tech
• Deploy AI apps using Azure OpenAI, Azure AI Foundry, Databricks AI tools and Docker
• Design evaluation frameworks to measure model quality, reliability and reproducibility
• Implement CI/CD pipelines, tests, code quality controls and GitHub processes
• Collaborate with stakeholders to translate business needs into scalable AI solutions
• Contribute to architecture decisions, roadmaps and backlog from prototype to production

Skills

AI Data Engineering

Python

SQL

ETL/ELT

Delta Lake

Spark Structured Streaming

CI/CD

Education

Bachelor’s or Master’s degree in Statistics, Mathematics, Econometrics, Computer Science or related field

Tools

Databricks

Azure AI Foundry

Azure OpenAI

LangChain

LangGraph

AutoGen

GitHub

Job description

We are looking for a Principal AI & Data Engineer to join our team in London, UK in a hybrid working mode. In this role, you will design, build and productionize enterprise-grade AI and GenAI solutions on Azure and Databricks platforms. You will develop scalable AI applications, intelligent agents and data-driven solutions powered by large language models and modern AI frameworks. The role requires hands-on expertise in AI data engineering, cloud technologies and GenAI adoption while driving innovation, reliability and scalability across enterprise environments.ResponsibilitiesDevelop and evaluate AI, GenAI and Agentic AI solutions using enterprise AI platforms and frameworksBuild and optimize Retrieval-Augmented Generation (RAG) applications, including embedding strategies, prompt engineering and evaluation processesDesign, develop and deploy AI agents using frameworks such as LangChain, LangGraph, AutoGen and similar technologiesCreate scalable AI workflows using Databricks, MLflow, Azure AI Foundry and cloud-native servicesBuild and maintain robust data pipelines using modern data engineering practices and streaming technologiesDeploy AI applications and services using Azure OpenAI, Azure AI Foundry, Databricks AI tools and DockerDesign and execute evaluation frameworks to measure model quality, accuracy, reliability and reproducibilityImplement CI/CD pipelines, testing frameworks, code quality controls and GitHub-based development processesCollaborate with stakeholders to translate business requirements into scalable AI solutionsContribute to architecture decisions, technical roadmaps and backlog definition while supporting delivery from prototype to productionRequirementsBachelor’s or Master’s degree in Statistics, Mathematics, Econometrics, Computer Science or a related field8+ years of experience delivering data engineering, machine learning or AI solutions in enterprise environmentsStrong AI Data Engineering expertise with experience building production-grade AI and GenAI systemsHands-on experience with Azure AI Foundry, Azure cloud services and Databricks AI technologiesDeep understanding of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) solutionsStrong experience with MLflow, GenAI evaluation frameworks and model benchmarking methodologiesAdvanced programming skills in Python and SQL with strong knowledge of data modeling and ETL/ELT processesSolid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processingExperience with Azure Data Factory (ADF), Qlik Replicate and modern data architecture practicesExcellent communication, stakeholder management, problem-solving skills and commitment to high-quality deliveryNice to haveExperience in commodity trading and supply operations, ideally within the energy sectorExperience with Microsoft Copilot Studio, Databricks Mosaic AI, Agent Bricks or GenieExperience with LangGraph, AutoGen or other agent orchestration frameworksFamiliarity with enterprise AI governance, security and compliance practicesExperience leading technical teams, defining technical strategy and mentoring engineersWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness coverPrivate medical insurance and dental careEmployee Assistance ProgramCompetitive group pension planCyclescheme, Techscheme and season ticket loansVarious perks such as free Wednesday lunch in-office, on-site massages and regular social eventsLearning and development opportunities including in-house training and coaching, professional certifications, and coursesIf otherwise eligible, participation in the discretionary annual bonus programIf otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program