Lead Data & AI Engineer

EPAM Systems, Inc.; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

EPAM Employee Stock Purchase Plan (ES

Private medical insurance

Dental care

Employee Assistance Program

Job summary

EPAM Systems, Inc. is seeking a Lead Data & AI Engineer in the UK to design and deliver advanced data solutions on cloud platforms. You'll build scalable data pipelines, ensure data quality, and enable AI feature engineering for analytics and intelligent applications.

Role requires hands-on expertise in data engineering, AI integration, governance, and enterprise-scale security. Hybrid UK model with multi-cloud readiness and strong collaboration with data scientists.

Qualifications

• Minimum 8+ years in data engineering with leadership experience.
• Strong SQL performance tuning for large-scale systems.
• Python programming in data engineering workflows.
• Distributed processing with PySpark.
• Azure Data Factory orchestration and Databricks usage.
• Experience in AI/ML integration and MLOps workflows.
• CI/CD for data and ML pipelines (GitHub Actions, Azure DevOps).
• GenAI-enabled development workflows and data governance.

Responsibilities

• Design robust data architectures on cloud platforms.
• Build large-scale ETL/ELT pipelines for heterogeneous data.
• Create data models for AI/ML pipelines and analytics.
• Develop streaming and batch pipelines with Databricks.
• Operationalize ML solutions with feature stores and registries.
• Collaborate with data scientists to deploy models and monitor them.
• Integrate GenAI into data workflows for automation.
• Ensure data governance, lineage, cataloging, and quality.

Skills

SQL Tuning

Python

PySpark

Delta Lake

Azure Data Factory

GenAI

MLOps

CI/CD

GitHub Actions

Tools

Databricks

dbt

Job description

We’re looking for a Lead Data & AI Engineer to join our team in the UK in a hybrid working mode. In this role, you will design and deliver advanced data solutions and AI-assisted capabilities on modern cloud platforms. You’ll build scalable data pipelines, optimize data quality and governance and enable ML feature engineering for analytics and intelligent applications. The role requires hands-on technical expertise in data engineering and AI integration while driving reliability, performance and security for enterprise-scale systems.ResponsibilitiesDesign and implement robust data architectures using cloud-native technologiesBuild large-scale ETL/ELT workflows to process heterogeneous datasetsCreate data models optimized for AI/ML pipelines and advanced analyticsDevelop streaming and batch pipelines leveraging tools like Azure Data Factory and DatabricksOperationalize ML solutions, integrating feature stores, model registries and inference endpointsCollaborate with data scientists to deploy and monitor AI models using enterprise MLOps frameworksIntegrate GenAI-assisted development methods into data workflows for automation and efficiencyEnsure data governance, lineage, cataloging and quality frameworks across platformsOptimize platform performance, manage compute cost efficiency and enable observability for critical workloadsContribute to best practices, mentoring engineering teams in advanced data and AI engineeringRequirementsMinimum 8+ years working in data engineering, with proven architecture and leadership experienceAdvanced proficiency in SQL for performance tuning at large scaleStrong programming skills in Python, with applied experience in data engineering workflowsExpertise in PySpark for distributed data processingHands-on experience with Databricks, including Delta Lake and performance optimizationKnowledge of Azure Data Factory for orchestration of pipelines (ETL/ELT)Familiarity with AI/ML pipeline development, including integration of models into productionDemonstrated exposure to Gen AI-assisted development workflows for accelerating data engineering tasksStrong understanding of CI/CD processes for data and ML pipelines such as GitHub Actions, Azure DevOpsAbility to manage enterprise-scale solutions across global environments with compliance in mindNice to havePrompt Engineering knowledge and experience building RAG workflowsFamiliarity with Microsoft Foundry platformsVersion control and CI/CD experience with GitHubUnderstanding of ETL/ELT optimization patterns beyond Azure stackKnowledge of data mesh or lakehouse architectural conceptsHands-on exposure to dbt (data build tool), MKDocs and similar developer productivity toolingWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness coverPrivate medical insurance and dental careEmployee Assistance ProgramCompetitive group pension planCyclescheme, Techscheme and season ticket loansVarious perks such as free Wednesday lunch in-office, on-site massages and regular social eventsLearning and development opportunities including in-house training and coaching, professional certifications, and coursesIf otherwise eligible, participation in the discretionary annual bonus programIf otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program