Lead Data & AI Engineer

EPAM Systems, Inc.; Amsterdam; Hybrid; EUR 90,000 - 140,000; Full time

Benefits offered by this job

26 paid holiday days

Pension plan scheme

Disability insurance

Long-term disability insurance

EPAM ESPP

Commuting costs reimbursement

Laptop + corporate SIM + device

Bike lease

Employee Assistance Program

HR development & training programs

Job summary

EPAM Systems, Inc. in Amsterdam is seeking a Lead Data & AI Engineer to design and deliver advanced data solutions on cloud platforms in a hybrid setup. You will build scalable pipelines, govern data quality, and enable ML feature engineering for analytics and intelligent applications.

You will lead data engineering efforts, partner with data scientists, implement robust MLOps practices, and drive performance, security and cost efficiency across enterprise-scale systems.

Qualifications

• 5+ years in data engineering with architecture and leadership experience.
• Strong SQL performance tuning for large-scale datasets.
• Python programming for data engineering workflows.
• Expertise in PySpark for distributed processing.
• Hands-on Databricks, Delta Lake and optimization.
• Knowledge of Azure Data Factory for pipeline orchestration.
• Experience with AI/ML pipelines and production integration.
• Gen AI development workflows to accelerate tasks.
• CI/CD for data/ML pipelines (GitHub Actions, Azure DevOps).
• English proficiency at upper-intermediate or higher.

Responsibilities

• Design and implement robust data architectures using cloud-native tech.
• Build large-scale ETL/ELT workflows for heterogeneous datasets.
• Create data models optimized for AI/ML pipelines and analytics.
• Develop streaming and batch pipelines with Azure Data Factory and Databricks.
• Operationalize ML solutions with feature stores and model registries.
• Collaborate with data scientists to deploy and monitor AI models in MLOps.
• Integrate GenAI-assisted methods into data workflows for automation.
• Ensure data governance, lineage, cataloging and quality across platforms.
• Optimize performance, control compute costs and enable observability.
• Mentor engineering teams in advanced data and AI engineering.

Skills

SQL

Python

PySpark

Databricks

Delta Lake

Azure Data Factory

GitHub Actions

Azure DevOps

MLOps

GenAI

Tools

Databricks

Delta Lake

dbt

MKDocs

GitHub

Azure Data Factory

Job description

We’re looking for a Lead Data & AI Engineer to join our team in the Netherlands in a hybrid working mode. In this role, you will design and deliver advanced data solutions and AI-assisted capabilities on modern cloud platforms. You’ll build scalable data pipelines, optimize data quality and governance and enable ML feature engineering for analytics and intelligent applications. The role requires hands-on technical expertise in data engineering and AI integration while driving reliability, performance and security for enterprise-scale systems.ResponsibilitiesDesign and implement robust data architectures using cloud-native technologiesBuild large-scale ETL/ELT workflows to process heterogeneous datasetsCreate data models optimized for AI/ML pipelines and advanced analyticsDevelop streaming and batch pipelines leveraging tools like Azure Data Factory and DatabricksOperationalize ML solutions, integrating feature stores, model registries and inference endpointsCollaborate with data scientists to deploy and monitor AI models using enterprise MLOps frameworksIntegrate GenAI-assisted development methods into data workflows for automation and efficiencyEnsure data governance, lineage, cataloging and quality frameworks across platformsOptimize platform performance, manage compute cost efficiency and enable observability for critical workloadsContribute to best practices, mentoring engineering teams in advanced data and AI engineeringRequirements5+ years working in data engineering, with proven architecture and leadership experienceAdvanced proficiency in SQL for performance tuning at large scaleStrong programming skills in Python, with applied experience in data engineering workflowsExpertise in PySpark for distributed data processingHands-on experience with Databricks, including Delta Lake and performance optimizationKnowledge of Azure Data Factory for orchestration of pipelines (ETL/ELT)Familiarity with AI/ML pipeline development, including integration of models into productionDemonstrated exposure to Gen AI-assisted development workflows for accelerating data engineering tasksStrong knowledge of CI/CD for data and ML pipelines (GitHub Actions, Azure DevOps) and experience with large-scale environmentsEnglish proficiency at B2 level (Upper-Intermediate) or higherNice to havePrompt Engineering knowledge and experience building RAG workflowsFamiliarity with Microsoft Foundry platformsVersion control and CI/CD experience with GitHubUnderstanding of ETL/ELT optimization patterns beyond Azure stackKnowledge of data mesh or lakehouse architectural conceptsHands-on exposure to dbt (data build tool), MKDocs and similar developer productivity toolingWe offer26 paid holiday daysPension plan schemeDisability insurance (WGA Shortfall insurance)Long-term disability insurance (WIA Top up insurance)EPAM Employee Stock Purchase Plan (ESPP)Commuting to work - costs reimbursementLaptop + corporate simcard + corporate mobile device (subject to certain eligibility requirements)Bike leaseEmployee Assistance ProgramCorporate Programs including Employee Referral Program with rewardsLearning and development opportunities including in-house training and coaching, professional certifications, and courses