Lead Data & AI Engineer

EPAM Systems Inc; Netherlands; Hybrid; EUR 110,000 - 170,000; Full time

Job summary

EPAM Systems Inc. in the Netherlands seeks a Lead Data & AI Engineer to design and deliver advanced data solutions on cloud platforms, with hands-on data engineering and AI integration to scale pipelines, ensure data quality and enable ML features for analytics and intelligent applications.

You will lead data architectures, orchestrate ETL/ELT workflows, collaborate with data scientists, apply GenAI-driven methods, and maintain security, observability and cost efficiency across enterprise

Qualifications

• 5+ years in data engineering with architecture and leadership
• Strong SQL tuning for large-scale data
• Python proficiency for data workflows
• Experience with Spark/PySpark for distributed processing
• Hands-on Databricks and Delta Lake
• Orchestration of pipelines with Azure Data Factory
• Familiarity with Gen AI in data engineering
• CI/CD for data/ML pipelines (GitHub Actions, Azure DevOps)
• English at B2 level or higher
• Nice-to-have: Prompt Engineering & RAG workflows
• Knowledge of data mesh or lakehouse concepts
• dbt and related developer tooling

Responsibilities

• Design and implement robust data architectures using cloud-native technologies
• Build large-scale ETL/ELT workflows to process heterogeneous datasets
• Create data models optimized for AI/ML pipelines and analytics
• Develop streaming and batch pipelines leveraging Databricks and Azure Data Factory
• Operationalize ML solutions with feature stores and model registries
• Collaborate with data scientists to deploy and monitor AI models in production
• Incorporate GenAI-assisted development into data workflows for efficiency
• Ensure data governance, lineage, cataloging and quality across platforms
• Optimize platform performance, compute costs, and observability
• Mentor engineering teams in advanced data and AI engineering

Skills

Data engineering

Leadership

SQL optimization

Python

CI/CD

GitHub Actions

Azure DevOps

Gen AI

MLOps

Cloud platforms

Tools

Databricks

Delta Lake

Azure Data Factory

dbt

MKDocs

Job description

We’re looking for a Lead Data & AI Engineer to join our team in the Netherlands in a hybrid working mode. In this role, you will design and deliver advanced data solutions and AI-assisted capabilities on modern cloud platforms. You’ll build scalable data pipelines, optimize data quality and governance and enable ML feature engineering for analytics and intelligent applications. The role requires hands-on technical expertise in data engineering and AI integration while driving reliability, performance and security for enterprise-scale systems.

Responsibilities

• Design and implement robust data architectures using cloud-native technologies
• Build large-scale ETL/ELT workflows to process heterogeneous datasets
• Create data models optimized for AI/ML pipelines and advanced analytics
• Develop streaming and batch pipelines leveraging tools like Azure Data Factory and Databricks
• Operationalize ML solutions, integrating feature stores, model registries and inference endpoints
• Collaborate with data scientists to deploy and monitor AI models using enterprise MLOps frameworks
• Integrate GenAI-assisted development methods into data workflows for automation and efficiency
• Ensure data governance, lineage, cataloging and quality frameworks across platforms
• Optimize platform performance, manage compute cost efficiency and enable observability for critical workloads
• Contribute to best practices, mentoring engineering teams in advanced data and AI engineering

Requirements

• 5+ years working in data engineering, with proven architecture and leadership experience
• Advanced proficiency in SQL for performance tuning at large scale
• Strong programming skills in Python, with applied experience in data engineering workflows
• Expertise in PySpark for distributed data processing
• Hands-on experience with Databricks, including Delta Lake and performance optimization
• Knowledge of Azure Data Factory for orchestration of pipelines (ETL/ELT)
• Familiarity with AI/ML pipeline development, including integration of models into production
• Demonstrated exposure to Gen AI-assisted development workflows for accelerating data engineering tasks
• Strong knowledge of CI/CD for data and ML pipelines (GitHub Actions, Azure DevOps) and experience with large-scale environments
• English proficiency at B2 level (Upper-Intermediate) or higher
• Nice to have Prompt Engineering knowledge and experience building RAG workflows
• Familiarity with Microsoft Foundry platforms
• Version control and CI/CD experience with GitHub
• Understanding of ETL/ELT optimization patterns beyond Azure stack
• Knowledge of data mesh or lakehouse architectural concepts
• Hands-on exposure to dbt (data build tool), MKDocs and similar developer productivity tooling