Senior Data Engineer - Azure Databricks, PySpark & ML Ops

EPAM Systems; Kraków; Hybrid; PLN 240,000 - 360,000; Full time

Benefits offered by this job

Hybrid by design

Remote work within Poland

Relocation opportunities

Stock Purchase Plan

English classes

Health insurance

Job summary

EPAM Systems seeks a Senior/Lead Data Software Engineer to build a scalable ML-ready platform, enhancing portfolio model development with advanced data governance and AI capabilities.

You will migrate from an IaaS Big Data platform to Azure-native Databricks, optimize data workflows, and improve data quality while collaborating with cross-functional teams to deliver compliant, high-value data solutions.

Qualifications

• 3+ years in data engineering or related roles
• Proficiency with Python and Spark
• Experience with Azure Databricks and PySpark
• Experience migrating big data platforms to Azure-native services
• Knowledge of data governance and regulatory compliance

Responsibilities

• Migrate and optimize 500+ data jobs using Azure Databricks
• Manage and process 12 TB of data efficiently
• Tune ML models for Azure environments using Java Spark and Delta tables
• Develop and maintain ETL/ELT pipelines with PySpark
• Collaborate to integrate GenAI capabilities into data workflows
• Monitor data quality and implement improvements for reliability

Skills

Python

Spark

Data engineering

Analytical thinking

Tools

Azure Databricks

PySpark

Delta tables

Docker

Kubernetes (AKS)

Terraform

Job description

EPAM Systems seeks a Senior/Lead Data Software Engineer to build a scalable ML-ready platform, enhancing portfolio model development with advanced data governance and AI capabilities.

You will migrate from an IaaS Big Data platform to Azure-native Databricks, optimize data workflows, and improve data quality while collaborating with cross-functional teams to deliver compliant, high-value data solutions.