Senior/Lead Data Software Engineer (Python, Spark, Azure)

KARIER

Wrocław

Hybrid

PLN 180,000 - 260,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible schedule
Remote work within Poland
Work abroad up to 60 days annually
Relocation opportunities
Health insurance
Multisport
Shopping vouchers

Job summary

EPAM is seeking a Senior/Lead Data Software Engineer to join a team building a scalable, ML-ready platform for portfolio model development with advanced data governance and AI capabilities.

You will migrate from an IaaS Big Data platform to Azure-native Databricks and optimize data workflows while ensuring data quality and regulatory compliance across multi-national setups.

Qualifications

  • Proficiency in Python and Spark with at least 3 years in data engineering roles.
  • Strong experience with Azure Databricks and PySpark.
  • Proven expertise in designing and implementing ETL/ELT solutions.
  • Experience migrating big data platforms to Azure-native services.
  • Proficiency with Delta tables for model tuning.
  • Knowledge of data governance and regulatory compliance frameworks.
  • Familiarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation.
  • Ability to manage large data volumes with high efficiency.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration abilities.
  • English proficiency at B2 level or higher.

Responsibilities

  • Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques.
  • Manage and process 12 TB of data efficiently across platforms.
  • Tune machine learning models for Azure environments using Java Spark and Delta tables.
  • Update and maintain libraries to address security vulnerabilities.
  • Develop and maintain ETL/ELT pipelines using PySpark and related technologies.
  • Collaborate with cross-functional teams to integrate GenAI capabilities into data workflows.
  • Monitor data quality and implement improvements to ensure accuracy and reliability.
  • Automate deployment and operational tasks using Terraform and GitLab CI/CD.
  • Support data governance initiatives to comply with regulatory standards.
  • Troubleshoot and resolve performance issues in data processing systems.
  • Document system processes and provide technical guidance to junior engineers.
  • Implement best practices for code quality and data security.
  • Participate in code reviews and knowledge sharing sessions.
  • Optimize costs associated with data storage and processing.

Skills

Python
Spark
English communication

Tools

Azure Databricks
PySpark
Delta tables
Docker
Kubernetes (AKS)
Terraform
GitLab CI/CD

Job description

We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready platform that enhances portfolio model development and deployment with advanced data governance and AI capabilities.

You will play a key role in migrating from an IaaS Big Data platform to Azure-native Databricks, optimizing data workflows and improving data quality. Join us to contribute to innovative solutions that boost client services and regulatory compliance.

Responsibilities
  • Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques
  • Manage and process 12 TB of data efficiently across platforms
  • Tune machine learning models for Azure environments using Java Spark and Delta tables
  • Update and maintain libraries to address security vulnerabilities
  • Develop and maintain ETL/ELT pipelines using PySpark and related technologies
  • Collaborate with cross-functional teams to integrate GenAI capabilities into data workflows
  • Monitor data quality and implement improvements to ensure accuracy and reliability
  • Automate deployment and operational tasks using Terraform and GitLab CI/CD
  • Support data governance initiatives to comply with regulatory standards
  • Troubleshoot and resolve performance issues in data processing systems
  • Document system processes and provide technical guidance to junior engineers
  • Implement best practices for code quality and data security
  • Participate in code reviews and knowledge sharing sessions
  • Optimize costs associated with data storage and processing
Requirements
  • Proficiency in Python and Spark with at least 3 years in data engineering roles
  • Strong experience with Azure Databricks and PySpark
  • Proven expertise in designing and implementing ETL/ELT solutions
  • Experience migrating big data platforms to Azure-native services
  • Proficiency with Delta tables for model tuning
  • Knowledge of data governance and regulatory compliance frameworks
  • Familiarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation
  • Ability to manage large data volumes with high efficiency
  • Excellent problem-solving and analytical skills
  • Strong communication and collaboration abilities
  • English proficiency at B2 level or higher
We offer
  • We gather like-minded people:
    • Engineering community of industry professionals
    • Friendly team and enjoyable working environment
    • Flexible schedule and opportunity to work remotely within Poland
    • Chance to work abroad for up to 60 days annually
    • Business-driven relocation opportunities
  • We provide growth opportunities:
    • Outstanding career roadmap
    • Leadership development, career advising, soft skills, and well-being programs
    • Certification (GCP, Azure, AWS)
    • Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru
    • English classes
  • We cover it all:
    • Stable income (Employment Contract or B2B)
    • Participation in the Employee Stock Purchase Plan
    • Benefits package (health insurance, multisport, shopping vouchers)
    • Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more
    • Referral bonuses
    • Corporate, social and well-being events
  • Please, note:
    • The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview.
    • We will reach out to selected candidates exclusively.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior/Lead Data Software Engineer (Python, Spark, Azure)
Senior/Lead Data Software Engineer (Python, Spark, Azure)

EPAM Systems Inc • Wrocław

Hybrid
PLN 180,000 - 280,000
Hybrid by design
Remote within Poland
relocation opportunities
+3
Senior/Lead Data Software Engineer (Python, Spark, Azure)
Senior/Lead Data Software Engineer (Python, Spark, Azure)

EPAM Systems Inc • Kraków

Hybrid
PLN 240,000 - 360,000
Hybrid by design
Remote work within Poland
Relocation opportunities
+3
Senior Data Engineer (Java/Scala/Python, Spark, AWS)
Senior Data Engineer (Java/Scala/Python, Spark, AWS)

EPAM Systems Inc • Warszawa

Hybrid
PLN 180,000 - 320,000
Hybrid work model
Remote within Poland
Relocation opportunities
+5
Senior Platform Engineer (Azure Infrastructure)
Senior Platform Engineer (Azure Infrastructure)

EPAM Systems Inc • Polska

Hybrid
PLN 180,000 - 260,000
Health insurance
Relocation opportunities
English classes
+1
Java Developer - Big Data
Java Developer - Big Data

EPAM Systems Inc • Kraków

Hybrid
PLN 150,000 - 210,000
Hybrid work mode
Remote within Poland
Career development
+3
Senior Data Software Engineer
Senior Data Software Engineer

EPAM Systems Inc • Poland

Hybrid
PLN 240,000 - 360,000
Hybrid work design
Remote within Poland
Opportunities to work abroad for up to
Senior AI Data Software Engineer
Senior AI Data Software Engineer

EPAM Systems Inc • Poland

Hybrid
PLN 180,000 - 240,000
Hybrid by design
Remote working within Poland
Relocation opportunities
+3
Lead AI Data Software Engineer
Lead AI Data Software Engineer

EPAM Systems Inc • Poland

Hybrid
PLN 220,000 - 320,000
Hybrid by design
Remote work within Poland
Career development programs
+2
Data Software Engineer (Upskilling position for Python Developers)
Data Software Engineer (Upskilling position for Python Developers)

EPAM Systems Inc • Polska

Hybrid
PLN 180,000 - 260,000
Hybrid work model
Remote work within Poland
Work abroad up to 60 days annually
+4
Senior Data Software Engineer (Databricks upskilling program)
Senior Data Software Engineer (Databricks upskilling program)

EPAM Systems Inc • Polska

Hybrid
PLN 180,000 - 240,000
Hybrid by design
Remote within Poland
Relocation opportunities
+1
Top locations