Senior Data Engineer: Azure Databricks & ML Platform

EPAM Systems; Town of Poland (NY); Vor Ort; USD 120.000 - 150.000; Vollzeit

Zusammenfassung

EPAM Systems is looking for a Senior/Lead Data Software Engineer to work on a scalable, ML-ready platform focused on enhancing portfolio model development. The role involves optimizing data workflows and ensuring compliance with data governance frameworks.

The ideal candidate will have proficiency in Python and Spark, with a strong background in Azure Databricks and data engineering practices. You will be responsible for migrating data jobs, developing ETL/ELT pipelines, and collaborating with cross-functional teams.

Qualifikationen

• At least 3 years of experience in data engineering roles.
• Strong experience with Azure Databricks and PySpark.
• Ability to manage large data volumes with high efficiency.

Aufgaben

• Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques.
• Develop and maintain ETL/ELT pipelines using PySpark and related technologies.
• Support data governance initiatives to comply with regulatory standards.

Kenntnisse

Python

Spark

Azure Databricks

ETL/ELT solutions

Data governance

Terraform

GitLab CI/CD

Docker

Kubernetes (AKS)

Jobbeschreibung

EPAM Systems is looking for a Senior/Lead Data Software Engineer to work on a scalable, ML-ready platform focused on enhancing portfolio model development. The role involves optimizing data workflows and ensuring compliance with data governance frameworks.

The ideal candidate will have proficiency in Python and Spark, with a strong background in Azure Databricks and data engineering practices. You will be responsible for migrating data jobs, developing ETL/ELT pipelines, and collaborating with cross-functional teams.