Senior Data Engineer - Databricks Data Migration

EPAM Systems; Abu Dhabi; On-site; AED 420,000 - 660,000; Full time

Benefits offered by this job

End of service gratuity

Private healthcare and life insurance

Employee assistance program

Wellness program

Annual expatriate travel allowance

Learning and certifications

Job summary

EPAM Systems in Abu Dhabi, UAE, is seeking a Senior Data Engineer to join onsite. You will lead production-grade data pipelines on Databricks using PySpark and SQL, migrating transformation logic from a legacy platform, and ensuring data quality across the data platform.

You will optimize performance through partitioning and Delta Lake, design validation processes, review code, mentor junior engineers, and collaborate in CI/CD and Agile delivery.

Qualifications

• 7+ years of data engineering delivery experience.
• Strong proficiency with Databricks, PySpark, SQL and Delta Lake.
• In-depth knowledge of ingestion patterns and Spark tuning.
• Experience in data migration or re-platforming projects.
• Professional-level English communication skills.
• Bachelor’s degree in computer science, information technology or related field.

Responsibilities

• Build, optimize and productionize data pipelines on Databricks using PySpark and SQL.
• Re-engineer and migrate transformation logic from a legacy platform to Databricks.
• Design data validation and reconciliation to ensure accuracy across platforms.
• Tune performance through partitioning, Delta optimization and ingestion strategies.
• Contribute to medallion-layer design and data quality frameworks.
• Review code and mentor junior engineers to promote best practices.
• Work within Git-based CI/CD setups and Agile delivery processes.

Skills

Databricks Platform

PySpark

SQL

Delta Lake

Spark tuning

Data migration

English communication

Education

Bachelor's degree in CS/IT or related field

Tools

Palantir Foundry

Unity Catalog governance

Git CI/CD

Job description

EPAM Systems in Abu Dhabi, UAE, is seeking a Senior Data Engineer to join onsite. You will lead production-grade data pipelines on Databricks using PySpark and SQL, migrating transformation logic from a legacy platform, and ensuring data quality across the data platform.

You will optimize performance through partitioning and Delta Lake, design validation processes, review code, mentor junior engineers, and collaborate in CI/CD and Agile delivery.