Senior Data Engineer

EPAM Systems; United Arab Emirates; On-site; AED 180,000 - 300,000; Full time

Job summary

EPAM Systems in Abu Dhabi, United Arab Emirates, is seeking a Senior Data Engineer to join an on-site team. You will work on a large-scale data platform migration, building production-ready pipelines using Databricks, PySpark and SQL in a dynamic delivery environment.

The ideal candidate has 7+ years of data engineering experience, strong Databricks/Delta Lake skills, and hands-on delivery in financial services or asset management contexts.

Qualifications

• 7+ years of data engineering experience with recent hands-on delivery.
• Strong proficiency with Databricks, PySpark, SQL and Delta Lake.
• In-depth knowledge of ingestion patterns, Spark performance tuning and partitioning strategies.
• Solid SQL expertise with practical ability to write and validate queries.
• Experience in data migration or re-platforming projects.
• Professional-level English communication skills.
• Bachelor’s degree in computer science, information technology or a related field.
• Databricks Certified Data Engineer Professional certification.
• Familiarity with Palantir Foundry.
• Background in financial services or asset management.
• Experience with Unity Catalog governance.
• Knowledge of large-scale data quality or reconciliation frameworks.

Responsibilities

• Build, optimize and productionize data pipelines on Databricks using PySpark and SQL.
• Re-engineer and migrate transformation logic from a legacy platform to Databricks.
• Design and perform data validation and reconciliation to ensure accuracy across platforms.
• Tune performance through partitioning, Delta optimization, cluster configuration and ingestion strategies.
• Contribute to medallion‑layer design and implement data quality frameworks.
• Review code and mentor junior engineers to promote best practices.
• Work within Git‑based CI/CD setups and follow Agile delivery processes.

Skills

Databricks

PySpark

SQL

Delta Lake

Data integration

Cloud data platforms

Agile delivery

English communication

Education

Bachelor’s degree in computer science / information technology / related field

Databricks Certified Data Engineer Professional

Tools

Palantir Foundry

Unity Catalog governance

Job description

We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode. This role involves working on a large-scale data platform migration using modern data technologies and building production-ready pipelines in a dynamic delivery environment. You will be part of a dedicated engineering team focused on high-impact work that requires strong hands‑on expertise.

Responsibilities

• Build, optimize and productionize data pipelines on Databricks using PySpark and SQL
• Re‑engineer and migrate transformation logic from a legacy platform to Databricks
• Design and perform data validation and reconciliation to ensure accuracy across platforms
• Tune performance through partitioning, Delta optimization, cluster configuration and ingestion strategies
• Contribute to medallion‑layer design and implement data quality frameworks
• Review code and mentor junior engineers to promote best practices
• Work within Git‑based CI/CD setups and follow Agile delivery processes

Requirements

• 7+ years of data engineering experience with recent hands‑on delivery
• Strong proficiency with Databricks platform, PySpark, SQL and Delta Lake
• In‑depth knowledge of ingestion patterns, Spark performance tuning and partitioning strategies
• Solid SQL expertise with practical ability to write and validate queries
• Experience in data migration or re‑platforming projects
• Professional‑level English communication skills
• Bachelor’s degree in computer science, information technology or a related field
• Databricks Certified Data Engineer Professional certification
• Familiarity with Palantir Foundry
• Background in financial services or asset management
• Experience with Unity Catalog governance
• Knowledge of large‑scale data quality or reconciliation frameworks