Senior Data Engineer | Databricks Pipelines & Migration

EPAM Systems; Abu Dhabi; On-site; AED 350,000 - 550,000; Full time

Benefits offered by this job

End of service gratuity

Private healthcare

Life insurance

Employee assistance program

Wellness program

Annual air travel allowance for expats

Salary reviews

Global travel medical and accident保险

Learning and development including in‑

Job summary

EPAM Systems in Abu Dhabi is seeking a Senior Data Engineer to join onsite. You will lead the migration of a large-scale data platform and build production-ready pipelines using Databricks, PySpark and SQL in a dynamic delivery environment.

You will design validation, optimize performance, and mentor junior engineers while contributing to CI/CD and Agile workflows within a collaborative engineering team.

Qualifications

• 7+ years of data engineering experience with hands-on delivery.
• Strong proficiency with Databricks, PySpark, SQL and Delta Lake.
• Expertise in data migration or re-platforming projects.
• Bachelor’s degree in computer science, information technology or related field.
• Experience with CI/CD and Agile delivery.

Responsibilities

• Build, optimize and productionize data pipelines on Databricks using PySpark and SQL.
• Migrate transformation logic from a legacy platform to Databricks.
• Design and perform data validation and reconciliation to ensure accuracy across platforms.
• Tune performance through partitioning, Delta optimization, cluster configuration and ingestion strategies.
• Contribute to medallion-layer design and implement data quality frameworks.
• Review code and mentor junior engineers to promote best practices.
• Work within Git-based CI/CD setups and follow Agile delivery processes.

Skills

Databricks

PySpark

SQL

Delta Lake

Data pipelines

Spark tuning

CI/CD

Mentoring

Agile

Education

Bachelor's degree in Computer Science / IT

Tools

Git

Job description

EPAM Systems in Abu Dhabi is seeking a Senior Data Engineer to join onsite. You will lead the migration of a large-scale data platform and build production-ready pipelines using Databricks, PySpark and SQL in a dynamic delivery environment.

You will design validation, optimize performance, and mentor junior engineers while contributing to CI/CD and Agile workflows within a collaborative engineering team.