Senior Data Engineer

Abu Dhabi Department of Economic Development; Abu Dhabi; On-site; AED 320,000 - 520,000; Full time

Job summary

Abu Dhabi Department of Economic Development seeks a senior data engineer to design, build, and operate scalable data pipelines across multi-source environments. You will drive data ingestion and transformation, ensure data quality, and enable analytics and AI/ML initiatives with trusted datasets.

You will mentor junior engineers and collaborate with product and analytics teams. Ideal candidates have 8+ years in data engineering, expertise in SQL, Python, Spark/Scala, and cloud platforms

Qualifications

• 8+ years of experience in data engineering, data platforms, or analytics engineering.
• Proven experience building and operating scalable, production-grade data pipelines.
• Experience in complex, multi-source enterprise environments.

Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines.
• Develop and optimize data ingestion, transformation, and processing workflows.
• Ensure data reliability, performance, and scalability.
• Implement data quality, validation, and monitoring within pipelines.
• Support analytics, BI, and AI/ML use cases with trusted datasets.
• Mentor junior data engineers and review code.

Skills

SQL tuning

Python

Spark/Scala

CI/CD

DevOps

Data modeling

Collaboration with architects

Analytics readiness

Education

Bachelor's degree in CS/Engineering or related field

Cloud or data engineering certifications are a plus

Tools

Data Warehouses

Data Lakes

Lakehouse architectures

AWS

Azure

GCP

Airflow

Azure Data Factory

Dagster

Kafka

Git

Job description

• Design, build, and maintain robust ETL/ELT pipelines.
• Develop and optimize data ingestion, transformation, and processing workflows.
• Ensure data reliability, performance, and scalability.
• Implement data quality, validation, and monitoring within pipelines.
• Support analytics, BI, and AI/ML use cases with trusted datasets.

Technical Skills:

• Strong expertise in SQL and performance tuning.
• Proficiency in Python, Spark, or Scala.
• Hands-on experience with Data Warehouses, Data Lakes, Lakehouse architectures.
• Hand-on experience with Cloud platforms (AWS, Azure, or GCP).
• Experience with orchestration tools (Airflow, Azure Data Factory, Dagster).
• Experience with streaming / real-time data (Kafka, event-driven pipelines).
• Familiarity with CI/CD, Git, and DevOps practices.
• Ability to work closely with Data Architects to implement target architectures.
• Experience integrating data from APIs, databases, files, and SaaS platforms.
• Understanding of data modeling and analytics-ready schemas.

Data Governance & Security:

• Awareness of data governance, data quality, and metadata practices.
• Experience implementing data access controls and security.
• Understanding of privacy and compliance requirements.
• Acts as a technical lead on data engineering initiatives.
• Mentors junior data engineers and reviews code.
• Strong collaboration with product, analytics, and business teams.

Experience:

• 8+ years of experience in data engineering, data platforms, or analytics engineering.
• Proven experience building and operating scalable, production-grade data pipelines.
• Experience working in complex, multi-source enterprise environments.

Education & Certifications:

• Bachelor’s degree in computer science, Engineering, or related field.
• Cloud or data engineering certifications are a plus.