DATA ENGINEER

Duncan & Ross; Abu Dhabi; On-site; AED 200,000 - 250,000; Full time

Job summary

A technology company in Abu Dhabi is seeking an experienced data engineer to design and maintain data pipelines. Candidates should have at least 5 years of experience in data engineering, strong Azure and SQL skills, and a collaborative mindset. This role ensures data integrity and optimizes workflows for performance. Excellent problem-solving skills are essential.

Qualifications

• 5 years of experience in a data engineering or data platform role.
• Strong experience with Azure Databricks, Data Lake Storage, PySpark, and Jupyter Notebooks.
• Proficiency in SQL, Python, and distributed data processing frameworks.

Responsibilities

• Design, construct, and maintain data pipelines for data ingestion, transformation, and storage.
• Ensure data integrity, quality, and security throughout the data lifecycle.
• Collaborate with data scientists and analysts to facilitate data accessibility.

Skills

Data engineering

Azure Databricks

Data Lake Storage

SQL

Python

Distributed data processing

ETL/ELT tools

Machine learning workflows

Tools

Azure Data Factory

Azure Synapse

Jupyter Notebooks

Job description

KEY RESPONSIBILITIES:

• Design construct and maintain data pipelines for data ingestion transformation and storage
• Ensure data integrity quality and security throughout the data lifecycle.
• Optimize data processing workflows for performance and scalability.
• Collaborate with data scientists and analysts to facilitate data accessibility.

QUALIFICATIONS:

• 5 years of experience in a data engineering or data platform role.
• Strong experience with Azure Databricks Data Lake Storage PySpark and Jupyter Notebooks.
• Proficiency in SQL Python and distributed data processing frameworks.
• Experience working with Azure Data Factory Azure Synapse or similar ETL/ELT tools.
• Understanding of data governance privacy and security best practices.
• Familiarity with supporting ML/AI workflows including data preparation for machine learning pipelines.
• Excellent problem-solving and communication skills

Vertical

Technology