Senior Data Engineer - ETL Pipelines, Azure & Spark

Mindtel; Dubai; On-site; AED 446,000 - 670,000; Full time

Job summary

Mindtel is seeking a Data Engineer in Dubai to design, build, and maintain robust ETL pipelines using Data Bricks and Azure Data Factory. You will transform raw data into actionable insights, manage data storage with SQL Server, and collaborate with cross-functional teams to ensure seamless integration of data across platforms.

The role emphasizes scalable data architectures, automation through DevOps practices, and writing efficient Python scripts for data processing and analysis.

Responsibilities

• Design, develop and maintain robust ETL pipelines using Data Bricks and Azure Data Factory.
• Transform raw data into actionable insights to support business decision-making.
• Manage and optimize data storage solutions using SQL Server.
• Collaborate with cross-functional teams and stakeholders to gather requirements and ensure seamless data integration.
• Develop and maintain scalable data architectures and data workflows.
• Automate data operations and enhance deployment processes using DevOps practices.
• Write and optimize Python scripts for efficient data processing and analysis.
• Monitor, troubleshoot and ensure accuracy and availability of data pipelines.
• Stay updated with industry trends and adopt new data engineering technologies and best practices.
• Ensure high data quality across platforms and facilitate ongoing improvements in data engineering processes.

Skills

Apache Hive

S3

Hadoop

Redshift

Spark

AWS

Apache Pig

NoSQL

Big Data

Data Warehouse

Kafka

Scala

Tools

Databricks

Azure Data Factory

SQL Server

Python

Job description

Mindtel is seeking a Data Engineer in Dubai to design, build, and maintain robust ETL pipelines using Data Bricks and Azure Data Factory. You will transform raw data into actionable insights, manage data storage with SQL Server, and collaborate with cross-functional teams to ensure seamless integration of data across platforms.

The role emphasizes scalable data architectures, automation through DevOps practices, and writing efficient Python scripts for data processing and analysis.