Senior Data Engineer (Hadoop, PySpark & Apache Airflow) | Go Ahead Consulting | Dubai, UAE

Go Ahead Consulting

Dubai

In loco

AED 350.000 - 550.000

Tempo pieno

2 giorni fa
Candidati tra i primi

Una candidatura completa in un minuto

Curriculum e lettera di presentazione personalizzati, pronti da inviare.

Supera i filtri ATS

Descrizione del lavoro

Go Ahead Consulting in Dubai, UAE, seeks a Senior Data Engineer to own the design, building, scaling and maintenance of high-performance big data pipelines, Hadoop clusters, PySpark transformations, and Apache Airflow workflows integrated with GitHub.

You will lead cross-functional collaboration with data scientists, software engineers and recruitment leadership, ensure data governance and security, and deliver reliable analytics solutions across GCC engagements for an international technology

Competenze

  • 5+ years of professional hands-on experience in big data engineering and distributed systems.
  • Proficiency in Hadoop ecosystem (HDFS, Hive, YARN) and PySpark.
  • Hands-on experience with Apache Airflow for workflow orchestration.
  • GitHub version control and collaborative software development.
  • Legal work authorization and UAE visa status for Dubai operations.

Mansioni

  • Architect and scale robust big data pipelines with Hadoop and PySpark.
  • Develop and optimize Airflow DAGs for workflows.
  • Manage CI/CD integrations via GitHub and code reviews.
  • Process large datasets for analytics and ML readiness.
  • Ensure data security, governance, and documentation.
  • Collaborate with data science, software engineering, and product teams.

Conoscenze

Big data engineering
Hadoop
PySpark
Airflow
GitHub
Data warehousing
SQL tuning
Stakeholder comms

Formazione

Bachelor's/Master's in CS/IT

Strumenti

HDFS
Hive
YARN
Docker
Kubernetes
Snowflake
BigQuery
Redshift
GitHub

Descrizione del lavoro

Position Summary

Driving advanced big data engineering, distributed data pipelines, Hadoop architectures, PySpark analytics, and Apache Airflow orchestration in Dubai, United Arab Emirates, Go Ahead Consulting is actively seeking an experienced Senior Data Engineer for a permanent full-time engagement. In this vital data platform and engineering role, you will take full operational, tactical, and technical ownership of designing, building, scaling, and maintaining high-performance big data processing pipelines, Hadoop clusters, PySpark transformations, and automated Airflow DAG workflows integrated with GitHub version control. Bringing deep technical proficiency in distributed computing, data warehousing, and modern data engineering frameworks, you will empower an elite international technology consulting and analytics corporation to achieve uncompromised data reliability, lightning-fast query execution, and superior business intelligence delivery. This prestigious career opportunity offers competitive remuneration, enterprise client exposure in the GCC region, and professional growth within a leading global technology recruitment and consulting corporation.

Detailed Job Description

As a Senior Data Engineer supporting Go Ahead Consulting on a professional basis for client operations in Dubai, UAE, you will take full operational, tactical, and technical ownership of big data architecture, pipeline development, workflow orchestration, and cross-functional engineering collaboration. Your day-to-day responsibilities include designing, developing, optimizing, and maintaining scalable data pipelines utilizing Hadoop, PySpark, and Apache Airflow, managing distributed data storage and processing across enterprise clusters, writing clean, efficient, and production-grade code tracked via GitHub, collaborating proactively with data scientists, software developers, and technical recruitment leadership at Go Ahead Consulting, adhering to strict corporate data governance and security standards, and managing application requirements by submitting your updated CV via direct email to hyd@goaheadconsulting.co.uk along with your notice period, current salary, expected salary, and current UAE visa status in the email body or application notes.

Key Responsibilities
  • Architect, design, build, and scale robust big data processing pipelines and distributed data architectures utilizing Hadoop and PySpark.
  • Develop, schedule, monitor, and optimize complex workflow orchestration DAGs using Apache Airflow.
  • Manage version control, code reviews, branching strategies, and CI/CD integration workflows via GitHub.
  • Process, transform, and clean large-scale structured and unstructured datasets to support downstream analytics and machine learning.
  • Optimize PySpark jobs, Spark configurations, and Hadoop cluster resource utilization for maximum query performance.
  • Collaborate proactively with cross-functional data science, software engineering, and product stakeholder squads.
  • Ensure strict adherence to enterprise data security baselines, privacy regulations, and governance frameworks.
  • Maintain meticulous technical documentation, data lineage maps, pipeline architecture blueprints, and operational runbooks.
  • Participate actively in agile sprint planning, architectural design reviews, and continuous improvement engineering sessions.
  • Provide technical leadership, troubleshooting support, and escalation expertise for mission-critical data pipelines.
Required Qualifications & Skills
  • Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related technical discipline.
  • Minimum of 5+ years of progressive professional hands-on experience in big data engineering and distributed systems.
  • Strong, demonstrable operational proficiency in Hadoop ecosystem technologies (HDFS, Hive, YARN) and PySpark.
  • Comprehensive working knowledge and practical hands-on experience with Apache Airflow for workflow orchestration.
  • Proven expertise in collaborative software development and version control using GitHub.
  • Solid practical grasp of data warehousing concepts, SQL tuning, ETL/ELT pipeline design, and performance optimization.
  • Strong analytical problem-solving skills and effective stakeholder communication capabilities.
  • Mandatory legal work authorization and current physical presence or visa status for operations in Dubai, UAE.
Nice-to-Have Skills
  • Professional vendor or cloud certifications such as Databricks Certified Associate Developer for Apache Spark, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer.
  • Prior professional working experience supporting Tier-1 global financial institutions, telecommunications providers, or enterprise consulting firms in the GCC.
  • Hands-on familiarity with cloud data warehouses (Snowflake, BigQuery, Redshift) and containerization tools (Docker, Kubernetes).
  • Experience utilizing enterprise agile tracking and IT service desk platforms (Jira, Confluence, ServiceNow).
  • Active participation in open-source big data communities, data engineering user groups, or PySpark contributor networks.

Salary/Rate: Market Competitive / Enterprise Big Data Package

Contract Duration: Permanent / Full-Time Employment

Notice Period: Required in application (Notice period, current salary, expected salary, and UAE visa status must be included)

Work Schedule: Professional Operations (Dubai, UAE)

Deadline: Open until filled (Urgent Dubai Senior Data Engineer Opening)

Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Senior Data Engineer - Hadoop, PySpark & Airflow Architect
Senior Data Engineer - Hadoop, PySpark & Airflow Architect

Go Ahead Consulting • Dubai

In loco
AED 350.000 - 550.000
Data Engineer (PySpark)
Data Engineer (PySpark)

Black Pearl Consult • Abu Dhabi

In loco
AED 260.000 - 380.000
Data Engineer | IT Excellence | Dubai, UAE
Data Engineer | IT Excellence | Dubai, UAE

Tech Junction Ltd • Dubai

In loco
AED 240.000 - 420.000
Senior Data Engineer – Azure Cloud & Data Platforms | CNS-ME | Abu Dhabi
Senior Data Engineer – Azure Cloud & Data Platforms | CNS-ME | Abu Dhabi

CNS-ME • Abu Dhabi

In loco
AED 320.000 - 520.000
Data Engineer
Data Engineer

Datamatics Technologies LLC • Sharjah

In loco
AED 134.000 - 201.000
Senior Data Engineer
Senior Data Engineer

NorthBay Solutions LLC • Abu Dhabi

In loco
AED 240.000 - 480.000
Data Engineer | Systems Limited | Dubai
Data Engineer | Systems Limited | Dubai

iSystems Ltd • Dubai

In loco
AED 201.000 - 335.000
Data Engineer – HRMS (ETL, Python & SQL) | IT People Gulf | Abu Dhabi
Data Engineer – HRMS (ETL, Python & SQL) | IT People Gulf | Abu Dhabi

IT People Gulf • Abu Dhabi

In loco
AED 180.000 - 300.000
Data & AI Solutions & Delivery Lead | Dubai, UAE
Data & AI Solutions & Delivery Lead | Dubai, UAE

Tech Junction Ltd • Dubai

In loco
AED 600.000 - 900.000
Senior Data Engineer
Senior Data Engineer

EPAM Systems Inc • Abu Dhabi

In loco
AED 420.000 - 660.000
End of service gratuity
Private healthcare and life insurance
Employee assistance program
+3
Luoghi più richiesti