Senior Data Engineer

BigData Technology Solutions; Dubai; On-site; AED 300,000 - 550,000; Full time

Job summary

BigData Technology Solutions in Dubai seeks a senior data engineer to lead enterprise data platforms, focusing on Informatica administration and development within Cloudera/Hadoop environments. You will design, deploy, monitor, and optimize data pipelines, data quality, and metadata governance across production and non-production systems.

The role requires hands-on expertise with Informatica PowerCenter/DEI, Cloudera or Hadoop, and modern data tooling such as Python, SQL, and cloud data

Qualifications

• 10+ years of relevant experience in data engineering, Informatica development, Informatica administration, or closely related roles.
• Strong hands-on experience supporting enterprise production environments and resolving data-platform incidents independently.
• Demonstrated experience across both Informatica and Cloudera or Hadoop environments.
• Experience working with high-volume data pipelines, data warehouses, or data lakes.
• Exposure to UAE or GCC enterprise or government environments is advantageous.

Responsibilities

• Support and maintain enterprise data engineering, ETL, data quality, metadata, and governance platforms.
• Develop, monitor, troubleshoot, and optimize Informatica workflows, mappings, applications, and data pipelines.
• Administer Informatica domains, nodes, repositories, Integration Services, and related application services.
• Support Informatica PowerCenter, BDM or DEI, IDQ, and EDC or Axon components in production and non-production environments.
• Investigate failed jobs, data-quality exceptions, performance issues, and platform incidents; complete root-cause analysis and implement appropriate corrective action.
• Support Cloudera Hadoop services and data processing across HDFS, Hive, Spark, and PySpark workloads.
• Perform deployments and migrations across development, quality assurance, user acceptance testing, and production environments.
• Use Linux, shell scripting, and SQL to automate routine tasks, validate data, and troubleshoot platform issues.
• Support user access, security, and permissions in line with established controls and operating procedures.
• Coordinate incident resolution and planned changes with application, database, infrastructure, business intelligence, and business stakeholders.
• Maintain accurate operational documentation, handover notes, change records, and support procedures.

Skills

PowerCenter

Informatica MDM/IICS

Cloudera/Hadoop

Python

SQL

Linux/Unix

Data pipelines

Tools

Informatica MDM/IICS

Kafka

NiFi

Kerberos

Ranger

Azure Data Factory

Databricks

Snowflake

Power BI

Kubernetes

Hadoop

Job description

Key Responsibilities

• Support and maintain enterprise data engineering, ETL, data quality, metadata, and governance platforms.
• Develop, monitor, troubleshoot, and optimize Informatica workflows, mappings, applications, and data pipelines.
• Administer Informatica domains, nodes, repositories, Integration Services, and related application services.
• Support Informatica PowerCenter, BDM or DEI, IDQ, and EDC or Axon components in production and non-production environments.
• Investigate failed jobs, data-quality exceptions, performance issues, and platform incidents; complete root-cause analysis and implement appropriate corrective action.
• Support Cloudera Hadoop services and data processing across HDFS, Hive, Spark, and PySpark workloads.
• Perform deployments and migrations across development, quality assurance, user acceptance testing, and production environments.
• Use Linux, shell scripting, and SQL to automate routine tasks, validate data, and troubleshoot platform issues.
• Support user access, security, and permissions in line with established controls and operating procedures.
• Coordinate incident resolution and planned changes with application, database, infrastructure, business intelligence, and business stakeholders.
• Maintain accurate operational documentation, handover notes, change records, and support procedures.

Skill Requirement

Informatica

• PowerCenter and BDM or DEI; administration of domains, nodes, repositories, Integration Services, and application services; ETL development, monitoring, troubleshooting, and performance tuning; IDQ; EDC or Axon.

Big Data

• Cloudera or Hadoop; HDFS; Hive; Spark or PySpark.

Core Platform Skills

• Linux or Unix; shell scripting; advanced SQL; Oracle or Microsoft SQL Server; data warehouse or data lake environments; enterprise production support.

• Informatica MDM or IICS
• Kafka, Impala, NiFi, Kerberos, or Ranger
• Python
• Microsoft Azure, Azure Data Factory, Databricks, or Snowflake
• Power BI, Vertica, or Kubernetes

Experience

• 10+ years of relevant experience in data engineering, Informatica development, Informatica administration, or closely related roles.
• Strong hands-on experience supporting enterprise production environments and resolving data-platform incidents independently.
• Demonstrated experience across both Informatica and Cloudera or Hadoop environments.
• Experience working with high-volume data pipelines, data warehouses, or data lakes.
• Exposure to UAE or GCC enterprise or government environments is advantageous.