Senior Big Data Engineer — Real-Time Streaming & Cloud

Teamware Solutions Inc.; Dubai; On-site; AED 360,000 - 540,000; Full time

Job summary

Teamware Human Resource Solutions LLC is seeking a Senior Data Engineer with 8–15 years of experience in Big Data and Data Engineering to join Dubai-based projects. You will design scalable ETL/ELT pipelines, build real-time streaming solutions with Kafka/Spark, and deploy data platforms across on-premise and cloud environments.

Ideal candidates will have Hadoop/Cloudera, Spark, Kafka, NiFi, Airflow/Oozie, and OpenShift/Kubernetes expertise, with a background in cloud platforms (AWS/Azure/GCP)

Qualifications

• 8–15 years of experience in Data Engineering / Big Data.
• Strong hands-on experience in Hadoop/Cloudera.
• Extensive experience with Apache Spark and Spark Streaming.
• Hands-on experience with Apache Kafka.
• Strong ETL/ELT and data pipeline development.
• Experience with Hive & HBase.
• Experience with AWS / Azure / GCP.
• Experience with Flink.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines for batch and real-time processing.
• Build and manage real-time streaming pipelines using Apache Kafka, Spark Streaming, and/or Flink.
• Develop high-performance data workflows across Cloudera/Hadoop and Cloud platforms.
• Build robust data ingestion frameworks using Kafka and Apache NiFi.
• Design and optimize data warehouse architectures and data models.
• Automate data workflows using Airflow and Oozie.
• Deploy and manage data applications on OpenShift/Kubernetes.
• Implement CI/CD pipelines and collaborate with DevOps teams for automated deployment.
• Drive data migration and integration across on-premise and cloud environments.
• Ensure strong data quality, governance, security, performance, and reliability.

Skills

Big Data

Data Engineering

Hadoop/Cloudera

Apache Spark

Apache Kafka

ETL/ELT

Hive/HBase

AWS/Azure/GCP

Flink

Airflow/Oozie

NiFi

OpenShift/Kubernetes

Data Migration

Data Quality & Governance

Education

Bachelor's degree in Computer Science/Engineering/Information Technology

Tools

Cloudera/Hadoop

Kafka/Spark/NiFi ecosystem

Job description

Teamware Human Resource Solutions LLC is seeking a Senior Data Engineer with 8–15 years of experience in Big Data and Data Engineering to join Dubai-based projects. You will design scalable ETL/ELT pipelines, build real-time streaming solutions with Kafka/Spark, and deploy data platforms across on-premise and cloud environments.

Ideal candidates will have Hadoop/Cloudera, Spark, Kafka, NiFi, Airflow/Oozie, and OpenShift/Kubernetes expertise, with a background in cloud platforms (AWS/Azure/GCP)