Senior Data Engineer: Real-Time Cloud Pipelines

TALENTMATE; Abu Dhabi; On-site; AED 180,000 - 300,000; Full time

Benefits offered by this job

Mission-driven work

Career growth

Flexible environment

Job summary

AI71 in Abu Dhabi is seeking a Senior Data Engineer to design, develop and maintain scalable data systems powering critical business decisions. You will lead robust data pipelines, ensure data quality and governance, and collaborate across cross-functional teams to deliver high-performance data platforms in production environments.

This role requires deep knowledge of modern data engineering practices, real-time processing, and cloud-native solutions, with hands-on work on Kafka, SQL/NoSQL, ELK

Qualifications

• Minimum 8+ years of data engineering experience in production systems.
• Strong Python programming and Linux shell scripting abilities.
• Expertise in SQL/NoSQL databases such as PostgreSQL and MongoDB.
• Experience building streaming pipelines with Kafka and ELK stack.

Responsibilities

• Design, develop and maintain scalable data pipelines for structured and real-time data.
• Ensure data quality, governance, timeliness and accuracy across systems.
• Develop data processing and optimization using denormalization, partitioning and caching.
• Build stream processing apps with Kafka and optimize for large data sets.
• Deploy on AWS/GCP, containerize tasks with Docker, orchestrate with Kubernetes.
• Collaborate with cross-functional teams including data scientists and DevOps.

Skills

Python

Linux shell

SQL/NoSQL

Apache Kafka

Docker

Kubernetes

AWS/Azure

ELK stack

Data modeling

Education

Bachelor’s or Master’s degree in CS/Engineering/Data Science

Tools

PostgreSQL

MongoDB

Apache Kafka

Elasticsearch

Kibana

Docker

Kubernetes

Job description

AI71 in Abu Dhabi is seeking a Senior Data Engineer to design, develop and maintain scalable data systems powering critical business decisions. You will lead robust data pipelines, ensure data quality and governance, and collaborate across cross-functional teams to deliver high-performance data platforms in production environments.

This role requires deep knowledge of modern data engineering practices, real-time processing, and cloud-native solutions, with hands-on work on Kafka, SQL/NoSQL, ELK