Middle Data Engineer

Suntech Innovation; United Arab Emirates; Hybrid; AED 180,000 - 320,000; Full time

Benefits offered by this job

Hybrid or remote working

Job summary

Suntech Innovation in the United Arab Emirates is seeking a data engineer to build the data pipelines powering experimentation and production systems. You will work mainly with Kafka, Spark on Databricks, and Airflow, modeling data for both experimentation and production serving, and partnering closely with data scientists and backend engineers on data contracts.

Requirements include 3+ years in data engineering or software engineering for data, strong Python (including PySpark), Kafka,

Qualifications

• 3+ years of data engineering or software engineering for data.
• Strong Python, including PySpark.
• Kafka and stream processing experience.
• Databricks experience and Airflow or equivalent.
• Strong SQL and data modeling with relational databases.
• Software engineering mindset and production-readiness.

Responsibilities

• Build and operate batch and streaming data pipelines (structured & unstructured data).
• Work primarily with Kafka, Spark on Databricks, and Airflow.
• Model and optimize data for both experimentation and production serving.
• Partner with data scientists and backend engineers on data contracts and reliability.
• Own pipeline quality: testing, monitoring, and performance.
• Participate in on-call rotation for the pipelines you build and operate.

Skills

Python

PySpark

Kafka

Airflow

Databricks

SQL

Data Modeling

Software Engineering Mindset

Tools

Kafka

Spark

Databricks

Docker

Kubernetes

Job description

About the team

We are a focused engineering team of 5 to 9 people at a multi-product techSaaS company. We work across personalization, recommendation, frauddetection, and some GenAI, through real-time, near-real-time, and batchpipelines. The team is a blend of Python backend engineers, ML engineers, QA(manual and automation), a data engineer, and data scientists. Our sharedmandate is to take experimentation from our data scientists and turn it into robust, production-grade systems, either as standalone services or integratedinto wider platforms.

The role

You will build the data pipelines that power our experimentation andproduction systems, the backbone behind fraud detection, recommendations,and personalization. You will work primarily with Kafka (sometimes Flink),Spark on Databricks, and Airflow, modelling and optimizing data for bothexperimentation and production serving, and partnering closely with datascientists and backend engineers on data contracts and reliability.

Responsibilities

• Build and operate batch and streaming data pipelines (structured &unstructured data)
• Work primarily with Kafka, Spark on Databricks, and Airflow
• Model and optimize data for both experimentation and production serving
• Partner with data scientists and backend engineers on data contracts and reliability
• Own pipeline quality: testing, monitoring, and performance
• Take part in the teams on-call rotation for the pipelines you build and operate

Requirements

• 3+ years of data engineering or software engineering for data
• Strong Python, including PySpark
• Kafka and stream processing
• Databricks experience
• Airflow or similar orchestration tools
• Strong SQL and data modeling, plus solid relational database skills
• Software engineering mindset

Nice to have

• Flink
• ML Feature stores
• MLOps and ML pipeline exposure
• Docker and Kubernetes
• SaaS or high-scale product domain experience

Why join us

• Work on high-scale, high-throughput systems
• Broad range of technologies, stacks, and problems
• Contribute to turning experimentation into production systems
• A strong engineering culture that values technical depth and ownership
• Flexibility through hybrid and remote working
• A distributed team across Dubai and Europe