Senior Data Engineer – Real-Time Payments & Event-Driven Systems

Talenzon; Dubai; On-site; AED 300,000 - 420,000; Full time

Job summary

Talenzon seeks a Senior Data Engineer to join the on-site Dubai team, focusing on real-time data processing, scalable pipelines, and robust data platforms for financial transactions. You will collaborate with software engineers, data scientists, and analytics teams to deliver low-latency, consistent data infrastructure for fraud detection and risk analytics.

The role requires strong Python/SQL skills, hands-on Kafka/Spark experience, and AWS expertise.

Qualifications

• 5+ years of experience in data engineering or distributed data platforms.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Kafka and event-driven architectures.
• Experience with Apache Spark or equivalent distributed processing technologies.
• Strong knowledge of data modelling, ETL/ELT, and streaming pipeline design.
• Experience with AWS services, including Amazon S3, Amazon Redshift, and Amazon CloudWatch.
• Understanding of event delivery semantics, idempotency, retry mechanisms, and data consistency.
• Experience with monitoring, data quality, and production troubleshooting.
• Excellent English communication skills.

Responsibilities

• Design and implement event-driven data architectures using Apache Kafka and managed streaming services.
• Build streaming pipelines for transaction events, payment activity, and operational telemetry.
• Implement data transformations, enrichment, validation, and routing.
• Manage event schemas and compatibility using schema registry technologies.
• Design idempotent processing patterns and strategies for duplicate events and failed messages.
• Develop data processing applications using Python, SQL, and Apache Spark.
• Build batch and streaming ingestion pipelines for structured and semi-structured data.
• Design analytical storage architectures using Amazon S3 and Amazon Redshift.
• Implement partitioning, indexing, and data lifecycle policies to improve performance and cost efficiency.
• Optimize data processing jobs for throughput, latency, and resource utilization.
• Implement data quality checks, reconciliation processes, and automated validation.
• Configure monitoring and alerting for pipeline failures, consumer lag, and processing delays.
• Develop recovery procedures for failed jobs and replayable event streams.
• Ensure data lineage, traceability, and auditability across pipelines.
• Collaborate with security and compliance teams on financial data handling requirements.

Skills

Python

SQL

Apache Kafka

Apache Spark

Data modelling

ETL/ELT

Streaming pipelines

AWS services

Data quality

English communication

Tools

Amazon S3

Amazon Redshift

Amazon CloudWatch

Schema registry

Job description

We are looking for a Senior Data Engineer with strong experience in real-time data processing, distributed systems, and event-driven architectures to join our client’s on-site team in Dubai.

This role focuses on building scalable data pipelines and streaming platforms for high-volume financial transactions, operational analytics, and fraud detection. You will work with software engineers, data scientists, and analytics teams to deliver reliable data infrastructure capable of processing large volumes of events with low latency and strong data consistency.

What You’ll Do

Real‑Time Data Engineering

• Design and implement event-driven data architectures using Apache Kafka and managed streaming services.
• Build streaming pipelines for transaction events, payment activity, and operational telemetry.
• Implement data transformations, enrichment, validation, and routing.
• Manage event schemas and compatibility using schema registry technologies.
• Design idempotent processing patterns and strategies for duplicate events and failed messages.

Data Processing & Storage

• Develop data processing applications using Python, SQL, and Apache Spark.
• Build batch and streaming ingestion pipelines for structured and semi-structured data.
• Design analytical storage architectures using Amazon S3 and Amazon Redshift.
• Implement partitioning, indexing, and data lifecycle policies to improve performance and cost efficiency.
• Optimize data processing jobs for throughput, latency, and resource utilization.

Reliability & Data Quality

• Implement data quality checks, reconciliation processes, and automated validation.
• Configure monitoring and alerting for pipeline failures, consumer lag, and processing delays.
• Develop recovery procedures for failed jobs and replayable event streams.
• Ensure data lineage, traceability, and auditability across pipelines.

Security & FinTech Requirements

• Implement encryption in transit and at rest, access controls, and sensitive-data protection.
• Apply data masking and tokenization where appropriate.
• Collaborate with security and compliance teams on financial data handling requirements.
• Support data pipelines used by fraud detection, transaction monitoring, and risk analytics systems.

What We’re Looking For

Required Skills & Experience

• 5+ years of experience in data engineering or distributed data platforms.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Kafka and event-driven architectures.
• Experience with Apache Spark or equivalent distributed processing technologies.
• Strong knowledge of data modelling, ETL/ELT, and streaming pipeline design.
• Experience with AWS services, including Amazon S3, Amazon Redshift, and Amazon CloudWatch.
• Understanding of event delivery semantics, idempotency, retry mechanisms, and data consistency.
• Experience with monitoring, data quality, and production troubleshooting.
• Excellent English communication skills.

Nice to Have

• Experience in payments, banking, FinTech, or transaction processing.
• Knowledge of Apache Flink or Kafka Streams.
• Experience with Debezium and change data capture (CDC).
• Familiarity with AWS Glue and AWS Lake Formation.
• Experience designing fraud detection or real-time risk analytics pipelines.

Location: Dubai, UAE Work Model: On-site Role Type: Full-Time Language: English required