Senior Real-Time Data Engineer for Payments & Streaming

Talenzon; Dubai; On-site; AED 300,000 - 420,000; Full time

Job summary

Talenzon seeks a Senior Data Engineer to join the on-site Dubai team, focusing on real-time data processing, scalable pipelines, and robust data platforms for financial transactions. You will collaborate with software engineers, data scientists, and analytics teams to deliver low-latency, consistent data infrastructure for fraud detection and risk analytics.

The role requires strong Python/SQL skills, hands-on Kafka/Spark experience, and AWS expertise.

Qualifications

• 5+ years of experience in data engineering or distributed data platforms.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Kafka and event-driven architectures.
• Experience with Apache Spark or equivalent distributed processing technologies.
• Strong knowledge of data modelling, ETL/ELT, and streaming pipeline design.
• Experience with AWS services, including Amazon S3, Amazon Redshift, and Amazon CloudWatch.
• Understanding of event delivery semantics, idempotency, retry mechanisms, and data consistency.
• Experience with monitoring, data quality, and production troubleshooting.
• Excellent English communication skills.

Responsibilities

• Design and implement event-driven data architectures using Apache Kafka and managed streaming services.
• Build streaming pipelines for transaction events, payment activity, and operational telemetry.
• Implement data transformations, enrichment, validation, and routing.
• Manage event schemas and compatibility using schema registry technologies.
• Design idempotent processing patterns and strategies for duplicate events and failed messages.
• Develop data processing applications using Python, SQL, and Apache Spark.
• Build batch and streaming ingestion pipelines for structured and semi-structured data.
• Design analytical storage architectures using Amazon S3 and Amazon Redshift.
• Implement partitioning, indexing, and data lifecycle policies to improve performance and cost efficiency.
• Optimize data processing jobs for throughput, latency, and resource utilization.
• Implement data quality checks, reconciliation processes, and automated validation.
• Configure monitoring and alerting for pipeline failures, consumer lag, and processing delays.
• Develop recovery procedures for failed jobs and replayable event streams.
• Ensure data lineage, traceability, and auditability across pipelines.
• Collaborate with security and compliance teams on financial data handling requirements.

Skills

Python

SQL

Apache Kafka

Apache Spark

Data modelling

ETL/ELT

Streaming pipelines

AWS services

Data quality

English communication

Tools

Amazon S3

Amazon Redshift

Amazon CloudWatch

Schema registry

Job description

Talenzon seeks a Senior Data Engineer to join the on-site Dubai team, focusing on real-time data processing, scalable pipelines, and robust data platforms for financial transactions. You will collaborate with software engineers, data scientists, and analytics teams to deliver low-latency, consistent data infrastructure for fraud detection and risk analytics.

The role requires strong Python/SQL skills, hands-on Kafka/Spark experience, and AWS expertise.