Data Engineer

Deeplight AI; Dubai; Hybrid; AED 180,000 - 260,000; Full time

Benefits offered by this job

Salary

Health insurance

Visa sponsorship

Dev & cert support

Subscriptions reimbursement

AI projects

Incentive program

Career growth

Job summary

DeepLight AI is seeking a Data Engineer to join the UAE core engineering squad. You will design scalable lakehouse architectures on AWS, build robust batch and real-time pipelines, and partner with data scientists to deploy ML-ready features.

Required banking/financial services experience, strong Python/SQL, and hands-on with Docker, Kubernetes (EKS), Terraform, and Git workflows. This role offers Visa sponsorship and growth within a cutting-edge AI company.

Qualifications

• Hands-on data engineering in banking/fintech with regulatory awareness.
• Deep command of AWS data services (S3, Glue, EMR, Redshift).
• Proficient in Python and SQL; Docker/Kubernetes, Terraform, and Git workflows.

Responsibilities

• Design scalable lakehouse architectures on AWS using Databricks/Delta Lake.
• Build batch and real-time ETL/ELT pipelines with PySpark, Kafka, and SQL.
• Optimize performance and storage across AWS (S3, Redshift, EMR, Athena, Glue).
• Collaborate with data scientists to build feature pipelines and ML workflows (MLflow, SageMaker).
• Ensure data security, governance, encryption, and audit trails for banking compliance.
• Implement IaC and CI/CD pipelines (Terraform, GitHub Actions/CodePipeline).

Skills

Python programming

SQL

Data engineering

Distributed processing

Cloud architecture

Tools

Databricks

Delta Lake

Apache Spark

PySpark

Apache Kafka

Docker

Kubernetes (EKS)

Terraform

Git workflows

Airflow / Prefect

Job description

DeepLight AI is a specialist AI and data consultancy dedicated to transforming the regional corporate landscape through bespoke, high-impact intelligent systems. We combine deep expertise in data engineering, cloud architecture, AI/ML platforms, and systems integration with a practical understanding of complex enterprise operations.

With deep roots across the financial services and banking sectors, we help major institutions bridge the gap between complex data infrastructure and actionable AI strategy. We don't build temporary fixes—we engineer secure, scalable, and resilient data foundations designed to power production AI at scale.

The Opportunity

We are seeking a Data Engineer with strong AWS and banking sector experience to join our core engineering squad in the UAE.

What You'll Do

• Build Next-Gen Lakehouse Architectures: Design, build, and optimize scalable data solutions using Lakehouse architectures (Databricks, Delta Lake) to unify data warehousing and data lake capabilities on AWS
• Develop High-Throughput Pipelines: Build robust batch and real-time ETL/ELT pipelines using PySpark, Apache Kafka, and SQL to ingest and transform complex financial datasets from core banking, market data, and transactional systems
• Optimize for Scale & FinOps: Maximize query performance and storage efficiency across AWS services (S3, Redshift, EMR, Athena, Glue) using advanced partitioning, indexing, and parallel processing to drive down latency and cloud compute costs
• Support Machine Learning & MLOps: Partner with Data Scientists and MLOps Engineers to build automated feature engineering pipelines, track experiments (MLflow), containerize workflows (Docker, EKS), and deploy models to production (AWS SageMaker)
• Enforce Banking-Grade Governance: Implement end-to-end data security, encryption, access controls, and audit trails to ensure 100% compliance with Central Bank data sovereignty and privacy mandates
• Automate Everything: Drive software engineering best practices by managing Infrastructure as Code (Terraform), authoring CI/CD pipelines (GitHub Actions/CodePipeline), and writing clean, well-tested Python code

Requirements

• Hands-on data engineering experience directly within banking, financial services, or fintech environments, with a strong understanding of transactional datasets and regulatory compliance
• Deep technical command of core AWS data services (S3, Glue, EMR, Redshift) alongside distributed processing engines (Apache Spark / PySpark, Apache Kafka)
• Proven track record building and managing Lakehouse architectures (Databricks, Delta Lake) with expert knowledge of data modeling, schema evolution, and ACID principles
• High proficiency in Python and SQL (Scala is a plus), alongside practical experience with Docker, Kubernetes (EKS), Terraform, and Git workflows
• Excellent analytical problem-solving skills with the ability to articulate complex technical concepts clearly to cross-functional teams and client stakeholders

What Gives You an Edge

• AWS Certified Data Engineer, AWS Solutions Architect, or Databricks Certified Data Engineer
• Hands-on experience with streaming analytics frameworks (Apache Flink, Spark Streaming) for real-time fraud detection or trade processing pipelines
• Production experience with modern workflow schedulers like Apache Airflow or Prefect

As an AI consultancy, our greatest asset is the expertise of our people. While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight. We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills. You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.

If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.

Benefits

• Competitive salary
• Comprehensive 'Gold Level' personal health insurance
• Individual Visa Sponsorship
• Professional development and certification support
• Subscription reimbursement relating to your role
• Opportunity to work on cutting-edge AI projects
• Monthly Employee Incentive program
• Career advancement opportunities in a rapidly growing AI company

This position offers a unique opportunity to shape the future of AI implementation while working with a talented team of professionals at the forefront of technological innovation. The successful candidate will play a crucial role in driving our company's success in delivering transformative AI solutions to our clients.

At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions.

Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates. If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.