Data Engineer

Deeplight AI; Abu Dhabi; On-site; AED 240,000 - 400,000; Full time

Benefits offered by this job

Competitive salary

Gold health insurance

Visa sponsorship

Dev & cert support

Subscription reimbursement

AI projects exposure

Incentive program

Career growth

Job summary

DeepLight AI is seeking a Data Engineer in the UAE to design, build, and optimize scalable Lakehouse data solutions powering analytics and modern ML models. You will work at the intersection of big data, cloud architecture, and MLOps within the banking sector, ensuring security and governance.

The role emphasizes cross-functional collaboration with data scientists, engineers, and stakeholders, and offers exposure to production-grade data pipelines, streaming analytics, and DevOps tooling in a

Qualifications

• Hands-on data engineering in banking/fintech environments.
• Strong command of AWS data services (S3, Glue, EMR, Redshift).
• Experience with lakehouse architectures (Databricks, Delta Lake).
• Proficient in Python and SQL; Docker/Kubernetes and Git workflows.
• Familiarity with ML workflows and MLOps concepts is a plus.

Responsibilities

• Build lakehouse architectures to unify data warehousing and data lake capabilities on AWS.
• Develop high-throughput batch and real-time ETL/ELT pipelines using PySpark, Kafka, and SQL.
• Optimize performance and storage across AWS services to reduce latency and costs.
• Support ML/AI initiatives by integrating feature engineering and deployment pipelines.
• Enforce data governance, security, and audit trails for banking-grade compliance.

Skills

Python

SQL

PySpark

Databricks

Delta Lake

AWS

Kafka

Docker

Kubernetes

Terraform

Git

Airflow

Tools

Databricks

Delta Lake

Kafka

Airflow

Terraform

SageMaker

Job description

Job Description:

DeepLight AI is a specialist AI and data consultancy dedicated to transforming the regional corporate landscape through bespoke, high-impact intelligent systems. We combine deep expertise in data engineering, cloud architecture, AI/ML platforms, and systems integration with a practical understanding of complex enterprise operations.

With deep roots across the financial services and banking sectors, we help major institutions bridge the gap between complex data infrastructure and actionable AI strategy. We dont build temporary fixes—we engineer secure, scalable, and resilient data foundations designed to power production AI at scale.

The Opportunity

We are seeking a Data Engineer with strong AWS and banking sector experience to join our core engineering squad in the UAE.

In this role, you will design, construct, and optimize scalable Lakehouse data solutions that power enterprise analytics and advanced machine learning models. Operating at the intersection of big data, cloud architecture, and MLOps, you will unify data warehousing and data lake capabilities to process complex, high-volume financial data while maintaining strict banking security, ACID compliance, and data governance standards.

What You’ll Do....

• Build Next-Gen Lakehouse Architectures: Design, build, and optimize scalable data solutions using lakehouse architectures (Databricks, Delta Lake) to unify data warehousing and data lake capabilities on AWS.


• Develop High-Throughput Pipelines: Build robust batch and real-time ETL/ELT pipelines using PySpark, Apache Kafka, and SQL to ingest and transform complex financial datasets from core banking, market data, and transactional systems.


• Optimize for Scale & FinOps: Maximize query performance and storage efficiency across AWS services (S3, Redshift, EMR, Athena, Glue) using advanced partitioning, indexing, and parallel processing to drive down latency and cloud compute costs.


• Support Machine Learning & MLOps: Partner with Data Scientists and MLOps Engineers to build automated feature engineering pipelines, track experiments (MLflow), containerize workflows (Docker, EKS), and deploy models to production (AWS SageMaker).


• Enforce Banking-Grade Governance: Implement end-to-end data security, encryption, access controls, and audit trails to ensure 100% compliance with Central Bank data sovereignty and privacy mandates.


• Automate Everything: Drive software engineering best practices by managing Infrastructure as Code (Terraform), authoring CI/CD pipelines (GitHub Actions/CodePipeline), and writing clean, well-tested Python code.

What We’re Looking For

• Hands-on data engineering experience directly within banking, financial services, or fintech environments, with a strong understanding of transactional datasets and regulatory compliance.


• Deep technical command of core AWS data services (S3, Glue, EMR, Redshift) alongside distributed processing engines (Apache Spark / PySpark, Apache Kafka).


• Proven track record building and managing Lakehouse architectures (Databricks, Delta Lake) with expert knowledge of data modeling, schema evolution, and ACID principles.


• High proficiency in Python and SQL (Scala is a plus), alongside practical experience with Docker, Kubernetes (EKS), Terraform, and Git workflows.


• Excellent analytical problem-solving skills with the ability to articulate complex technical concepts clearly to cross-functional teams and client stakeholders.

What Gives You an Edge

• AWS Certified Data Engineer, AWS Solutions Architect, or Databricks Certified Data Engineer.


• Hands-on experience with streaming analytics frameworks (Apache Flink, Spark Streaming) for real-time fraud detection or trade processing pipelines.


• Production experience with modern workflow schedulers like Apache Airflow or Prefect.

As an AI consultancy, our greatest asset is the expertise of our people.
While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.
Were looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.
You will be the face of our firm, tasked with building trust, articulating the \"why\" behind your technical decisions, and effectively \"selling\" your vision to high-level stakeholders.
If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.

The benefits youll enjoy as part of this role include:

• Competitive salary


• Comprehensive Gold Level personal health insurance


• Individual Visa Sponsorship


• Professional development and certification support


• Subscription reimbursement relating to your role


• Opportunity to work on cutting-edge AI projects


• Monthly Employee Incentive program


• Career advancement opportunities in a rapidly growing AI company
  

This position offers a unique opportunity to shape the future of AI implementation while working with a talented team of professionals at the forefront of technological innovation. The successful candidate will play a crucial role in driving our companys success in delivering transformative AI solutions to our clients.
At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions.

Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates.

If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.

Requirements: