Data Engineer

Venquis; Abu Dhabi; On-site; AED 240,000 - 480,000; Full time

Job summary

Venquis is seeking an AWS Glue Data Engineer to join the Data Factory Squad in Abu Dhabi/Dubai. You will build scalable ingestion pipelines, using AWS Glue and PySpark, to migrate source systems into the Lakehouse ingestion zone.

You will implement CDC, full/delta loads, and historical loading strategies, optimize performance, and automate deployments via Git and Terraform. Collaboration with Data Architect and Data Assurance is essential.

Qualifications

• Strong experience in AWS Glue, PySpark, and ETL pipeline development.
• Knowledge of Lakehouse architecture and Medallion design principles.
• Familiarity with CDC, delta loads, and historical data ingestion strategies.

Responsibilities

• Data Ingestion Development: Build and implement AWS Glue jobs for Bronze layer ingestion using defined standards and templates.
• Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.
• Performance Optimization: Optimize Glue job performance (DPU allocation, parallelization, partitioning).
• Migration & Automation: Migrate source tables to Bronze layer with templates, later using AI-enabled acceleration; ensure version control and automated deployment via Git and Terraform.
• Governance & Monitoring: Ensure jobs comply with data contracts and are properly monitored; prepare documentation.

Skills

AWS Glue

PySpark

ETL pipeline development

Lakehouse architecture

Medallion design principles

CDC

delta loads

historical data ingestion

Tools

AWS Glue

S3

Athena

Lambda

Git

Terraform

Soda Core

Job description

Role: AWS Glue Data Engineer – Data Factory Squad

Location: Abu Dhabi – Dubai Employment Type: Full-time

Overview

We are seeking a skilled AWS Glue Data Engineer to join our Data Factory Squad, responsible for migrating source systems into the Lakehouse ingestion zone. This role focuses on building scalable ingestion pipelines, optimizing performance, and ensuring compliance with architectural and data assurance standards.

Key Responsibilities

• Data Ingestion Development
• Build and implement AWS Glue jobs for Bronze layer ingestion using defined standards and templates.
• Implement correct loading methods based on source requirements (CDC, full load, delta, snapshot).
• Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.
• Performance Optimization
• Optimize Glue job performance (DPU allocation, parallelization, partitioning) according to best practices.
• Collaborate with platform teams to ensure tooling and optimization alignment.
• Migration & Automation
• Aggressively migrate source tables to Bronze layer, initially using manual approaches with standards/templates, later leveraging AI-enabled acceleration.
• Ensure jobs are version-controlled and production deployment is automated via Git and Terraform.
• Governance & Monitoring
• Implement source system connectivity into CDP in collaboration with source system owners.
• Ensure jobs comply with data contracts and are properly monitored.
• Prepare documentation and handover to operational support teams.
• Collaboration
• Work closely with Data Architect for ingestion patterns and standards.
• Coordinate with Data Assurance Lead to apply quality checks across all jobs.
• Partner with platform engineers for tooling and optimization.

Required Skills & Experience

• Technical Expertise
• Strong experience in AWS Glue, PySpark, and ETL pipeline development.
• Knowledge of Lakehouse architecture and Medallion design principles.
• Familiarity with CDC, delta loads, and historical data ingestion strategies.
• Tools & Technologies
• AWS services: Glue, S3, Athena, Lambda.
• Git, Terraform for CI/CD automation.
• Understanding of data quality frameworks (e.g., Soda Core) is a plus.
• Able to identify ways to automate their work / repetitive tasks.
• Soft Skills
• Ability to work in a fast-paced environment and deliver aggressive migration targets.
• Strong collaboration and communication skills.
• Work with Jira and agile way of working.

Preferred Qualifications

• 5+ years in data engineering roles, with hands-on experience in AWS Glue.
• Experience in financial services or banking domain is an advantage.