Staff Engineer (Platform Engineer)

Nagarro; Abu Dhabi; On-site; AED 120,000 - 200,000; Full time

Job summary

A prominent tech company is seeking a skilled individual to manage AWS Glue jobs for data ingestion and optimize performance for a Lakehouse architecture. The role involves developing ETL pipelines, executing historical data loading, and ensuring compliance with data contracts. Strong expertise in AWS Glue, collaboration with teams, and a proficiency in Git and Terraform are essential. This full-time position is based in Abu Dhabi and does not allow for remote work.

Qualifications

• Strong experience in AWS Glue and ETL pipeline development.
• Knowledge of Lakehouse architecture and Medallion design principles.
• Ability to work in a fast-paced environment and meet targets.

Responsibilities

• Build and implement AWS Glue jobs for Bronze layer ingestion.
• Optimize Glue job performance and collaborate with platform teams.
• Migrate source tables to Bronze layer and automate deployments.

Skills

AWS Glue

PySpark

ETL pipeline development

Lakehouse architecture

CDC delta loads

Agile working

Jira

Tools

AWS Services: Glue, S3, Athena, Lambda

Git

Terraform

Job description

Must have Skills

• AWS Glue Sodacore Lakehouse architecture CDC Medallion design principles

Key Responsibilities

• Data Ingestion Development
  • Build and implement AWS Glue jobs for Bronze layer ingestion using defined standards and templates.
  • Implement correct loading methods based on source requirements (CDC full load delta snapshot).
  • Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.
• Performance Optimization
  • Optimize Glue job performance (DPU allocation parallelization partitioning) according to best practices.
  • Collaborate with platform teams to ensure tooling and optimization alignment.
• Migration & Automation
  • Aggressively migrate source tables to Bronze layer initially using manual approaches with standards/templates later leveraging AI-enabled acceleration.
  • Ensure jobs are version-controlled and production deployment is automated via Git and Terraform.
• Governance & Monitoring
  • Implement source system connectivity into CDP in collaboration with source system owners.
  • Ensure jobs comply with data contracts and are properly monitored.
  • Prepare documentation and handover to operational support teams.
• Collaboration
  • Work closely with Data Architect for ingestion patterns and standards.
  • Coordinate with Data Assurance Lead to apply quality checks across all jobs.
  • Partner with platform engineers for tooling and optimization.

Required Skills & Experience

• Technical Expertise
  • Strong experience in AWS Glue PySpark and ETL pipeline development.
  • Knowledge of Lakehouse architecture and Medallion design principles.
  • Familiarity with CDC delta loads and historical data ingestion strategies.
• Tools & Technologies
  • AWS services: Glue S3 Athena Lambda.
  • Git Terraform for CI/CD automation.
  • Understanding of data quality frameworks (e.g. Soda Core) is a plus.
  • Able to identify ways to automate their work / repetitive tasks.
• Soft Skills
  • Ability to work in a fast-paced environment and deliver aggressive migration targets.
  • Strong collaboration and communication skills.
  • Work with Jira and agile way of working.

Remote Work : No

Employment Type : Full-time