Big Data Engineer - Government Sector

Apt Resources; Abu Dhabi; On-site; AED 120,000 - 200,000; Full time

Job summary

A government client in Abu Dhabi is seeking an experienced Big Data Engineer with 10-12 years of hands-on big data experience. You will design and implement large-scale data solutions to support AI/ML initiatives and aid in public sector digital transformation. The role involves development of robust data pipelines, management of data lakes, and collaboration on machine learning processes. Strong expertise in Python, SQL/NoSQL databases, and cloud data platforms is required, preferably Azure.

Qualifications

• 10-12 years of hands-on big data experience.
• Expertise in Python and SQL/NoSQL databases.
• Proven experience with Airflow for workflow orchestration.

Responsibilities

• Build robust data pipelines using Python SQL/NoSQL and Airflow.
• Design efficient data models for analytics.
• Prepare and process datasets for machine learning applications.

Skills

Python

SQL/NoSQL

Airflow

ETL/ELT

Cloud data platforms

Job description

Description

Apt Resourcesis seeking an experiencedBig Data Engineerfor agovernment client in Abu Dhabi. You will design and implement large-scale data solutions to support AI/ML initiatives and public sector digital transformation.

Key Responsibilities:

Data Pipeline Development:

• Build robust data pipelines usingPython SQL/NoSQL and Airflow
• DevelopETL/ELT processesfor structured/unstructured data
• Managedata lakesand optimize storage solutions

Data Infrastructure:

• Design efficientdata modelsfor analytics
• Implementdata governanceand quality frameworks
• Work withcloud-based data platforms(Azure preferred)

AI/ML Support:

• Prepare and process datasets for machine learning applications
• Collaborate with ML teams on feature engineering

Requirements

• 10-12 yearsof hands-on big data experience
• Expertise in:
• PythonandSQL/NoSQLdatabases
• Airflowfor workflow orchestration
• ETL/ELTpipeline development
• Cloud data platforms(Azure AWS or GCP)

Benefits

To be discussed