AI Data Engineer: Build Scalable Data Pipelines

Gender; Muscat; On-site; OMR 18,000 - 32,000; Full time

Job summary

Gender is seeking an experienced Data Engineer in Muscat to design, develop, and maintain the data infrastructure for AI initiatives. You will build and manage pipelines that ingest, transform, and deliver data for analytics and decision making.

The role requires 6+ years in data management, strong expertise in Kafka, Airflow, Spark, and programming in Java or Python. You will collaborate across teams to ensure data quality, security, governance, and scalable, efficient data processing.

Qualifications

• Bachelor's degree in CS/DS/SE or related quantitative field.
• At least six years of experience in data management, integration, modeling, and data quality.
• Proven experience developing and maintaining data pipelines for AI projects.

Responsibilities

• Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and load it into the appropriate data storage systems
• Collaborates with AI developers, data scientists and analysts to optimize models for data quality, security, and governance
• Integrates data from different sources, including databases, data warehouses, APIs, and external systems
• Ensures data consistency and integrity during the integration process, performing data validation and cleaning as needed
• Transforms raw data into a usable format by applying data cleansing, aggregation, filtering, and enrichment techniques
• Optimizes data pipelines and data processing workflows for performance, scalability, and efficiency
• Monitors and tunes data systems, identifies and resolves performance bottlenecks, and implements caching and indexing strategies to enhance query performance
• Implements data quality checks and validations within data pipelines to ensure the accuracy, consistency, and completeness of data
• Supports the governance of data and algorithms used for analysis, analytical applications, and automated decision making

Skills

Problem solving

Cross-functional collaboration

Business acumen

Data governance awareness

Education

Bachelor's degree in CS/DS/SE

Master's degree preferred

Tools

Kafka

Airflow

Spark

Java

Python

C/C++

SQL

R

SAS

Excel

Snowflake

Databricks

AWS

GCP

Azure

Job description

Gender is seeking an experienced Data Engineer in Muscat to design, develop, and maintain the data infrastructure for AI initiatives. You will build and manage pipelines that ingest, transform, and deliver data for analytics and decision making.

The role requires 6+ years in data management, strong expertise in Kafka, Airflow, Spark, and programming in Java or Python. You will collaborate across teams to ensure data quality, security, governance, and scalable, efficient data processing.