AI-Driven Data Engineer: Pipelines & MLOps

Inception42; Abu Dhabi; On-site; AED 180,000 - 300,000; Full time

Benefits offered by this job

Healthcare

Education support

Leave benefits

Global team

Job summary

Inception42, a G42 company, is the region's leading AI innovator with domain-specific and industry-agnostic products. We are seeking a Data Engineer in Abu Dhabi to build scalable data pipelines and AI solutions using Azure Agent Foundry, OpenAI, LangChain, and Databricks.

You will collaborate with data scientists, ML engineers, and product teams to deliver reliable analytics and AI features while supporting MLOps and DevOps processes.

Qualifications

• Bachelor's in Computer Science, Engineering, Data Science, or related field.
• 3+ years experience in Data Engineering, AI Engineering, or ML Engineering roles.
• Experience delivering scalable AI/data solutions in cloud environments (Azure preferred).
• Strong proficiency in Python, SQL, PySpark, and ML/AI frameworks.
• Experience with Azure Agent Foundry, OpenAI APIs, LLMOps, RAG systems, and vector databases.
• Hands-on experience with Azure Databricks, Azure cloud services, and data platforms.
• Knowledge of big data tools (Spark, Hadoop, Flink) and modern data storage (SQL, NoSQL).
• Experience with REST/GraphQL APIs, Kafka/RabbitMQ, Docker/Kubernetes.
• Solid understanding of data modeling, ETL, and data quality best practices.

Responsibilities

• Develop and deploy AI agents using Azure Agent Foundry, OpenAI, LangChain, and N8N.
• Build RAG pipelines, embeddings, vector database integrations, and knowledge graph solutions.
• Design and maintain scalable data pipelines, ETL/ELT workflows, and data architectures.
• Build and maintain APIs and data services for AI and analytics workloads; optimize ML on Databricks.
• Ensure data quality, reliability, governance and compliance across systems.
• Collaborate with data scientists, ML engineers, and product teams to deliver intelligent features.
• Support MLOps and DevOps with CI/CD, model lifecycle, and container deployments.

Skills

Python

SQL

PySpark

ML/AI frameworks

REST/GraphQL APIs

Kafka/RabbitMQ

Docker/Kubernetes

Data modeling

ETL/ELT

Communication

Education

Bachelor's in Computer Science, Engineering, Data Science, or related field

Tools

Azure Agent Foundry

OpenAI APIs

LangChain

N8N

Azure Databricks

Azure cloud services

Spark

Hadoop

Job description

Inception42, a G42 company, is the region's leading AI innovator with domain-specific and industry-agnostic products. We are seeking a Data Engineer in Abu Dhabi to build scalable data pipelines and AI solutions using Azure Agent Foundry, OpenAI, LangChain, and Databricks.

You will collaborate with data scientists, ML engineers, and product teams to deliver reliable analytics and AI features while supporting MLOps and DevOps processes.