AI Data Engineer: Scalable Pipelines & MLOps on Azure

INCEPTION ARTIFICIAL INTELLIGENCE L.L.C - O.P.C; Abu Dhabi; On-site; AED 250,000 - 380,000; Full time

Benefits offered by this job

Healthcare

Education support

Leave benefits

Job summary

Inception42 in Abu Dhabi is seeking a Data Engineer to build scalable data pipelines and develop AI solutions using Azure Agent Foundry, OpenAI, LangChain, and Databricks. You will design ETL/ELT workflows, ensure data quality, and collaborate with data scientists and product teams to deliver intelligent features.

The role emphasizes MLOps, DevOps integration, and containerized deployments in a cloud-first environment across enterprise-scale AI systems.

Qualifications

• Bachelor’s degree in CS/engineering/data science or related field.
• 3+ years in Data/AI/Data Engineering or ML Engineering roles.
• Experience delivering scalable AI/data solutions in cloud environments (Azure preferred).
• Strong proficiency in Python, SQL, PySpark, and ML/AI frameworks.

Responsibilities

• Develop and deploy AI agents using Azure Foundry, OpenAI, LangChain, and N8N.
• Build RAG pipelines, embeddings, vector DB integrations, and knowledge graphs.
• Design and maintain scalable data pipelines, ETL/ELT workflows, and data architectures.
• Build and maintain APIs and data services supporting AI/analytics workloads; optimize ML workflows on Azure Databricks and integrate with Azure Fabric.
• Ensure data quality, reliability, governance, and compliance across systems.
• Collaborate with data scientists, ML engineers, and product teams to deliver intelligent features.
• Support MLOps and DevOps processes including CI/CD, model lifecycle management, and containerized deployments.

Skills

Python

SQL

PySpark

LLMOps

Data Modeling

ETL

Problem-Solving

Communication

Education

Bachelor’s in Computer Science, Engineering, Data Science

Tools

Azure Agent Foundry

OpenAI APIs

LangChain

N8N

Azure Databricks

Azure Cloud Services

Spark

Hadoop

Flink

REST/GraphQL APIs

Docker

Kubernetes

Vector Databases

Job description

Inception42 in Abu Dhabi is seeking a Data Engineer to build scalable data pipelines and develop AI solutions using Azure Agent Foundry, OpenAI, LangChain, and Databricks. You will design ETL/ELT workflows, ensure data quality, and collaborate with data scientists and product teams to deliver intelligent features.

The role emphasizes MLOps, DevOps integration, and containerized deployments in a cloud-first environment across enterprise-scale AI systems.