Lead Data Engineer, Group 42 (G42)

Analytics Insight; Abu Dhabi; On-site; AED 180,000 - 220,000; Full time

Job summary

Analytics Insight is seeking a Lead Data Engineer in Abu Dhabi to build scalable data pipelines for AI applications. This role involves collaboration with AI engineers to develop reliable data infrastructure for vector search and RAG systems. Candidates should possess a bachelor's degree in computer science or engineering, with eight years of experience in data engineering and exceptional programming skills in Python and SQL. Experience with ETL workflows, NoSQL databases, and Kubernetes is also essential.

Qualifications

• Eight years of experience in data engineering and distributed systems.
• Experience creating and implementing RAG pipelines for AI systems.
• Practical knowledge of ETL pipeline development frameworks.

Responsibilities

• Design and optimize scalable data pipelines for advanced AI workloads.
• Develop ETL and ELT workflows for structured and unstructured data.
• Build vector database indexing and similarity search pipelines.

Skills

Python programming

SQL proficiency

NoSQL databases

Data modeling

ETL and ELT workflows

Kubernetes

Education

Bachelor’s degree in computer science or engineering

Tools

Vector databases

Graph databases

Job description

# Lead Data Engineer, Group 42 (G42)United Arab Emirates, Abu DhabiMay 13, 2026Full TimeApply Now### Job DescriptionThe Lead Data Engineer at Inception builds scalable data pipelines which support both AI and LLM and RAG system activities. The role focuses on building reliable data infrastructure which enables vector search and retrieval systems while maintaining data quality across multiple platforms. The project requires teamwork between AI engineers and engineering teams to create secure data solutions which operate at high performance for enterprise systems.**Job ID**: 2805**Date Posted**: NA**Expiration Date:** NA**Apply:** Click Here**Main Duties*** Design and optimize scalable data pipelines for advanced AI and LLM workloads efficiently.* Develop ETL and ELT workflows for structured, unstructured, and streaming enterprise data systems.* Build vector database indexing and similarity search pipelines supporting intelligent retrieval systems.* Enable RAG, semantic search, and enterprise knowledge retrieval solutions across distributed environments.* Ensure data quality, monitoring, observability, and reliability across large-scale production data pipelines.**Essential Qualifications*** Requires applicants to have a bachelor’s degree in computer science or engineering or a related technical field.* Eight years of experience in data engineering and distributed systems and AI infrastructure development.* Possesses advanced skills in Python programming which he uses for data processing and automation and API development and handling distributed workloads.* Demonstrates SQL proficiency and possesses practical experience with NoSQL databases in enterprise-level settings.* Possesses practical knowledge of vector databases and data modeling and ETL pipeline development frameworks.**Preferred Qualifications*** Experience in creating and implementing RAG pipelines which meet production standards for use in enterprise AI systems.* Expertise in using graph databases together with hybrid search systems to develop advanced enterprise data retrieval solutions.* Maintains knowledge about LLM deployment together with inference optimization and caching methods that support scalable AI system development.* Acquired knowledge about cloud security procedures together with data management practices and identity access control systems that operate on multiple platforms.* Worked with Kubernetes together with distributed systems to create infrastructure deployments that deliver both scalability and reliability.