AI Engineer

Cognitive; Abu Dhabi; On-site; AED 350,000 - 650,000; Full time

Job summary

Cognitive in Abu Dhabi is seeking a senior AI/ML engineer to design, develop, and deploy scalable generative AI solutions and manage internal LLM deployments for production use. You will design robust RAG pipelines, architect and maintain Airflow DAGs, optimize vector databases (Pinecone/Milvus), monitor 99% uptime, and drive the full MLOps lifecycle from experimentation to serving.

Strong Python, databases, and cloud experience required; 8+ years in software engineering; bachelor's in CS/AI or

Qualifications

• Bachelor’s degree in CS/AI/related field.
• 8+ years in software engineering overall.
• Experience building LLM apps and production RAG pipelines.
• Production Python and DB experience.
• Open-source contributions to AI/MLOps libraries.

Responsibilities

• Design and implement robust RAG pipelines to integrate LLMs with proprietary data sources.
• Architect and maintain Apache Airflow DAGs for data ingestion, cleaning, embedding generation, and model retraining.
• Manage and optimize vector databases (Pinecone, Milvus, Weaviate) for embeddings.
• Maintain and monitor internal LLM deployments with high uptime and GPU resource optimization.
• Train, fine-tune, and deploy deep learning/LLM models across production MLOps lifecycle.

Skills

LLM development

RAG pipelines

Python

Airflow

Kubernetes

Distributed systems

Education

Bachelor's in CS/AI/SE

Tools

Docker

Kubernetes

LangChain

Pinecone

Milvus

Weaviate

Airflow

vLLM

Job description

You will be designing, developing and deploying scalable generative AI solutions while managing and maintaining LLM deployments for internal AI tools, ensuring high availability and efficiency for internal teams, and training and deploying deep learning and LLMs models to production.

Responsibilities

• Design and implement robust RAG pipelines to integrate Large Language Models (LLMs) with proprietary data sources, ensuring high accuracy and low latency in responses.
• Architect and maintain complex Apache Airflow DAGs to automate data ingestion, cleaning, embedding generation, and model retraining workflows.
• Manage and optimize Vector Databases (e.g., Pinecone, Milvus, Weaviate) for efficient storage and retrieval of high-dimensional embeddings.
• Maintain and monitor internal LLM deployments (hosting, scaling, and versioning), ensuring 99% uptime, managing GPU resources and optimizing inference for internal AI usage.
• Train, fine-tune, and deploy deep learning and LLM models to production environments, managing the full MLOps lifecycle from experimentation to serving.

Qualifications

• At least Bachelor's Degree in Computer Science, Software Engineering, Data Science, AI or related field
• 8+ years in Software Engineering overall
• Solid experience in building LLM applications and RAG pipelines, in production
• Production experience with Python and DBs.
• Open-source contributions to AI/MLOps libraries.
• Good foundation in linear algebra and statistics.

Required Skills

• Programming: Python (Must), Java (nice to have: GO, Scala)
• GenAI and LLMs: RAG, LangChain, LlamaIndex, RAG pipeline design. Nice to have: PEFT/LoRA fine-tuning.
• Data Processing: Apache Airflow, DAGs. Nice to have: PySpark or Ray.
• Databases: Vector DBs (Pinecone/Milvus), SQL/NoSQL, embedding models and semantic search
• MLOps: Docker, Kubernetes, vLLM, Cloud (AWS/Azure/GCP), CI/CD pipelines. Nice to have: GPU resource management and optimization (CUDA).