AI / Data Engineer

Business Umbrella; Abu Dhabi; On-site; AED 446,000 - 670,000; Full time

Job summary

Business Umbrella seeks a senior Data Engineer in Abu Dhabi to build robust ingestion pipelines for structured and unstructured data using Python. You will prepare data for LLM fine-tuning and create high-quality datasets for training and evaluation.

You will implement dataset versioning with DVC or LakeFS, generate embeddings with HuggingFace or Sentence Transformers, and manage vector indexes with FAISS or Weaviate to optimize retrieval.

Qualifications

• 10+ years in Data Engineering
• 2+ years in AI-adjacent data roles
• Proficiency in Python, Pandas, and text processing tools
• Familiar with tokenization libraries (HuggingFace Tokenizers, SentencePiece)
• Experience managing datasets and object storage (MinIO, NFS)
• Understanding of LLM data constraints (context windows, formatting, prompt injection)

Responsibilities

• Build ingestion pipelines for structured/unstructured data using Python
• Clean, normalize, and prepare data formats for LLM fine-tuning (e.g. JSONL, CSV)
• Create high-quality task-specific datasets for training and evaluation
• Apply versioning to datasets using DVC or LakeFS for reproducibility
• Generate embeddings using HuggingFace or Sentence Transformers
• Manage vector indexes (FAISS, Weaviate) and optimize retrieval workflows
• Tokenize and chunk longform data for context window optimization

Skills

Python

Pandas

Text processing

Tools

HuggingFace Tokenizers

SentencePiece

MinIO

NFS

FAISS

Weaviate

DVC

LakeFS

Job description

• Build ingestion pipelines for structured/unstructured data using Python
• Clean normalize and prepare data formats suitable for LLM finetuning (e.g. JSONL CSV)
• Create highquality taskspecific datasets for training and evaluation
• Apply versioning to datasets using DVC or LakeFS for reproducibility
• Generate embeddings using HuggingFace or Sentence Transformers
• Manage vector indexes (FAISS Weaviate) and optimize retrieval workflows
• Tokenize and chunk longform data for context window optimization

Requirements

• 10 years experience in Data Engineering role
• 2 years experience in AIadjacent data role
• Proficiency in Python pandas and text processing tools
• Familiarity with tokenization libraries (HuggingFace Tokenizers SentencePiece)
• Experience managing datasets and object storage (MinIO NFS)
• Understanding of LLM data constraints (context windows formatting prompt injection)