NLP Data Engineer – Foundation Models & Data Pipelines

Institute of Foundation Models; Abu Dhabi; On-site; AED 240,000 - 420,000; Full time

Job summary

Institute of Foundation Models in Abu Dhabi invites a Data Engineer specializing in NLP and large-scale data processing to rapidly collect, curate, and prepare high‑quality datasets for cutting-edge NLP research. You will build scalable data pipelines, refine LLM outputs, and collaborate with researchers to ensure data quality and reproducibility.

You will design and maintain web crawling workflows, APIs, and automated data workflows, leveraging Python and modern data technologies to support

Qualifications

• Bachelor's degree in CS or related field.
• Extensive experience in data engineering, processing, and automation using Python.
• Proven ability to design web crawling solutions and data pipelines.
• Strong knowledge of data structures, databases, and SQL.
• Experience with cloud infra and distributed processing (AWS, Spark, Kafka, Kubernetes).
• Excellent problem-solving and collaboration skills.

Responsibilities

• Rapidly collect, curate, and preprocess datasets per NLP researchers' specs, typically within 1–2 days.
• Develop and maintain web crawling solutions, APIs, and automated workflows.
• Refine outputs from LLMs to generate structured datasets for model training.
• Implement scalable data pipelines for storage, retrieval, and distribution to research teams.
• Collaborate with researchers and engineers to ensure data quality and relevance.
• Document data collection methodologies and pipeline architecture clearly.

Skills

Python

Data engineering

Web crawling

LLM data refinement

Education

Bachelor's degree in Computer Science

Master's degree or equivalent in Computer Science / Data Engineering

Tools

AWS

Spark

Kafka

Kubernetes

SQL

Job description

Institute of Foundation Models in Abu Dhabi invites a Data Engineer specializing in NLP and large-scale data processing to rapidly collect, curate, and prepare high‑quality datasets for cutting-edge NLP research. You will build scalable data pipelines, refine LLM outputs, and collaborate with researchers to ensure data quality and reproducibility.

You will design and maintain web crawling workflows, APIs, and automated data workflows, leveraging Python and modern data technologies to support