AI Data Engineer (Document Intelligence & GenAI)

Krisvconsulting Services Pte Ltd; Singapore; On-site; SGD 60,000 - 100,000; Full time

Job summary

Krisvconsulting Services Pte Ltd is seeking a data engineering professional to design, build, and maintain scalable document ingestion and content extraction pipelines using OCR, LLMs, and vector Databases. You will develop embedding models and semantic search to enhance enterprise AI performance while collaborating with AI, platform, and data teams across Azure, Fabric, Databricks, and Delta Lake.

The role emphasizes governance, security, and CI/CD best practices.

Qualifications

• Bachelor's degree in CS, Data Engineering, AI, or related field.
• 1–3 years in Data Engineering, AI data platforms, or document intelligence.
• Strong Python and SQL skills; experience with PySpark or Apache Spark.
• Hands-on experience with OCR, Vision Language Models (VLMs), LLMs, prompt engineering, and PEFT/LoRA.
• Experience building document processing pipelines and semantic search capabilities.
• Familiarity with cloud platforms like Azure, Fabric, Databricks, Delta Lake, Kafka, LangChain, LlamaIndex.
• Understanding of CI/CD, version control, data governance, security and access controls.

Responsibilities

• Design, develop, and maintain scalable document ingestion and content extraction pipelines capable of processing structured and unstructured data.
• Build and optimize AI-powered document processing solutions using OCR, Vision Language Models (VLMs), Large Language Models (LLMs), and multimodal AI technologies.
• Develop and enhance embedding models, semantic search capabilities, and vector databases for knowledge retrieval.
• Implement knowledge ingestion, indexing, vectorization, and Retrieval-Augmented Generation (RAG) pipelines to support AI assistants.
• Build batch and streaming data pipelines using modern data engineering frameworks and integrate data from APIs, databases, cloud platforms, and enterprise systems.
• Collaborate with AI engineers, platform engineers, and data teams to integrate AI pipelines with cloud platforms such as Azure, Fabric, Databricks, and Delta Lake.
• Apply data governance, security, version control, and CI/CD best practices while ensuring reliable, scalable AI data pipelines.
• Continuously evaluate and implement emerging AI, document intelligence, and data engineering technologies to improve platform performance and automation.

Skills

Python

SQL

Spark

Prompt engineering

Education

Bachelor's degree in Computer Science or related field

Tools

PySpark

Apache Spark

LangChain

LlamaIndex

Delta Lake

Azure

Databricks

Kafka

Job description

Key Responsibilities

• Design, develop, and maintain scalable document ingestion and content extraction pipelines capable of processing structured and unstructured data, including PDFs, Office documents, images, audio, and video.
• Build and optimize AI-powered document processing solutions using OCR, Vision Language Models (VLMs), Large Language Models (LLMs), and multimodal AI technologies for intelligent content extraction.
• Develop and enhance embedding models, semantic search capabilities, and vector databases to improve knowledge retrieval and enterprise-specific AI performance.
• Implement knowledge ingestion, indexing, vectorization, and Retrieval-Augmented Generation (RAG) pipelines to support enterprise AI assistants and agentic AI applications.
• Build batch and streaming data pipelines using modern data engineering frameworks and integrate data from APIs, databases, cloud platforms, and enterprise systems.
• Collaborate with AI engineers, platform engineers, and data teams to integrate AI pipelines with cloud platforms such as Azure, Microsoft Fabric, Databricks, and Delta Lake.
• Apply data governance, security, version control, and CI/CD best practices while ensuring reliable, scalable, and maintainable AI data pipelines.
• Continuously evaluate and implement emerging AI, document intelligence, and data engineering technologies to improve platform performance and automation.

Key Requirements

• Bachelor's Degree in Computer Science, Data Engineering, Artificial Intelligence, Engineering, or a related discipline.
• 1–3 years of experience in Data Engineering, AI Data Platforms, Document Intelligence, or related fields. Fresh graduates with relevant projects are encouraged to apply.
• Strong programming skills in Python, SQL, and experience with PySpark or Apache Spark for large-scale data processing.
• Hands‑on experience with OCR, Vision Language Models (VLMs), Large Language Models (LLMs), prompt engineering, Text‑to‑Speech technologies, and AI model fine‑tuning techniques such as PEFT and LoRA.
• Experience building document processing pipelines, semantic search solutions, vector databases, and Retrieval-Augmented Generation (RAG) architectures.
• Familiarity with cloud platforms and modern data technologies such as Microsoft Azure, Microsoft Fabric, Databricks, Delta Lake, Kafka, LangChain, LlamaIndex, or similar frameworks.
• Understanding of CI/CD, version control, data governance, metadata management, and secure data access principles.
• Strong analytical, problem-solving, communication, and documentation skills with the ability to work independently and collaboratively in Agile environments.