Senior AI Data Engineer: AI Harness Architect & Pipelines

NCS Group; Singapore; On-site; SGD 120,000 - 210,000; Full time

Job summary

NCS Group in Singapore seeks an autonomous AI Engineer to design and build the AI harness, agent framework, and reusable components for a modern data platform based on Iceberg, Trino and Parquet. The role enables multi-agent collaboration, stateful workflows, and end-to-end data engineering capabilities.

Depending on experience, you will focus on task execution, independent delivery, or technical leadership, while collaborating across teams to discover Iceberg catalogs, generate metadata, ingest

Qualifications

• Experience with Generative AI, Agentic AI, or LLM-based solutions.
• Proficiency in Python, including FastAPI, Pydantic and Pandas.
• Experience designing and implementing AI Harness or multi-agent frameworks.
• Hands-on experience with LangChain & LangGraph.
• Experience with Apache Iceberg, Trino, and Parquet-based lakehouse architectures.
• Experience building data ingestion, transformation, and orchestration pipelines.
• Strong knowledge of metadata management, data lineage, and data governance practices.
• Experience developing REST APIs, SDKs, and reusable engineering frameworks.
• Experience with observability, monitoring, tracing, and operational support practices.
• Strong understanding of distributed systems, event-driven architectures, and microservices patterns.
• Proven ability to design, develop, and operationalize enterprise-grade AI agents for data discovery, ingestion, transformation, optimization, governance, and observability.
• Familiar with modern data engineering technologies, including DuckDB, dbt, and related engineering tools for data transformation, modelling, and pipeline development.
• Senior engineers will be expected to have demonstrated evidence of leading technical designs, guiding junior engineering team members, working independently with clients and delivery teams, and translating AI/data requirements into scalable, secure, and deployable solutions.

Responsibilities

• Design and implement the foundational AI Harness that provides agent orchestration, workflow execution and state management.
• Develop specialized agents for data discovery, ingestion, transformation, and governance.
• Ensure observability, security, and governance across pipelines and platforms.
• Collaborate with teams to deliver enterprise-grade AI solutions for data platforms.

Skills

Generative AI

Agentic AI

Python

FastAPI

Pydantic

Pandas

LangChain

LangGraph

Iceberg

Trino

Parquet

APIs

Data governance

Observability

Distributed systems

Lead technical design

Tools

DuckDB

dbt

Job description

NCS Group in Singapore seeks an autonomous AI Engineer to design and build the AI harness, agent framework, and reusable components for a modern data platform based on Iceberg, Trino and Parquet. The role enables multi-agent collaboration, stateful workflows, and end-to-end data engineering capabilities.

Depending on experience, you will focus on task execution, independent delivery, or technical leadership, while collaborating across teams to discover Iceberg catalogs, generate metadata, ingest