AI Harness Architect - Senior Data Engineer

NCS PTE. LTD.; Singapore; On-site; SGD 120,000 - 160,000; Full time

Job summary

NCS PTE. LTD. is seeking an autonomous AI Engineer to design and build the AI Harness for a modern data platform based on Apache Iceberg, Trino, and Parquet.

You will focus on creating the agent framework to enable multi-agent collaboration, stateful workflows, and end-to-end data engineering capabilities. Responsibilities include developing specialized agents to discover Iceberg catalogs, profile datasets, generate metadata, ingest data from diverse sources, and optimize data structures.

Qualifications

• Experience with Generative AI, Agentic AI, or LLM-based solutions.
• Proficiency in Python, including FastAPI, Pydantic and Pandas.
• Experience designing and implementing AI Harness or multi-agent frameworks.
• Hands-on experience with LangChain & LangGraph.
• Experience with Apache Iceberg, Trino, and Parquet-based lakehouse architectures.
• Experience building data ingestion, transformation, and orchestration pipelines.
• Strong knowledge of metadata management, data lineage, and data governance practices.
• Experience developing REST APIs, SDKs, and reusable engineering frameworks.
• Experience with observability, monitoring, tracing, and operational support practices.
• Strong understanding of distributed systems, event-driven architectures, and microservices patterns.
• Proven ability to design, develop, and operationalize enterprise-grade AI agents for data discovery, ingestion, transformation, optimization, governance, and observability.
• Familiar with modern data engineering technologies, including DuckDB, dbt, and related engineering tools for data transformation, modelling, and pipeline development.
• Senior engineers will be expected to have demonstrated evidence of leading technical designs, guiding junior engineering team members, and translating AI/data requirements into scalable, secure, and deployable solutions.

Responsibilities

• Design and implement the foundational agent framework that provides: agent orchestration and coordination, workflow execution and state management, memory and context handling, tool and function calling, agent-to-agent communication, human approval workflows, logging and audit trails, error handling and recovery, security and access controls, observability and monitoring.

Skills

Python

FastAPI

Pydantic

Pandas

LangChain

LangGraph

APIs

Observability

Distributed systems

Data orchestration

Tools

Apache Iceberg

Trino

Parquet

DuckDB

dbt

Job description

NCS PTE. LTD. is seeking an autonomous AI Engineer to design and build the AI Harness for a modern data platform based on Apache Iceberg, Trino, and Parquet.

You will focus on creating the agent framework to enable multi-agent collaboration, stateful workflows, and end-to-end data engineering capabilities. Responsibilities include developing specialized agents to discover Iceberg catalogs, profile datasets, generate metadata, ingest data from diverse sources, and optimize data structures.