AI Harness Architect for Multi-Agent Data Platform

NCS; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

NCS is seeking an autonomous AI Consultant to design and build the AI harness, the agent framework, and reusable engineering components for a modern data platform based on Apache Iceberg, Trino, and Parquet.

Your role enables multi-agent collaboration, task execution, state management, and end-to-end data engineering capabilities as you craft specialized agents for data discovery, ingestion, transformation, and governance, with strong emphasis on security and observability.

Qualifications

• Experience with Generative AI, Agentic AI, or LLM-based solutions.
• Proficiency in Python, including FastAPI, Pydantic and Pandas.
• Experience designing and implementing AI Harness or multi-agent frameworks.
• Hands-on experience with LangChain & LangGraph.
• Experience with Apache Iceberg, Trino, and Parquet-based lakehouse architectures.
• Experience building data ingestion, transformation, and orchestration pipelines.
• Strong knowledge of metadata management, data lineage, and data governance practices.
• Experience developing REST APIs, SDKs, and reusable engineering frameworks.
• Experience with observability, monitoring, tracing, and operational support practices.
• Strong understanding of distributed systems, event-driven architectures, and microservices patterns.
• Proven ability to design, develop, and operationalize enterprise-grade AI agents for data discovery, ingestion, transformation, optimization, governance, and observability.
• Familiar with modern data engineering technologies, including DuckDB, dbt, and related engineering tools for data transformation, modelling, and pipeline development.

Responsibilities

• Design and implement the AI Harness framework for agent orchestration, workflow execution, state management, and logging.
• Develop specialized agents for Iceberg catalog discovery, metadata generation, data ingestion, and data quality checks.
• Create Trino SQL, Iceberg table definitions, and governance artifacts.
• Ensure security, access controls, observability, and disaster recovery are built into the framework.
• Collaborate with data engineers to build reusable components and pipelines for end-to-end data workflows.
• Monitor pipelines and agents to detect failures and bottlenecks and generate alerts.

Skills

Generative AI

Python

FastAPI

LangChain

LangGraph

Apache Iceberg

Trino

Parquet

Data governance

Observability

Tools

DuckDB

dbt

REST API development

Job description

NCS is seeking an autonomous AI Consultant to design and build the AI harness, the agent framework, and reusable engineering components for a modern data platform based on Apache Iceberg, Trino, and Parquet.

Your role enables multi-agent collaboration, task execution, state management, and end-to-end data engineering capabilities as you craft specialized agents for data discovery, ingestion, transformation, and governance, with strong emphasis on security and observability.