AI Engineer

BASIL TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 180,000 - 280,000; Full time

Job summary

BASIL TECHNOLOGIES PTE. LTD. in Singapore seeks an experienced software engineer for production-grade agentic and GenAI systems end-to-end.

You will ship services with robust APIs, reusable components, and secure pipelines that connect LLMs, tools, knowledge, and enterprise systems. The role blends software engineering with modern AI practices (RAG, agent orchestration, policy chains, evals) to deliver measurable business outcomes at scale.

Qualifications

• 6+ years in software engineering on production services, not just prototypes.
• Proficient in Python or TypeScript/Node.js and a second language.
• Hands-on with containers, Kubernetes, CI/CD, and cloud platforms.
• Practical LLM experience: building RAG/agent apps and safety patterns.
• Data design and batch/stream pipelines; SQL and vector DB knowledge.
• Security basics: secrets, policies, IAM, and observability.

Responsibilities

• Design multi-agent systems with planning, tool-use, and delegation; expose via REST/gRPC APIs.
• Implement tool adapters and model gateway integrations with safe sandboxes.
• Stand up RAG services and ingestion pipelines for docs, tickets, and ERP/CRM data.
• Build evaluation harnesses and wire quality metrics into CI.
• Package services as containers; deploy to Kubernetes with Helm/Argo CD; monitor costs.
• Collaborate with Product/Data/Platform to define SLAs/SLOs and governance.
• Mentor engineers on AI-as-software practices and run design reviews.

Skills

Python

TypeScript/Node.js

Go/Java

Kubernetes

CI/CD

LLM safety

SQL

Vector DB

Education

Bachelor's degree in Computer Science

Tools

Docker

Terraform

OpenTelemetry

Vault

Airflow/Prefect

FastAPI

REST/gRPC

GitHub Actions

Argo CD

Helm

Job description

What will you do:

Design, build, and operate production-grade agentic and GenAI systems-end to end. You'll ship services (not just notebooks): robust APIs, reusable components, and secure pipelines that connect LLMs, tools, knowledge, and enterprise systems. You'll pair strong software engineering with modern AI practices (RAG, agent orchestration, policy chains, evals) to deliver measurable business outcomes at scale.

1. Agent & Application Engineering

• Design multi-agent systems (MAS) with planning, tool-use, and delegation (e.g., LangGraph/Semantic Kernel); expose them via REST/gRPC APIs (FastAPI/Express/Java/Go).
• Implement tool adapters (SQL, search, document stores, web calls, code exec) with strict type contracts and safe sandboxes.
• Build model gateway integrations (OpenAI/Azure OpenAI/Bedrock/Vertex; self-hosted vLLM/TGI) with routing, rate-limits, retries, and fallback chains.

2. Retrieval, Data & Knowledge

• Stand up RAG services: chunking, enrichment, embeddings, indexing, hybrid/vector search (pgvector/Pinecone/Weaviate; OpenSearch/Azure AI Search).
• Implement ingestion pipelines (Airflow/Prefect/Celery/Ray) for docs, tickets, chat, and ERP/CRM data; handle PII redaction and metadata governance.
• Optimize retrieval quality (chunking strategies, re-rankers, query rewriting) with offline/online evaluation and A/B tests.

3. Quality, Testing & Evaluation

• Treat prompts and graphs as code: version, diff, and test them (unit tests for prompts/tools; golden sets; regression suites)
• Build evaluation harnesses (latency, cost, accuracy, toxicity, hallucination, guardrail hit-rates); wire into CI.
• Add drift detection for conversational systems; implement safe shutdown and auto-rollback.

4. Platform & Operations

• Package services as containers; deploy to Kubernetes with Helm/Argo CD; configure autoscaling, HPA/VPA, and resource quotas.
• Implement policy chains and guardrails (OPA/Gatekeeper for policy, Presidio for PII, Trivy for image scanning).
• Instrument deep observability: tracing (OpenTelemetry), metrics (Prometheus), logs (ELK/OPENSEARCH), cost meters per request/model.

5. Security & Compliance

• Manage secrets (HashiCorp Vault/KMS), signed images, SBOMs; enforce least-privilege IAM.
• Build tenant isolation and data residency controls; implement red/blue team prompts and jailbreak defenses.

6. Integration & Enterprise Workflows

• Ship connectors and events for SAP/CRM/ITSM and Kafka topics; design idempotent, retry-safe processors.
• Automate business workflows with pro-code services first; expose low-code surfaces only where appropriate.

7. Collaboration & Leadership

• Partner with Product, Data, and Platform teams to define SLAs/SLOs and success metrics.
• Mentor engineers on \"AI as software\" practices; run design reviews and postmortems.

Qualifications

• 6+ years in software engineering (prod services, not just prototypes), including 1+ year leading small projects.
• Strong in one systems language (Python/TypeScript/Go/Java) and comfortable in a second.
• Hands‑on with containers, Kubernetes, CI/CD (GitHub Actions/GitLab/Jenkins), IaC (Terraform), and cloud (Azure/AWS/GCP).
• Practical LLM experience: building RAG/agent apps, prompt design, tool-use, and safety patterns.
• Data skills: designing schemas, batch/stream pipelines, and search indexes; proficiency with SQL and one vector DB.
• Testing mindset: unit/integration tests, load tests, golden datasets for LLM evals.
• Security basics: secrets, policies, scanning, and least-privilege IAM.

Preferred Qualifications

• Agent orchestration (LangGraph, Semantic Kernel) and distributed compute (Ray) in production.
• Search/retrieval tuning (BM25 + vector hybrid, re-ranking, query planning).
• Observability at scale with OpenTelemetry; cost/perf optimization across model/router layers.
• Experience in regulated or high-throughput domains (e.g., telco, finance, healthcare); multi-tenant and data-residency patterns.
• Domain integrations (SAP/CRM/ITSM), event-driven architectures (Kafka/Debezium), and policy enforcement (OPA/Gatekeeper).
• Familiarity with TM Forum APIs / BSS-OSS patterns is a plus (if in telco context).

Tech Stack (Illustrative)

• Languages: Python, TypeScript/Node.js (plus Go/Java bonus)
• Frameworks: FastAPI/Express, LangGraph/Semantic Kernel, Ray/Celery, Airflow/Prefect
• Storage/Search: Postgres, Redis, S3/Blob; pgvector/Pinecone/Weaviate; OpenSearch/Azure AI Search
• LLM Runtime: OpenAI/Azure OpenAI/Bedrock/Vertex; vLLM/TGI; inference routers/gateways
• Platform: Docker, Kubernetes, Helm, Argo CD, Terraform, Vault, Istio
• Observability: OpenTelemetry, Prometheus/Grafana, ELK/OpenSearch
• Quality & Safety: pytest/Jest, prompt/unit test harnesses, guardrails, Presidio, Trivy