Senior AI Observability Engineer (Remote)

OnBoard; United States; On-site; USD 180,000 - 230,000; Full time

Benefits offered by this job

Fully remote work with equipment

US-based employees

Comprehensive medical plans

401k with match

PTO and holidays

Job summary

OnBoard is a board intelligence platform trusted by over 6,000 organizations to simplify governance, and we are building AI-powered product experiences across RAG pipelines, semantic search, summarization, and emerging agentic workflows. We are looking for a Senior AI Engineer to help make those experiences reliable, measurable, cost-effective, and safe in production.

This is a hands-on engineering role for someone who wants to build AI systems, not just observe them.

Qualifications

• 5+ years of software engineering experience building production systems.
• Hands-on experience building or operating LLM-powered features, RAG systems, AI workflows, or similar AI applications.
• Strong engineering ability in Python, C#/.NET, or both.
• Practical understanding of prompts, embeddings, vector search, retrieval quality, orchestration patterns, and LLM application architecture.
• Experience designing evaluations, quality metrics, or regression frameworks for AI or software systems.

Responsibilities

• Build and improve AI-powered product systems using LLMs, RAG, semantic search, and agentic workflows.
• Contribute to production codebases in Python, C#/.NET, or related technologies.
• Improve prompt, retrieval, context assembly, ranking, grounding, and response-generation patterns.
• Own observability, reliability, and cost signals; instrument AI interactions and pipelines.
• Define evaluation pipelines and scoring methodologies for AI features and datasets.

Skills

5+ years experience

Python

C#/.NET

LLM applications

Observability

Production systems

Tools

Arize

Langfuse

LangSmith

Weights & Billing (W&B)

OpenTelemetry

Azure Monitor

Job description

OnBoard is a board intelligence platform trusted by over 6,000 organizations to simplify governance, and we are building AI-powered product experiences across RAG pipelines, semantic search, summarization, and emerging agentic workflows. We are looking for a Senior AI Engineer to help make those experiences reliable, measurable, cost-effective, and safe in production.

This is a hands-on engineering role for someone who wants to build AI systems, not just observe them.