Senior AI Engineer: Observability & Quality (Remote)

Uncover; Northern (KY); Hybrid; USD 150,000 - 210,000; Full time

Benefits offered by this job

Fully remote work

High-quality medical plans

401K with company match

PTO and holidays

Life insurance

Job summary

OnBoard is seeking a Senior AI Engineer to build reliable, cost‑effective AI features across RAG pipelines and agentic workflows. You will design evaluation pipelines, instrument production signals, and partner with product teams to ship high‑quality AI experiences.

You will own observability, latency, and costs, defining what good looks like and how we measure it in production. A strong software background and clear communication are essential.

Qualifications

• 5+ years of software engineering experience building production systems.
• Hands-on experience building or operating LLM-powered features, RAG, AI workflows, or similar AI applications.
• Strong programming ability in Python, C#/.NET, or both.
• Experience designing evaluations, quality metrics, or regression frameworks for AI or software systems.
• Strong observability fundamentals: tracing, logging, metrics, alerting, and production debugging.

Responsibilities

• Design and implement AI features using LLMs, RAG, semantic search, and agentic workflows.
• Contribute to production codebases in Python, C#/.NET, or related technologies.
• Improve prompts, retrieval, context assembly, ranking, grounding, and response-generation patterns.
• Help teams evaluate models, costs, latency, and safety considerations.
• Establish quality gates, testing, and evaluation pipelines for AI features.
• Scale AI quality across teams with reusable libraries and templates.
• Document standards for tracing, metadata, and incident response.

Skills

LLM apps

Python

C#/.NET

AI observability

CI/CD

AzureDevOps

Stakeholder comms

Product judgment

AI tools

Tools

Azure DevOps

Langfuse

LangSmith

Arize

OpenTelemetry

Job description

OnBoard is seeking a Senior AI Engineer to build reliable, cost‑effective AI features across RAG pipelines and agentic workflows. You will design evaluation pipelines, instrument production signals, and partner with product teams to ship high‑quality AI experiences.

You will own observability, latency, and costs, defining what good looks like and how we measure it in production. A strong software background and clear communication are essential.