AI Retrieval & Grounding Engineer: Grounded, Reliable AI

Chatoly Assistant; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Chatoly Assistant is seeking an engineer to own and improve the retrieval and confidence systems that determine what the AI assistant can answer. The role focuses on grounding replies in approved customer content, spotting uncertainty, and reducing unsupported claims through data-backed improvements.

You will build retrieval pipelines, define confidence thresholds for human handoff, and create evaluation datasets drawn from real conversations.

Qualifications

• Production experience shipping LLM features.
• Practical knowledge of embeddings, vector search, chunking, and reranking techniques.
• Strong Python or TypeScript skills and the ability to reason about the complete system.
• Experience building evaluation pipelines and defending quality conclusions with data.

Responsibilities

• Own retrieval pipelines that ground responses in customer content.
• Define confidence scoring and thresholds for handoff to humans.
• Create evaluation harnesses and datasets from real conversations.
• Make hallucinations and other failure modes visible and measurable.
• Balance response quality with latency and operating costs as usage grows.
• Assess system changes using data rather than subjective impressions.

Skills

Python

TypeScript

Data analysis

System design

Evaluation pipelines

Tools

Embeddings

Vector search

Chunking

Reranking

Job description

Chatoly Assistant is seeking an engineer to own and improve the retrieval and confidence systems that determine what the AI assistant can answer. The role focuses on grounding replies in approved customer content, spotting uncertainty, and reducing unsupported claims through data-backed improvements.

You will build retrieval pipelines, define confidence thresholds for human handoff, and create evaluation datasets drawn from real conversations.