Remote AI Agent Runtime Engineer - Observability

Lucentive; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Remote work with CET overlap

Occasional travel

Job summary

Lucentive seeks an engineer to build and maintain the runtime that makes AI agent outputs traceable and reviewable across engagements. The role integrates tools, memory, and retrieval to support diverse validation needs and provides observability into how outputs are produced.

You will implement extraction pipelines with confidence scores and source references, handle imperfect inputs, and ship validators and audit trails.

Qualifications

• Experience shipping AI or agent systems into production with strong typed software engineering practices.
• Experience building evaluation or monitoring systems that detect meaningful regressions.
• Ability to handle imperfect real-world inputs such as scans, inconsistent documents, missing fields, or conflicting records.
• Familiarity with end-to-end application stacks; Next.js and Convex are used, though equivalent experience is relevant.
• Care for versioning and reviewing prompts or other inputs to reasoning systems.
• English fluency for engineering collaboration and written review.

Responsibilities

• Develop the agent runtime that brings together tools, memory, and retrieval across differing validation needs.
• Build extraction pipelines that attach confidence information and source references to values, including incomplete inputs.
• Maintain evaluations using reference sets, regression checks, and drift monitoring.
• Implement validators from specifications and record validation results in the audit trail.
• Design provider interfaces that support mock providers during development and production providers without a rewrite.
• Provide observability to let teams understand agent behavior in production.

Job description

Lucentive seeks an engineer to build and maintain the runtime that makes AI agent outputs traceable and reviewable across engagements. The role integrates tools, memory, and retrieval to support diverse validation needs and provides observability into how outputs are produced.

You will implement extraction pipelines with confidence scores and source references, handle imperfect inputs, and ship validators and audit trails.