Agentic AI Analyst: Design & Validate AI Tool Integrations

Invoca; United States; Remote; USD 94,000 - 134,000; Full time

Benefits offered by this job

Day-One Health Benefits

Mental Wellbeing

Wellness Subsidy

Flexible Time Off

Job summary

Invoca is seeking an Agentic AI Analyst to shape and refine agentic AI within our product organization. You’ll partner with Product, Engineering, Data Science, and Customer Success to evaluate and operationalize agentic use cases that improve customer outcomes.

You’ll focus on integration layers, tools, skills, evaluation harnesses, and simulations to stress-test agents before production, ensuring measurable improvements and reliable performance.

Qualifications

• 3+ years in product, analytics, or technical customer-facing roles in SaaS/enterprise software.
• Hands-on experience with agentic AI systems in production, including multi-step reasoning and tool use.

Responsibilities

• Design and validate agentic integrations (MCP tools) with tool specifications and real-world scenarios.
• Author and maintain reusable agent skills and a library of instructions and workflows.
• Build and run agent evals with golden datasets, rubrics, and decision pipelines.
• Simulate agent behavior in synthetic environments to stress test long-running workflows.
• Own end-to-end agent behavior design from planning to escalation and fallback.

Skills

Agentic AI

MCP tools

Skills authoring

Eval design

Simulation

LangChain

Multi-agent orchestration

Independent initiative

Tools

MCP servers

APIs & enterprise systems

LangChain

LangGraph

Job description

Invoca is seeking an Agentic AI Analyst to shape and refine agentic AI within our product organization. You’ll partner with Product, Engineering, Data Science, and Customer Success to evaluate and operationalize agentic use cases that improve customer outcomes.

You’ll focus on integration layers, tools, skills, evaluation harnesses, and simulations to stress-test agents before production, ensuring measurable improvements and reliable performance.