Remote Agentic AI Analyst: Shape Conversational Revenue

Invoca; San Francisco, Northern (CA, KY); Hybrid; USD 94,000 - 134,000; Full time

Benefits offered by this job

Remote-friendly

Stock options

Health benefits

Professional development

Job summary

Invoca is seeking an Agentic AI Analyst to shape and refine agentic AI applications at scale. You will partner with Product, Eng, Data Science, and Customer Success to identify and operationalize agentic use cases that improve customer outcomes and AI performance.

Responsibilities include designing MCP tool integrations, authoring agent skills, building evals, and running simulations to stress-test agents before production. Remote-first role with SF Bay Area preference.

Qualifications

• Experience with agentic AI systems in production, including multi-step reasoning and tool use.
• Ability to design or integrate tools for AI agents (MCP servers, tool schemas) or connect agents to APIs/enterprise systems.
• Experience designing and running evals for agentic systems with golden data sets and success criteria.
• Experience with simulation/testing for agents (synthetic users, mocked tools, scenario generation).

Responsibilities

• Design evaluation frameworks for agentic behaviors with golden datasets and grading rubrics.
• Build and operate simulation environments to stress-test multi-step agent workflows.
• Define tools agents use, author tool specifications, and validate tool usage.
• Write and maintain reusable agent skills, and manage a versioned library.
• Own end-to-end agent behavior design including planning, state management, escalation, and fallback rules.
• Collaborate with Sales Eng, Customer Success, and customers to extract value-based use cases.

Skills

Agentic AI systems

Tool integration design

Eval design & execution

Simulation & testing

LangChain / LangGraph

Cross-functional collaboration

Independent execution

Communication skills

Tools

MCP servers

Function/tool schemas

APIs & webhooks

LangChain

LangGraph

Job description

Invoca is seeking an Agentic AI Analyst to shape and refine agentic AI applications at scale. You will partner with Product, Eng, Data Science, and Customer Success to identify and operationalize agentic use cases that improve customer outcomes and AI performance.

Responsibilities include designing MCP tool integrations, authoring agent skills, building evals, and running simulations to stress-test agents before production. Remote-first role with SF Bay Area preference.