Remote Agentic AI Architect

Invoca; Denver (CO); Remote; USD 94,000 - 134,000; Full time

Benefits offered by this job

Health benefits

401(k) plan

Wellness subsidy

Paid time off

Job summary

Invoca is seeking an Agentic AI Analyst to shape agentic AI applications in our product organization. You will collaborate with product managers, engineers, data scientists, and customer success to identify and operationalize agentic use cases that improve customer outcomes.

You will design and validate the integration layer for agents, author skills, build evaluation harnesses, and run simulations to stress-test agents before production.

Qualifications

• 3+ years of experience in product, analytics, or technical customer-facing roles within SaaS or enterprise software.
• Hands-on experience with agentic AI systems in production environments, including multi-step reasoning and tool use.
• Experience designing or integrating tools for AI agents, such as MCP servers, function/tool schemas, or API integrations.
• Experience with agent design and architecture, prompts, prompts, and iteration based on results.
• Experience designing and running evals for Agentic systems with golden data sets and success criteria.
• Experience with simulation or synthetic testing approaches for agents (e.g., synthetic users, mocked tools).
• Experience with AI orchestration frameworks (e.g., LangChain, LangGraph).
• Demonstrated ability to work independently on medium-scope initiatives and balance tradeoffs.

Responsibilities

• Design and validate agentic integrations (MCP tools) with tool specifications and MCP server prototypes.
• Author and maintain reusable agent skills and workflows.
• Build and run agent evals with golden datasets and success criteria.
• Simulate agent behavior using synthetic callers and mock backends.
• Design end-to-end agent behavior including planning, state management, and escalation.
• Validate use cases with customers and GTM, analyzing transcripts and data.
• Monitor production metrics like task completion and tool-call success to guide improvements.
• Tune retrieval, context management, and model selection with production data.
• Triage production issues, capture reproducible steps, and land fixes.
• Document agent designs, tool specifications, and best practices for reuse across teams.
• Enable cross-functional teams by creating collateral and training on agentic AI features.
• Design multi-agent and long-running workflows with stateful sessions.

Skills

3+ years SaaS

Agentic AI systems

Tool design & integration

Eval design

Simulation testing

LangChain/LangGraph

Independent work

Cross-functional collaboration

Strong communication

Tools

MCP servers

APIs integration

LangChain

LangGraph

Job description

Invoca is seeking an Agentic AI Analyst to shape agentic AI applications in our product organization. You will collaborate with product managers, engineers, data scientists, and customer success to identify and operationalize agentic use cases that improve customer outcomes.

You will design and validate the integration layer for agents, author skills, build evaluation harnesses, and run simulations to stress-test agents before production.