Senior AI/ML Engineer

Vertisystem (A MOURI Tech Company); United States; On-site; USD 138,000 - 141,000; Full time

Job summary

Vertisystem (A MOURI Tech Company) in the United States is seeking a Senior AI/ML Engineer to architect high‑throughput LLM agent orchestration and MCP integration for tax and compliance products. You will design asynchronous backends using Python, asyncio and FastAPI, implement evaluation benchmarks, gating, and multi‑tenant IAM‑DS entitlement for reliable, audit‑defensible execution.

Join a production‑oriented team focused on performance, observability with Datadog and OpenTelemetry, and

Qualifications

• 5+ years of software engineering experience in Python with high-throughput asynchronous services.
• 2+ years of hands-on experience building production LLM applications, agentic workflows, function calling, tool use, and multi-agent interaction protocols.
• Solid understanding of Model Context Protocol (MCP), RESTful API design, Agent-to-Agent orchestration, and Knowledge Graph/RAG patterns.

Responsibilities

• Agent Orchestration & Backend Development: Design, implement, and optimize multi-agent orchestration workflows (A2A), prompt templating, dynamic intent routing, and conversation memory using Python, asyncio, and FastAPI.
• MCP & Tool Integration: Standardize and integrate MCP servers and tools across Client’s products (AvaTax, Returns, Tax Research, ECM, Licensing).
• Evaluations & Benchmarking: Build automated evaluation pipelines (LLM-as-a-judge, deterministic assertions, harness evals) to benchmark agent intent accuracy, routing precision (HSR/FSR), and execution latency.
• Gating & Entitlement Routing: Implement dynamic cohort-based feature gating and IAM-DS entitlement evaluation to support controlled beta rollouts and multi-tenant access controls.
• Performance & Reliability: Optimize token consumption, response streaming, context caching, rate-limiting, and error handling for high-concurrency agent interactions.
• Production Readiness & Observability: Build telemetry pipelines, structured logging, and monitor dashboards across Datadog, Snowflake, and Open Telemetry to track model accuracy and system health.

Skills

Python

Asyncio

FastAPI

LLM

Agent orchestration

LangChain

Tool use

Tools

AWS ECS/EKS

Lambda

SQS/Kafka

Snowflake

Vector DBs

CI/CD

Job description

Duration: 12 - Months Contract with Possible Extension

Pay Range: $100- $102 Per Hour on W2

Job Brief:

Client’s AI/ML team is building and scaling the Product - the unified AI agent hub for tax and compliance, and the Avalara Avi Agent (AAA) orchestration backbone. We are seeking Senior AI/ML Engineers with deep hands-on expertise in LLM agent orchestration, Model Context Protocol (MCP), and evaluation pipelines. In this role, you will develop high-performance asynchronous orchestration services, integrate domain-specific MCP tool suites across Client's tax products, build evaluation benchmarks, and implement cohort-based gating to ensure reliable, audit-defensible execution in high-concurrency production environments.

Responsibilities:

• Agent Orchestration & Backend Development: Design, implement, and optimize multi-agent orchestration workflows (A2A), prompt templating, dynamic intent routing, and conversation memory using Python, `asyncio`, and FastAPI.
• MCP & Tool Integration:Standardize and integrate Model Context Protocol (MCP) servers and tools across Client’s products (AvaTax, Returns, Tax Research, ECM, Licensing).
• Evaluations & Benchmarking:Build automated evaluation pipelines (LLM-as-a-judge, deterministic assertions, harness evals) to benchmark agent intent accuracy, routing precision (HSR/FSR), and execution latency.
• Gating & Entitlement Routing:Implement dynamic cohort-based feature gating and IAM-DS entitlement evaluation to support controlled beta rollouts and multi-tenant access controls.
• Performance & Reliability: Optimize token consumption, response streaming, context caching, rate-limiting, and error handling for high-concurrency agent interactions.
• Production Readiness & Observability:Build telemetry pipelines, structured logging, and monitor dashboards across Datadog, Snowflake, and Open Telemetry to track model accuracy and system health.

Required Qualifications & Technical Skills

• Professional Experience: 5+ years of software engineering experience in Python with strong expertise in high-throughput asynchronous services (`asyncio`, FastAPI).
• LLM & Agent Systems: 2+ years of hands-on experience building production LLM applications, agentic workflows, function calling, tool use, and multi-agent interaction protocols (e.g., LangChain, Lang Graph, LlamaIndex, or custom frameworks).
• Protocols & Architectures: Solid understanding of Model Context Protocol (MCP), RESTful API design, Agent-to-Agent (A2A) orchestration, and Knowledge Graph/RAG patterns.
• Evaluation Methodologies: Demonstrated experience establishing automated LLM evaluation datasets, synthetic test benches, and scoring metrics for intent detection and tool selection.
• Cloud & Data Stack: Experience with AWS cloud infrastructure (ECS/EKS, Lambda, SQS/Kafka), vector databases, and enterprise data warehouses (Snowflake).
• Software Craftsmanship: Strong commitment to unit/integration testing, CI/CD automation, clear API documentation, and agile team collaboration.

Preferred Experience

• Prior experience working with enterprise SaaS, tax/compliance technology, or complex multi-step workflow automation.
• Hands-on experience with streaming architecture, semantic caching, and LLM latency optimization.
• Familiarity with IAM/RBAC entitlement models and enterprise data security standards (SOC 2, ISO).