AI Prompt & Agent Developer

ProLegion; San Francisco (CA); On-site; USD 120,000 - 190,000; Full time

Job summary

ProLegion is hiring an AI Prompt & Agent Developer to own behavioral slices of our voice agents, from shared behavior to deployment-specific nuances. You’ll write prompts, design subagent architectures, build evals, and push automation gains one measurable win at a time, informed by real call data.

You’ll work with dashboards, call-review tools, and automated agents to accelerate improvements and ensure robust deployment outcomes across customer calls.

Qualifications

• Shipped prompts that broke production and learned from it.
• Meticulous, data-driven, and highly organized under pressure.
• Strong writing and communication skills for prompts and responses.
• Analytical and experimental mindset with evidence-based changes.
• Comfort with Python and TypeScript for tooling and scripting.

Responsibilities

• Write and maintain the prompts that run in production across deployments.
• Ship iteratively against real call data, using dashboards and review tooling.
• Build evaluation harnesses, test suites, and offline eval sets.
• Onboard new customers with agent configurations and workflows.
• Simulate scenarios, measure outcomes, and monitor performance.

Skills

Prompt engineering

Data-driven decision making

Analytical mindset

Strong writing

Problem solving

Tools

Python

TypeScript

Job description

We're hiring an AI Prompt & Agent Developer to own behavioral slice(s) of our voice agents. That behavior splits into two categories: behavior shared across every deployment, and behavior specific to a subset of deployments. You're someone who actually enjoys looking at the data, because the data informs everything else. You'll write prompts, design subagent architectures, build evals, and push automation rates up one small, measurable win at a time.

Responsibilities

• Write and maintain the prompts that run in production. Intent classification, information extraction, availability negotiation, closing phrases, insurance verification flows, objection handling, edge-case recovery. You own behavior that touches every customer call.
• Ship iteratively against real call data. Every morning, you'll listen to failed calls from yesterday. Every afternoon, you'll deploy a fix. You'll be using and helping to develop dashboards, call review tooling, and automated agents to accelerate the work.
• Build evaluation harnesses. You'll develop offline eval sets, run automated prompt optimization (we use GEPA-style approaches), and establish the test suites that let us ship changes without breaking live deployments.
• Human-in-the-loop onboarding. New customers come online constantly. You'll work with and iterate on our internal AI agents that translate a practice's intake form, their scheduling rules, and their quirks into an agent configuration. Every week, you'll be designing new evaluation metrics for these customers and helping to improve existing ones.
• QA and continuous improvement. You'll simulate real-world customer scenarios, measure outcomes, and monitor production agent performance so you can catch drift early and fix it fast.

What we're looking for

• You've shipped prompts that broke production. Doesn't matter if it was at OpenAI, a chatbot startup, a research lab, or your own project. What matters is that you've felt the specific pain of a prompt that worked beautifully in dev and broke the second it hit real users.
• You're meticulous and careful. Looking at data for long stretches energizes you, as long as there's a signal. You stay organized when five things are in flight. We deploy multiple times a day, and we also run healthcare workflows where a bad change costs real money for real practices. You know the difference between moving fast and breaking things.
• Writing sensibility. The best prompt engineers are good writers. You notice register, rhythm, and word choice. You can tell why "Hello, cornerside dental? This is Ava, how can I help you out today?" sounds warmer than "Hello, Cornerside Dental, this is Ava. How can I help you out today" out of a TTS.
• Analytical and empirical. You are relentlessly data-driven. Before you make changes, you proactively run experiments and measure. You don't ship because "I think this is better." You justify a change with "this moved booking rate from 78.2% to 81.4% on n=412 calls."
• Comfort with code. You don't need to be a senior engineer, but you should read Python fluently and TypeScript comfortably, and you can get almost any coding task done by pairing with modern AI coding tools.