Remote AI Ops Specialist: Launch & Optimize Customer AI

Health Note, Inc.; United States; Remote; USD 100,000 - 150,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401(k) with company match

Flexible PTO

Job summary

Health Note is seeking an AI Operations Specialist to own customer AI implementations from discovery to launch and continuous optimization. You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and collaborate with customers and internal teams to ensure reliable real-world results.

This hands-on, customer-facing IC role requires translating workflows into AI behavior, designing prompts, building evaluations, and guiding production

Qualifications

• Hands-on experience with LLM-powered products or workflows in production.
• Experience building evaluations to improve unreliable AI behavior.
• Strong technical troubleshooting skills for complex systems.
• Experience owning customer implementations from discovery through launch and optimization.
• Healthcare technology familiarity or conversational/voice AI is helpful but not required.

Responsibilities

• Own customer AI implementations from discovery through launch and ongoing optimization.
• Translate customer workflows and requirements into clear AI behavior, implementation plans, and testing criteria.
• Design and improve prompts and workflow logic for customer-facing AI agents.
• Build and run evaluations, test plans, and regression checks to improve reliability and consistency.
• Investigate production issues across prompts, workflows, integrations, data, APIs, and application behavior.
• Determine whether problems can be resolved directly or require product or engineering support, and upscale with clear evidence and reproduction steps.
• Facilitate customer testing and troubleshooting, and communicate status, risks, blockers, and required decisions clearly.
• Drive open issues to resolution across customers and internal teams, including when ownership or next steps are ambiguous.
• Use recurring customer and production issues to improve implementation practices and product behavior.

Skills

LLM workflows

Evaluation design

Troubleshooting

Customer implementations

Healthcare tech familiarity

Tools

APIs

Job description

Health Note is seeking an AI Operations Specialist to own customer AI implementations from discovery to launch and continuous optimization. You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and collaborate with customers and internal teams to ensure reliable real-world results.

This hands-on, customer-facing IC role requires translating workflows into AI behavior, designing prompts, building evaluations, and guiding production