Remote AI Operations Specialist - Deploy & Optimize

Rainfall Ventures

United States

On-site

USD 120,000 - 180,000

Full time

10 days ago
Application generator

An application made for this job — a tailored resume and cover letter that speak straight to the posting.

Get past ATS filters

Benefits offered by this job

Health insurance
Dental insurance
Vision insurance
401(k) with company match
Flexible PTO

Job summary

Rainfall Ventures is seeking an AI Operations Specialist to own customer AI implementations from discovery through launch and ongoing optimization. You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and work directly with customers and internal teams to ensure reliable deployments in real-world settings.

This is a customer-facing, technically focused IC role requiring comfort moving between AI behavior, implementation details, customer

Qualifications

  • Hands-on experience with LLM-powered products or workflows in production.
  • Experience building evaluations and improving unreliable AI behavior.
  • Strong troubleshooting skills and ability to investigate complex systems without immediate engineering escalation.
  • Experience owning customer implementations from discovery through launch and optimization with customers.
  • Experience with healthcare tech and/or conversational/voice AI is helpful but not required.

Responsibilities

  • Own customer AI implementations from discovery through launch and ongoing optimization.
  • Translate customer workflows and requirements into clear AI behavior, implementation plans, and testing criteria.
  • Design and improve prompts and workflow logic for customer-facing AI agents.
  • Build and run evaluations, test plans, and regression checks to improve reliability and consistency.
  • Investigate production issues across prompts, workflows, integrations, data, APIs, and application behavior.
  • Determine whether problems can be resolved directly or require product or engineering support, and elevate with clear evidence and reproduction steps.
  • Facilitate customer testing and troubleshooting, and communicate status, risks, blockers, and required decisions clearly.
  • Drive open issues to resolution across customers and internal teams, including when ownership or next steps are ambiguous.
  • Use recurring customer and production issues to improve implementation practices and product behavior.

Skills

LLM workflows in production
Troubleshooting
Customer implementations
Production issues investigation
Healthcare/AI familiarity

Job description

Rainfall Ventures is seeking an AI Operations Specialist to own customer AI implementations from discovery through launch and ongoing optimization. You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and work directly with customers and internal teams to ensure reliable deployments in real-world settings.

This is a customer-facing, technically focused IC role requiring comfort moving between AI behavior, implementation details, customer

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Ops Specialist: Launch & Optimize LLMs
Remote AI Ops Specialist: Launch & Optimize LLMs

Health Note, Inc. • Northern (KY)

Hybrid
USD 120,000 - 160,000
Health, dental, and vision insurance
401(k) with company match
Flexible PTO and company-paid holidays
+2
AI Operations & Enablement Lead
AI Operations & Enablement Lead

Rain • New York (NY)

Hybrid
USD 150,000 - 220,000
Unlimited time off
Flexible working
Health and wellness stipend
+4
AI Operations Specialist
AI Operations Specialist

Health Note, Inc. • Northern (KY)

Hybrid
USD 120,000 - 160,000
Health, dental, and vision insurance
401(k) with company match
Flexible PTO and company-paid holidays
+2
Remote AI Inference & Post-Training Specialist
Remote AI Inference & Post-Training Specialist

Togetherai • San Francisco (CA)

On-site
USD 270,000 - 300,000
Startup equity
Health insurance
Remote work flexibility
Customer Engineering Lead — AI-Driven Partner Solutions
Customer Engineering Lead — AI-Driven Partner Solutions

Rain • New York (NY)

Hybrid
USD 200,000 - 280,000
Unlimited time off
Flexible work location
Health benefits
+4
AI Operator Deployment Lead
AI Operator Deployment Lead

Eloquent AI, Inc. • San Francisco (CA), Northern (KY)

Hybrid
USD 130,000 - 170,000
Senior AI Deployment Engineer — Enterprise Solutions
Senior AI Deployment Engineer — Enterprise Solutions

Optimizely • New York (NY)

On-site
USD 103,000 - 175,000
Senior AI Deployment Engineer
Senior AI Deployment Engineer

Worky • New York (NY)

On-site
USD 103,000 - 175,000
Restaurant d'entreprise
Remote CRM Ops Specialist | Optimize Sales & Automation
Remote CRM Ops Specialist | Optimize Sales & Automation

YO AI Labs • Illinois

Remote
USD 69,000 - 117,000
Client-Facing AI Deployment Engineer
Client-Facing AI Deployment Engineer

CloudFactory Limited • Northern (KY)

Hybrid
USD 120,000 - 180,000