Remote AI Operations Lead: Build Impactful Workflows

Horizontal Talent; San Francisco (CA); Remote; USD 54,000 - 103,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

Retirement plan

Job summary

Horizontal Talent is seeking an AI Operations Specialist for a fully remote role. You will drive high-impact AI projects by mapping workflows, building prompts and pipelines, and ensuring quality as models and vendors evolve.

You will collaborate with product, Legal and Policy, producing clear labeling guidelines and data for scalable labeling at scale. Strong communication and autonomous execution are essential for success in this role.

Qualifications

• 5-7 years in operations, program management, analytics, consulting or applied AI, with results.
• You use AI daily and have built something with it that moved the needle, e.g., a workflow, automation or analysis.
• You're comfortable with numbers: accuracy, precision and recall; SQL and notebooks help.
• You're curious about how work gets done, including exceptions, and can measure processes.
• You've worked on quality in some form: QA, audits, or guidelines.
• Clear written communication with business leads, PMs and engineers.
• You can work with limited direction; a technical background isn't required.
• We will teach you calibration of LLM judges and you’ll work with a team that has done it many times.

Responsibilities

• Decide which projects will drive the largest business impact and build the case for them.
• Map current processes with the team and design AI-enabled workflows.
• Create AI prompts, pipelines, and monitor quality and drift post-launch.
• Build test sets, measure accuracy, and calibrate LLM judges.
• Help product teams agree on launch criteria for AI features with evidence.
• Translate policy into concrete reviewer criteria with cross-functional partners.
• Collaborate with SMEs to write labeling guidelines and produce human-labeled data.
• Oversee labeling at scale with vendors and verify against metrics.

Skills

Operations experience

AI workflow design

Analytics & metrics

SQL & notebooks

Stakeholder communication

Autonomy

Tools

Python

APIs

SQL

Job description

Horizontal Talent is seeking an AI Operations Specialist for a fully remote role. You will drive high-impact AI projects by mapping workflows, building prompts and pipelines, and ensuring quality as models and vendors evolve.

You will collaborate with product, Legal and Policy, producing clear labeling guidelines and data for scalable labeling at scale. Strong communication and autonomous execution are essential for success in this role.