Policy & Data Quality Specialist — AI QA (Contract)

Cypress HCM; San Francisco (CA); Hybrid; USD 83,000 - 110,000; Full time

Job summary

Cypress HCM is seeking a Policy & Data Quality Specialist on a contract basis in San Francisco. The role focuses on training AI models, ensuring decisions adhere to policies, creating golden sets, calibrating vendor sites, and managing vendor relationships.

Experience in policy analysis and quality assurance is valued; content moderation background helps but is not required. This hybrid role demands strong analytical skills and the ability to collaborate with policy and engineering teams.

Qualifications

• 4+ years of experience in technology consulting, content moderation, quality assurance, or related fields.
• Strong understanding of policy development and moderation guidelines.
• Experience developing evals and golden sets.
• Familiarity with data analysis and reporting tools.
• Analytical, detail-oriented, and able to work independently.

Responsibilities

• Policy & Data Quality Assurance: Review and evaluate decisions made by expert models and vendors to ensure accuracy, consistency, and adherence to policies and guidelines.
• Golden Set Creation: Develop and maintain golden sets of content to test and calibrate expert models and vendor performance, ensuring that they meet our quality standards.
• Vendor Site Calibration: Collaborate with vendor sites to identify areas for improvement, provide feedback, and implement calibration plans to enhance their performance.
• Vendor Management: Build and maintain strong relationships with vendors, providing guidance, support, and performance metrics to ensure they meet our quality expectations.
• Process and Policy Improvement: Identify opportunities to improve policies and processes, workflows, and tools, and collaborate with internal stakeholders to implement changes.
• Data Analysis: Analyze data and metrics to identify trends, patterns, and insights that inform quality assurance strategies and vendor performance.
• Communication: Communicate effectively with internal stakeholders, including operations teams, policy teams, engineering teams and leadership, to ensure alignment and collaboration on quality assurance initiatives.
• Training and Development: Participate in training and development programs to stay up-to-date on policies, procedures, and industry best practices.

Skills

Policy analysis

Data quality

Vendor management

Communication skills

Attention to detail

Job description

Cypress HCM is seeking a Policy & Data Quality Specialist on a contract basis in San Francisco. The role focuses on training AI models, ensuring decisions adhere to policies, creating golden sets, calibrating vendor sites, and managing vendor relationships.

Experience in policy analysis and quality assurance is valued; content moderation background helps but is not required. This hybrid role demands strong analytical skills and the ability to collaborate with policy and engineering teams.