Contingent Policy & Data Quality Specialist

OpenKrill; United States; Hybrid; USD 55,000 - 96,000; Part time

Job summary

OpenKrill is seeking a Policy & Data Quality Specialist as a Contingent Worker to train AI models, ensure decision accuracy, and calibrate vendor performance. The role requires rigorous policy knowledge, data analysis skills, and strong collaboration with internal teams.

The position involves exposure to potentially objectionable content and requires the ability to work in a fast-paced environment with multiple priorities. Hybrid work in an office setting is offered.

Qualifications

• 4+ years of experience in technology consulting, content moderation, quality assurance, or a related field.
• Strong understanding of policy development, moderation procedures, and guidelines.
• Experience developing evals and golden sets.
• Familiarity with data analysis and reporting tools preferred.
• Strong attention to detail and ability to maintain accuracy in a fast-paced environment.
• Excellent analytical, problem-solving, and communication skills.
• Ability to work independently and collaboratively as part of a team.
• Experience working with vendors or external partners is valuable.

Responsibilities

• Policy & Data Quality Assurance: Review and evaluate decisions made by expert models and vendors to ensure accuracy, consistency, and adherence to policies and guidelines.
• Golden Set Creation: Develop and maintain golden sets of content to test and calibrate expert models and vendor performance, ensuring that they meet our quality standards.
• Vendor Site Calibration: Collaborate with vendor sites to identify areas for improvement, provide feedback, and implement calibration plans to enhance their performance.
• Vendor Management: Build and maintain strong relationships with vendors, providing guidance, support, and performance metrics to ensure they meet our quality expectations.
• Process and Policy Improvement: Identify opportunities to improve policies and processes, workflows, and tools, and collaborate with internal stakeholders to implement changes.
• Data Analysis: Analyze data and metrics to identify trends, patterns, and insights that inform quality assurance strategies and vendor performance.
• Communication: Communicate effectively with internal stakeholders, including operations teams, policy teams, engineering teams and leadership, to ensure alignment and collaboration on quality assurance initiatives.
• Training and Development: Participate in training and development programs to stay up-to-date on policies, procedures, and industry best practices.

Skills

Policy analysis

Quality assurance

Data analysis

Communication

Vendor management

Independent & collaborative work

Job description

OpenKrill is seeking a Policy & Data Quality Specialist as a Contingent Worker to train AI models, ensure decision accuracy, and calibrate vendor performance. The role requires rigorous policy knowledge, data analysis skills, and strong collaboration with internal teams.

The position involves exposure to potentially objectionable content and requires the ability to work in a fast-paced environment with multiple priorities. Hybrid work in an office setting is offered.