HR Expert – AI Evaluation & Task Development Specialist
Position: HR Expert – AI Evaluation & Task Development Specialist
Type: Hourly Contract
Compensation: $95/hour
Location: Remote – US Preferred, Open Worldwide
About the Opportunity
An opportunity for experienced Human Resources professionals to contribute their expertise to the development and evaluation of advanced AI agents.
This role involves creating, refining, and reviewing realistic HR evaluation tasks based on simulated workplace environments. These scenarios include company policies, employee records, case files, emails, and HR systems that reflect the day-to-day responsibilities of professional HR teams.
You will help ensure that AI-generated tasks reflect real workplace situations, reference answers are accurate, and evaluation rubrics meet the standards expected of experienced HR professionals.
Key Responsibilities
- Refine AI-generated task prompts to ensure they reflect realistic requests from HR managers and directors without revealing the expected answers.
- Review reference answers against source documents, employee records, company policies, and HR system data to identify and correct inaccuracies.
- Develop and improve grading rubrics with clear pass/fail criteria, exact values, and professional judgment standards.
- Evaluate AI-generated responses and determine whether model failures are based on fair, accurate, and defensible criteria.
- Create and review HR scenarios across multiple functional areas, including talent acquisition, onboarding, performance management, succession planning, compensation, benefits, and leave administration.
- Evaluate tasks involving employee relations, workplace investigations, employee separations, HR policies, and employment-law compliance.
- Contribute expertise in people analytics, workforce planning, and organizational design.
- Maintain accuracy, consistency, and practical relevance across HR evaluation tasks.
Required Qualifications
- 3+ years of professional HR experience in one or more human resources functions.
- Bachelor's degree in Human Resources, Business Administration, or a related discipline.
- Practical experience owning or managing talent programs, employee relations matters, compensation design, or other core HR functions.
- Working knowledge of employment laws and regulations, including Title VII, FLSA, ADA, or equivalent UK and EU employment legislation.
- Ability to review complex HR documentation, identify inconsistencies, and develop accurate reference answers.
- Strong analytical skills and professional judgment when evaluating HR decisions and workplace scenarios.
- Excellent written communication skills and attention to detail.
Preferred Qualifications
- SHRM-SCP, SPHR, or an internationally recognized equivalent HR certification.
- Experience in HR leadership or senior HR management roles.
- Background working at large organizations or companies with established HR processes.
- Previous experience contributing to AI training, evaluation, or reinforcement learning projects.
- Prior experience working on AI evaluation projects is an advantage.
Areas of Expertise
- Talent Acquisition & Employee Onboarding
- Performance Management & Succession Planning
- Compensation & Benefits Administration
- Leave Management
- Employee Relations & Workplace Investigations
- Employee Separations & Offboarding
- HR Policy & Employment-Law Compliance
- People Analytics & Organizational Design
- Engagement Type: Hourly contract.
- Work Arrangement: Fully remote.
- Location Eligibility: United States preferred; qualified professionals worldwide may apply.
- Compensation: $95 per hour.
- Onboarding: Short onboarding session followed by a paid skills assessment.
- Growth Opportunities: Strong performers may progress into reviewer and world lead positions.