Web Developer

AgileGrid Solutions; United States; Remote; USD 52,000 - 93,000; Part time

Benefits offered by this job

Remote work

Flexible schedule

Ongoing training

Calibration sessions

Job summary

AI Trainer Jobs is seeking a Web Developer evaluator to support AI model assessment. You will review prompts and responses, compare outputs, and provide structured feedback to improve model performance.

The role is remote within the United States with flexible scheduling, requiring at least 10 hours per week and contributing to cutting-edge AI research and tooling.

Qualifications

• Experience evaluating AI or content-related data
• Ability to consistently apply rubrics across large datasets
• Excellent written communication to articulate issues and rubric references

Responsibilities

• Evaluate web developer model outputs against a rubric and tag severity of responses
• Compare paired responses and provide rationale for stronger answer
• Identify hallucinations, instruction-following failures, and unsafe content with structured tags
• Capture ambiguous prompts and advise rubric updates to improve evaluation
• Calibrate assessments against gold-standard examples weekly
• Review browser automation and multi-step search agents for reliability and accuracy
• Audit batches for rubric consistency and suggest clarifications or updates

Skills

Evaluation experience

Multi-page rubrics

Strong written communication

Attention to detail

Asynchronous work

Linguistics background (advantage)

Inter-rater calibration (advantage)

Job description

About The Company

AI Trainer Jobs is a leading organization dedicated to advancing artificial intelligence through rigorous data annotation and evaluation services. Our company specializes in providing high-quality training data for AI models, ensuring that cutting‑edge technologies perform effectively across various applications. With a focus on innovation, accuracy, and reliability, AI Trainer Jobs collaborates with global clients to enhance the capabilities of machine learning systems. Our team comprises experts in AI, data annotation, linguistics, and quality assurance, all committed to maintaining the highest standards in AI training and evaluation processes.

About The Company

AI Trainer Jobs is a leading organization dedicated to advancing artificial intelligence through rigorous data annotation and evaluation services. Our company specializes in providing high‑quality training data for AI models, ensuring that cutting‑edge technologies perform effectively across various applications. With a focus on innovation, accuracy, and reliability, AI Trainer Jobs collaborates with global clients to enhance the capabilities of machine learning systems. Our team comprises experts in AI, data annotation, linguistics, and quality assurance, all committed to maintaining the highest standards in AI training and evaluation processes.

About The Role

The Web Developer role at AI Trainer Jobs is a remote evaluation position designed to support our AI model assessment initiatives. As a Web Developer evaluator, you will play a crucial part in reviewing web developer evaluation prompts and responses against our established quality rubric. Your primary responsibilities include comparing paired outputs, identifying edge cases, and providing structured, detailed feedback that the modeling team can utilize to improve and retrain AI systems. This role offers an opportunity to work remotely within the US, providing flexible scheduling while contributing to the development of advanced AI technologies.

Qualifications

• Prior experience in evaluation, annotation, or human‑rater roles related to web development, AI, or content moderation.


• Strong ability to apply multi‑page rubrics consistently across large batches of data.


• Excellent written communication skills with the capacity to clearly articulate issues and relevant rubric clauses.


• High attention to detail and the ability to identify prompt or response issues accurately.


• Reliable availability for at least 10 hours per week in an asynchronous work environment.


• Background in linguistics, trust & safety review, or related fields is advantageous.


• Experience with inter‑rater agreement metrics and calibration processes is a plus.

Responsibilities

• Evaluate web developer model outputs against a versioned rubric, assigning severity tags for different types of responses.


• Compare paired responses to determine the stronger answer and provide a detailed rationale for your choice.


• Identify and label hallucinations, instruction‑following failures, and unsafe content using structured tags.


• Capture ambiguous prompts and communicate these cases back to the program team for rubric updates and improvements.


• Maintain high reviewer‑quality scores by calibrating your assessments against gold‑standard examples on a weekly basis.


• Review browser automation and multi‑step search agents to verify source reliability and response accuracy.


• Assist in auditing batches for rubric consistency, report any drift, and suggest clarifications or updates to the evaluation rubric.

Benefits

At AI Trainer Jobs, we offer competitive hourly rates based on experience and performance, with compensation confirmed after the interview process. Our remote work setup provides flexibility to balance your professional and personal life. You will have access to ongoing training and calibration sessions to enhance your evaluation skills. Additionally, working with a collaborative team focused on cutting‑edge AI development offers valuable experience in the rapidly evolving AI industry. We also promote a supportive work environment that encourages continuous learning and professional growth.

Equal Opportunity

AI Trainer Jobs is an equal opportunity employer committed to fostering an inclusive and diverse workplace. We do not discriminate based on race, color, religion, gender, sexual orientation, age, disability, or any other protected characteristic. We believe that diversity enhances our ability to innovate and deliver exceptional results, and we welcome applicants from all backgrounds to join our team.