AI Quality & Evaluation Specialist

Element Ops; Philippines; Remote; PHP 3.769.000 - 6.281.000; Vollzeit

Zusammenfassung

Element Ops is seeking an AI Quality & Evaluation Specialist to improve the reliability of an AI-powered platform used in real-time security operations. You will review AI alerts, label errors, and provide feedback to enhance performance.

The role emphasizes careful analysis, documentation, and collaboration with cross-functional teams. The ideal candidate has 3+ years in operations, QA, or data review, strong English communication, and comfort working remotely with US time zones.

Qualifikationen

• Experience analyzing information and identifying errors.
• Strong written English communication to document decisions andFeedback.
• Ability to work independently, adapt to guidelines, and maintain accuracy.

Aufgaben

• Review AI-generated alerts and actions for accuracy and relevance.
• Label incorrect outputs and provide actionable feedback.
• Document evaluation reasons and maintain logs in internal systems.
• Evaluate correct outputs to identify successful decisions and quality standards.
• Identify recurring errors and share findings with teams.
• Maintain records of evaluations, error classifications, and feedback.
• Apply critical thinking when reviewing AI outputs and flag unclear cases.
• Collaborate with operations to support platform improvements.
• Help refine evaluation criteria, guidelines, and workflows.

Kenntnisse

Analytical thinking

Attention to detail

Critical thinking

Written English

Independent work

Fast learner

Inquisitive mindset

Process improvement

SaaS aptitude

Tools

ChatGPT familiarity

Jobbeschreibung

We’re hiring an AI Quality & Evaluation Specialist to help improve the quality and reliability of an AI-powered platform used in real-time security operations. The platform monitors live inputs such as video, audio, location data, GPS, telemetry, and chat messages to identify potential incidents, send alerts to officers, and elevate events for operational review.

This role follows a human-in-the-loop approach, where human judgment and feedback help AI systems improve their performance and decision-making. You’ll contribute to the ongoing development of AI technology by helping ensure its outputs are accurate, appropriate, and reliable in real-world situations.

This is a great opportunity for someone who enjoys working with technology, analyzing information, solving problems, and improving processes. If you’re detail-oriented, naturally curious, and interested in helping AI systems become more effective over time, we’d love to hear from you.

Responsibilities:

• Review AI-generated alerts, messages, and actions to determine whether they are correct, appropriate, and supported by the available information.
• Identify and label incorrect AI outputs, including false positives, inaccurate location or status information, inappropriate messages, and other errors.
• Document why an AI-generated action was incorrect and provide clear, actionable feedback to help improve its accuracy and decision-making.
• Evaluate correct AI outputs to help identify successful decisions and maintain consistent quality standards.
• Identify recurring errors and patterns in AI-generated outputs and communicate findings to the appropriate teams.
• Maintain accurate records of evaluations, error classifications, and feedback using internal tools and systems.
• Apply critical thinking and sound judgment when reviewing alerts, messages, and other AI-generated actions.
• Follow established evaluation guidelines and flag unclear or unusual cases for further review when needed.
• Collaborate with operations and other internal teams to share findings and support continuous improvements to the AI platform.
• Help improve evaluation criteria, review guidelines, and workflows to make the quality review process more consistent and effective.

Requirements:

• 3+ years of experience in Operations, Quality Assurance, Data Review, Process Improvement, Customer Support, or a related field.
• Experience reviewing AI-generated outputs, evaluating AI systems, or working with AI-powered tools is a big plus.
• Experience working with US-based companies or teams is a plus.
• Experience as a Team Lead or Subject Matter Expert is a plus.
• Familiarity with AI tools such as ChatGPT and a genuine interest in understanding how AI systems work and improve.
• Strong analytical and critical thinking skills, with the ability to assess information, identify errors, and distinguish correct outputs from incorrect or misleading ones.
• Excellent attention to detail and the ability to recognize inconsistencies, false alarms, data errors, and inappropriate messages.
• Strong written English communication skills, especially when documenting errors, explaining decisions, and providing actionable feedback.
• A process improvement mindset, with the ability to identify recurring issues and suggest practical solutions.
• Naturally inquisitive and comfortable asking questions to understand how systems work and why errors occur.
• A fast learner who takes feedback well and adapts to changing guidelines and processes.
• Able to work independently while maintaining accuracy, consistency, and productivity.
• Comfortable navigating SaaS tools and learning new systems in a fast-paced environment.
• Must be available to work graveyard hours aligned with U.S. time zones.
• Reliable internet connection and dedicated work-from-home setup required.