QA Engineer (Remote)

Joblet-AI; India; On-site; INR 600,000 - 1,200,000; Full time

Job summary

Joblet-AI is seeking QA Engineers to contribute to software quality evaluation for a high-volume AI training project in India. You will apply QA expertise to help train next-generation AI systems and ensure the quality of model outputs through real-world input, with no prior AI experience required.

Specialists with strong QA backgrounds are encouraged to apply, including those with data annotation or AI evaluation experience.

Qualifications

• Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or similar role.
• Strong QA fundamentals: test case design, bug tracking, regression testing, and both manual and automated testing.
• Prior paid human data experience supporting AI training (annotation, labeling, RLHF, AI response evaluation) is a plus.
• Excellent analytical and problem-solving skills with meticulous attention to detail.
• Ability to communicate complex findings clearly in written English (B2 level or above).
• No formal degree required—practical testing experience takes precedence.
• Reliable internet connection and readiness to begin promptly.

Responsibilities

• Evaluate and rate AI-generated outputs against quality criteria and rubrics.
• Design and assess test cases for functional, regression, and edge cases.
• Review bug reports to ensure reproducibility, complete documentation, and proper severity.
• Identify, isolate, and document defects with precise reproduction steps.
• Provide detailed, actionable feedback to support rapid resolution by developers.
• Collaborate to refine evaluation guidelines and improve testing standards.

Skills

QA Engineer

Test design

Bug tracking

Automated testing

English communication

Hands-on testing

Education

No formal degree required

Tools

Bug tracking tools

Job description

QA Engineers to contribute expertise in software quality evaluation for a customer's high-volume project.

In this role, you'll apply your expertise to help train next‑generation AI systems.

Your work will shape how models learn, reason, and perform through high-quality, real‑world input.

No prior experience in AI is required — your domain knowledge is what matters.

This opportunity is open to specialists with strong professional QA backgrounds.

Prior paid experience with human data annotation, RLHF, model output evaluation, or rubric‑based grading for AI training is a plus, but not required.

Scope of Work

1. Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying answers that may appear correct but are actually inaccurate or incomplete.
2. Design and assess comprehensive test cases for functional, regression, and edge‑case scenarios, including negative and boundary tests.
3. Review bug reports and test documentation to ensure issues are reproducible, documentation is complete, and severity is properly assessed.
4. Identify, isolate, and document defects with precise reproduction steps and utilize structured tracking mechanisms.
5. Provide detailed, actionable written feedback and annotations to support efficient resolution by developers without the need for clarification.
6. Collaborate with project teams to refine evaluation guidelines and ensure continuous improvement of testing standards and methodologies.

Preferred Qualifications

• 1. Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or similar role in software quality assurance.
• 2. Strong command of QA fundamentals, including test case design, bug tracking, regression testing, and both manual and automated testing strategies.
• 4. Prior paid human data experience supporting AI training — annotation, labeling, RLHF, AI response evaluation, model evaluation, or rubric‑based grading — is a plus.
• 5. Excellent analytical and problem‑solving skills with meticulous attention to detail.
• 6. Ability to communicate complex findings clearly in written English, providing specific, actionable feedback (B2 level or above).
• 7. No formal degree required—practical, demonstrable testing experience takes precedence.
• 8. Reliable internet connection and readiness to begin promptly.

What You’ll Do

1. Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying answers that may appear correct but are actually inaccurate or incomplete.
2. Design and assess comprehensive test cases for functional, regression, and edge‑case scenarios, including negative and boundary tests.
3. Review bug reports and test documentation to ensure issues are reproducible, documentation is complete, and severity is properly assessed.
4. Identify, isolate, and document defects with precise reproduction steps and utilize structured tracking mechanisms.
5. Provide detailed, actionable written feedback and annotations to support efficient resolution by developers without the need for clarification.
6. Collaborate with project teams to refine evaluation guidelines and ensure continuous improvement of testing standards and methodologies.