QA/Red Teaming Expert

Innodata Inc.; India; On-site; INR 5,328,597 - 7,104,796; Full time

Job summary

A technology company in India is seeking a Red Teaming, Prompt Evaluation, or AI/LLM Quality Assurance Expert to rigorously test AI-generated content. The ideal candidate will evaluate outputs for vulnerabilities and recommend mitigations. Proven experience in AI red teaming, excellent analytical skills, and a strong QA background are required. Diverse candidates are encouraged to apply.

Qualifications

• Proven experience in AI red teaming or adversarial prompt design.
• Familiarity with prompt engineering and ethical considerations in AI.
• Strong background in Quality Assurance for AI/ML systems.

Responsibilities

• Conduct Red Teaming exercises to identify harmful outputs from LLMs.
• Evaluate AI prompts to uncover potential failure modes.
• Collaborate with teams to report risks and suggest mitigations.

Skills

Red Teaming

Prompt Evaluation

AI/LLM Quality Assurance

Adversarial Testing

Analytical Writing

Job description

We are seeking highly analytical and detail-oriented professionals with hands-on experience in Red Teaming, Prompt Evaluation, and AI/LLM Quality Assurance. The ideal candidate will help us rigorously test and evaluate AI-generated content to identify vulnerabilities, assess risks, and ensure compliance with safety, ethical, and quality standards.

Job Title: Red Teaming, Prompt Evaluation, or AI/LLM Quality Assurance Expert

Key Responsibilities:

• Conduct Red Teaming exercises to identify adversarial, harmful, or unsafe outputs from large language models (LLMs).
• Evaluate and stress-test AI prompts across multiple domains (e.g., finance, healthcare, security) to uncover potential failure modes.
• Develop and apply test cases to assess accuracy, bias, toxicity, hallucinations, and misuse potential in AI-generated responses.
• Collaborate with data scientists, safety researchers, and prompt engineers to report risks and suggest mitigations.
• Perform manual QA and content validation across model versions, ensuring factual consistency, coherence, and guideline adherence.
• Create evaluation frameworks and scoring rubrics for prompt performance and safety compliance.
• Document findings, edge cases, and vulnerability reports with high clarity and structure.

Requirements:

• Proven experience in AI red teaming, LLM safety testing, or adversarial prompt design.
• Familiarity with prompt engineering, NLP tasks, and ethical considerations in generative AI.
• Strong background in Quality Assurance, content review, or test case development for AI/ML systems.
• Understanding of LLM behaviour's, failure modes, and model evaluation metrics.
• Excellent critical thinking, pattern recognition, and analytical writing skills.
• Ability to work independently, follow detailed evaluation protocols, and meet tight deadlines.

Preferred Qualifications:

• Prior work with teams like Open AI, Anthropic, Google DeepMind, or other LLM safety initiatives.
• Experience in risk assessment, red team security testing, or AI policy & governance.

Innodata Inc. is an equal opportunity employer and welcomes applications from diverse candidates.