Content Specialist III (AI Content Evaluation & Quality)

Intelliswift - An LTTS Company; United States; Teletrabalho; USD 70 000 - 100 000; Tempo integral

Resumo da oferta

Intelliswift - An LTTS Company is seeking a Content Specialist III to evaluate, improve, and shape AI content across diverse topics. The role focuses on high-quality, accurate model responses, requiring exceptional writing and editing skills and the ability to apply structured guidelines consistently.

The position is remote in the United States, with a 6+ month contract (W2). You will refine prompts, document evaluation outcomes, and collaborate cross-functionally to enhance AI behavior and

Qualificações

• Bachelor's degree or equivalent practical experience.
• 5+ years in writing, editing, journalism, content production, or related fields.
• Exceptional writing, editing, proofreading, and communication skills.
• Deep expertise in at least one subject area; apply it when evaluating content.
• Strong analytical thinking, judgment, and ability to apply guidelines.
• Attention to detail and commitment to quality.

Responsabilidades

• Evaluate AI-generated responses across diverse subjects and conversation types.
• Test new model capabilities and document success cases, failures, and edge cases.
• Assess responses using detailed quality standards, scoring rubrics, and evaluation frameworks.
• Provide actionable feedback on model performance, reasoning quality, tone, factual accuracy, and instruction adherence.
• Refine prompts and evaluation criteria to improve output quality and consistency.
• Perform quality reviews of both human and AI-generated evaluations.
• Conduct fact-checking, content validation, and root-cause analysis on model failures.
• Collaborate with cross-functional teams to improve evaluation processes and model performance.
• Document findings clearly and communicate recommendations effectively.
• Adapt quickly to changing priorities and evolving project requirements.

Conhecimentos

Writing

Editing

Content Production

Technical Writing

Policy Analysis

Research

STEM

Formação académica

Bachelor's degree or equivalent

Descrição da oferta de emprego

Job Title: Content Specialist III (AI Content Evaluation & Quality)

Location: Remote (U.S. East Coast or West Coast Preferred)

Duration: 6+ Month Contract (Extension Possible)

Contract: W2

We are seeking an experienced Content Specialist III to help evaluate, improve, and shape next-generation AI products. This role combines content expertise, analytical thinking, and AI evaluation to ensure high-quality, accurate, and helpful model responses across a wide range of topics.

The ideal candidate is an exceptional writer and editor who can apply structured guidelines consistently, identify quality issues, provide detailed feedback, and help improve AI behavior through prompt refinement and rigorous evaluation techniques.

Responsibilities

• Evaluate AI-generated responses across diverse subjects and conversation types.
• Test new model capabilities and document success cases, failures, and edge cases.
• Assess responses using detailed quality standards, scoring rubrics, and evaluation frameworks.
• Provide actionable feedback on model performance, reasoning quality, tone, factual accuracy, and instruction adherence.
• Refine prompts and evaluation criteria to improve output quality and consistency.
• Perform quality reviews of both human and AI-generated evaluations.
• Conduct fact-checking, content validation, and root-cause analysis on model failures.
• Collaborate with cross-functional teams to improve evaluation processes and model performance.
• Document findings clearly and communicate recommendations effectively.
• Adapt quickly to changing priorities and evolving project requirements.

Required Qualifications

Bachelor's degree or equivalent practical experience.

5+ years of experience in one or more of the following:

• Writing
• Editing
• Journalism
• Content Production
• Technical Writing
• Policy Analysis
• Research
• STEM-related disciplines

• Exceptional writing, editing, proofreading, and communication skills.
• Deep expertise in at least one subject area and the ability to apply that expertise when evaluating content.
• Strong analytical thinking and judgment.
• Ability to interpret and consistently apply detailed guidelines and evaluation criteria.
• Strong attention to detail and commitment to quality.

Preferred Qualifications

• Experience working with AI systems through evaluation, annotation, prompt development, quality assessment, red teaming, or model testing.
• Experience performing fact-checking, source verification, and failure analysis.
• Familiarity with structured quality review processes and rubric-based evaluations.
• Ability to work independently in a fast-paced environment with shifting priorities.
• Strong problem-solving and documentation skills.