Quality Assurance Analyst

RecruiterForge

United States

Remote

USD 105,000 - 150,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Medical insurance
Dental insurance
Vision insurance
Maternity leave
Paternity leave
Disability insurance
Leadership development
Company merchandise

Job summary

RecruiterForge is seeking a QA Analyst - LOIS for Word to own the QA strategy for an AI-powered legal drafting product. You will evaluate AI outputs, define QA processes, and design automation to ensure accuracy and reliability across document workflows.

The role collaborates with Product and Engineering, supporting rapid releases from a QA perspective. Remote position with SF-area ties and compensation aligned to experience.

Qualifications

  • 2+ years of QA experience with ownership of test strategy for a product or feature
  • Experience building QA processes and automation from scratch in a fast-moving environment
  • Comfort with ambiguity and rapid releases with strong analytical and problem-solving skills
  • Attention to functional correctness and AI output quality, including defining correctness when there is no single right answer
  • Clear, concise written communication and actionable, evidence-backed bug reports
  • Experience with bug tracking tools and test management workflows

Responsibilities

  • Define test strategy, QA process, and playbook for the product
  • Partner early with Product and Engineering so test plans are ready before features ship
  • Evaluate AI output for accuracy, citation reliability, and appropriately scoped reasoning
  • Design and maintain evaluation sets for nondeterministic output and detect regressions
  • Test AI-generated document edits for accuracy and formatting across document structures
  • Build automated guardrails in CI/CD for critical workflows and write reproducible bug reports
  • Work with legal domain experts to translate customer uses into test coverage
  • Suggest improvements from a product/design perspective even when functionality works
  • Document discrepancies by comparing behavior across surfaces

Skills

Test strategy
QA automation
Bug reporting
CI/CD
Analytical thinking
Communication

Job description

The QA Analyst - LOIS for Word owns quality for an AI-powered legal drafting product, building the test strategy, processes, and automation. The role evaluates functional behavior, AI output, and user workflows while bringing quality considerations into feature planning before release.

Test Strategy and AI Evaluation
  • Define the test strategy, QA process, and playbook for the product.
  • Partner early with Product and Engineering so test plans are ready before features ship.
  • Evaluate AI output for accuracy, citation reliability, and appropriately scoped reasoning.
  • Design and maintain evaluation sets for nondeterministic output and detect regressions over time.
  • Test AI-generated document edits for accuracy, formatting fidelity, and behavior across document structures.
  • Build automated guardrails in CI/CD for critical workflows and write structured, reproducible bug reports.
  • Work with legal domain experts to understand different customer uses and translate them into test coverage.
  • Apply a product and design perspective, suggesting improvements even where functionality works.
  • Compare behavior across product surfaces and document discrepancies.
QA Experience
  • 2+ years of QA experience, including ownership of test strategy for a product or major feature.
  • A record of building QA processes and automation from scratch in a fast-moving environment.
  • Comfort with ambiguity and rapid releases, with strong analytical and problem-solving skills.
  • Attention to functional correctness and AI output quality, including defining correctness when there is no single right answer.
  • Clear, concise written communication and actionable, evidence-backed bug reports.
  • Experience with bug tracking tools and test management workflows.
Preferred:
  • Testing AI or LLM products, especially output accuracy and evaluation design.
  • CI/CD pipelines and automated-test release gates.
  • Design or UX awareness, legal technology experience or comfort reading legal contracts, and programming beyond test automation.
Work and Compensation

This full-time remote position is associated with San Francisco, California, in Engineering – Quality. Salary ranges from $105,000–$150,000. The compensation package reflects location, qualifications, education, work experience, skills, and performance; the final offer is weighted by primary work location and expertise.

Benefits and Hiring

Full-time benefits include medical, dental, and vision insurance and maternity and paternity leave. Short- and long-term disability, opportunities to learn from leadership, and company merchandise are also available.

AI tools may assist with application review, resume analysis, response assessment, and identifying inconsistencies or verification signals. People make final hiring decisions. Candidates may request more information about how their data is processed.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

QA Analyst - LOIS for Word
QA Analyst - LOIS for Word

Filevine • San Francisco (CA)

On-site
USD 105,000 - 150,000
Medical, Dental & Vision insurance
Maternity & paternity leave
Short & long-term disability
+1
Remote QA Analyst for AI Legal Drafting
Remote QA Analyst for AI Legal Drafting

RecruiterForge • United States

Remote
USD 105,000 - 150,000
Medical insurance
Dental insurance
Vision insurance
+5
QA Architect for AI-Powered Legal Drafting (LOIS)
QA Architect for AI-Powered Legal Drafting (LOIS)

Filevine • San Francisco (CA)

On-site
USD 105,000 - 150,000
Medical, Dental & Vision insurance
Maternity & paternity leave
Short & long-term disability
+1
Legal Engineer
Legal Engineer

NetDocuments • United States

Hybrid
USD 155,000 - 165,000
Healthcare premiums covered
401(k) with 4% match
Flexible paid time off
+2
AI Solutions QA Analyst
AI Solutions QA Analyst

McDermott Will & Schulte • United States

Remote
USD 90,000 - 130,000
Remote-first
AI Test Engineer (LLM and AI QA)
AI Test Engineer (LLM and AI QA)

Synoptek • United States

Remote
USD 90,000 - 120,000
Legal Quality Assurance Lead (QAL)
Legal Quality Assurance Lead (QAL)

SuperAnnotate AI, Inc. • United States

Remote
USD 136,000 - 194,000
Legal Quality Assurance Lead (QAL)
Legal Quality Assurance Lead (QAL)

SME.careers • United States

Remote
USD 55,000 - 96,000
AI Solutions QA Analyst
AI Solutions QA Analyst

McDermott • United States

Remote
USD 93,000 - 142,000
Discretionary bonus
Competitive benefits package
QA Engineer
QA Engineer

7Seventy • United States

Hybrid
USD 110,000 - 135,000
Equity