LLM QA Engineer: AI Testing & Evaluation

Codefeast; United States; On-site; USD 90,000 - 140,000; Full time

Job summary

Codefeast is seeking a QA professional to contribute to LLM training and evaluation workflows by testing software, reviewing AI-generated outputs, and validating model behavior.

You will assess outputs for functional correctness, accuracy, reliability, edge-case handling, and adherence to requirements, and create detailed test cases, scenarios, and benchmarks to drive improvement across complex AI tasks.

Qualifications

• 3–8 years of professional experience in QA, software testing, quality engineering, or related roles.
• Strong experience with manual and automated testing, test case design, debugging, API testing, and regression testing.
• Experience with or strong interest in LLM evaluation, AI-assisted testing, prompt engineering, or AI training workflows.

Responsibilities

• Test software and review AI-generated outputs for quality and reliability.
• Validate LLM behavior across edge cases and requirements.
• Create and review test cases, evaluation criteria, bug reports, and quality benchmarks for AI-driven workflows.
• Provide detailed feedback to improve LLM performance and software quality across complex tasks.

Skills

Manual testing

Automated testing

Test case design

API testing

Regression testing

Job description

Codefeast is seeking a QA professional to contribute to LLM training and evaluation workflows by testing software, reviewing AI-generated outputs, and validating model behavior.

You will assess outputs for functional correctness, accuracy, reliability, edge-case handling, and adherence to requirements, and create detailed test cases, scenarios, and benchmarks to drive improvement across complex AI tasks.