AI Quality Engineer for Legal AI Workflows

Newcode.ai; Oslo fylke; On-site; NOK 700,000 - 900,000; Full time

Job summary

Newcode.ai is seeking a QA Engineer to own evaluation systems for our AI products, working with Software Engineering and Legal Engineering to ensure real-world performance in legal workflows. You will design both automated and human-in-the-loop evaluation methods and build benchmarks to quantify quality.

You will translate complex model outputs into actionable improvements, develop robust testing pipelines, and establish standards for what “good” looks like across legal cases, collaborating

Qualifications

• Strong experience with Python or similar programming languages.
• Experience building testing, evaluation, or quality frameworks for AI products.
• Understanding of LLMs, retrieval systems, and AI evaluation methodologies.
• Analytical mindset with strong problem-solving skills.
• Ability to communicate technical findings to diverse stakeholders.

Responsibilities

• Design and build automated and human-in-the-loop evaluation frameworks for AI-powered products and workflows.
• Develop metrics and benchmarks to measure AI quality, accuracy, reliability, and consistency.
• Create evaluation datasets, test cases, and regression suites for complex legal use cases.
• Analyze model outputs to identify failure modes and improvement opportunities.
• Collaborate with Legal Engineers and Software Engineers to integrate evaluations into product workflows.

Skills

Python

AI/ML testing

Quality assurance

Data analysis

Cross-functional collaboration

Job description

Newcode.ai is seeking a QA Engineer to own evaluation systems for our AI products, working with Software Engineering and Legal Engineering to ensure real-world performance in legal workflows. You will design both automated and human-in-the-loop evaluation methods and build benchmarks to quantify quality.

You will translate complex model outputs into actionable improvements, develop robust testing pipelines, and establish standards for what “good” looks like across legal cases, collaborating