AI Quality Engineer — Legal Tech & AI Evaluation

Newcode.ai; Oslo; On-site; NOK 650,000 - 900,000; Full time

Job summary

Newcode.ai is a fast-growing legal tech and agentic AI company transforming how legal work is done. We are seeking a QA Engineer to own evaluation frameworks for our AI products, collaborating with Software Engineering and Legal Engineering to measure performance in real-world workflows.

You will design automated and human-in-the-loop testing, develop metrics, and turn insights into concrete improvements that uphold quality across our AI portfolio.

Qualifications

• Strong software engineering or technical background, with experience working on AI/ML systems.
• Experience building testing, evaluation, or quality frameworks for AI-powered products.
• Strong Python and/or similar programming experience.
• Understanding of LLMs, generative AI, retrieval systems, agents, and/or AI evaluation methodologies.

Responsibilities

• Design and build automated and human-in-the-loop evaluation frameworks for AI-powered products and workflows.
• Develop metrics and benchmarks to measure accuracy, reliability, consistency, and overall AI quality.
• Build evaluation datasets, test cases, and regression suites for complex legal use cases.
• Analyze model outputs to identify failure modes, patterns, and opportunities for improvement.

Skills

Python

AI/ML systems

Evaluation methodologies

Stakeholder communication

Job description

Newcode.ai is a fast-growing legal tech and agentic AI company transforming how legal work is done. We are seeking a QA Engineer to own evaluation frameworks for our AI products, collaborating with Software Engineering and Legal Engineering to measure performance in real-world workflows.

You will design automated and human-in-the-loop testing, develop metrics, and turn insights into concrete improvements that uphold quality across our AI portfolio.