AI Quality & Validation Specialist Porto (hybrid)

Tonic App; Porto; Hybrid; EUR 55,000 - 75,000; Full time

Job summary

Tonic App in Porto, Portugal is seeking an AI Quality & Validation Specialist to own how we evaluate our AI products before production. You will define quality criteria, run evaluations, and sign off on releases as a gatekeeper, ensuring safety and regulatory alignment across markets.

You will design validation systems, create KPIs, maintain audit-ready documentation, and coordinate with product, data, engineering, clinicians, and compliance to scale validation as new features and markets expand.

Qualifications

• 2-4 years or more in a role where evaluating quality was the job itself: model or LLM evaluation, applied AI quality, validation, or an equivalent function.
• You have built an evaluation framework from scratch or have managed and evolved one.
• Clear technical writing for a regulatory or audit audience. Validation reports and records are a primary deliverable of this role.
• Comfortable working with data and interpreting quality and performance metrics.

Responsibilities

• Own the release gate. Define the quality criteria a version must meet to reach production and run the evaluation against them.
• Build and maintain the validation system. Design test sets, define KPIs and evaluators, link them to the state-of-the-art evaluation practices and keep them current as the product and its clinical scope evolve.
• Make validation reproducible. Maintain a traceable chain from requirement to test case to dataset version to result to release decision, so the same conclusion can be reached again months later by someone else.
• Monitor products in production. Track quality and perform post-market surveillance to inform internal stakeholders on key findings.
• Stand behind the process in audits. Work with our compliance lead, who owns the quality management system, to keep validation documentation audit-ready and to represent the evaluation process during internal and external audits.
• Scale evaluation without scaling costs linearly. Validation coverage grows with every new feature, model and market, and adding reviewers is not a viable answer. You will coordinate the human review work that sits with our clinical team and find ways to reduce the human load.

Skills

Strong communication

Cross-functional collaboration

Regulatory writing

Tools

Python

SQL

Job description

About the role

Tonic builds AI products used by healthcare professionals in their daily practice. Tonic AI is a CE-marked Class IIa medical device, in live clinical use across multiple European markets. As the AI Quality & Validation Specialist you own how we evaluate our AI products, and your sign-off is a mandatory gate for every production deployment. Our quality processes are certified and audited, so what you build has to hold up when an external assessor asks how we concluded a given version was safe to release. This is not a traditional QA role. AI products are probabilistic, so quality cannot be reduced to right-or-wrong tests. The core of the job is deciding what “good enough” means, making that decision measurable and reproducible. You will work closely with Product, Data & AI, Engineering and other teams to continuously improve our products.

What you will do

• Own the release gate. Define the quality criteria a version must meet to reach production and run the evaluation against them.


• Build and maintain the validation system. Design test sets, define KPIs and evaluators, link them to the state-of-the-art evaluation practices and keep them current as the product and its clinical scope evolve.


• Make validation reproducible. Maintain a traceable chain from requirement to test case to dataset version to result to release decision, so the same conclusion can be reached again months later by someone else.


• Monitor products in production. Track quality and perform post-market surveillance to inform internal stakeholders on key findings.


• Stand behind the process in audits. Work with our compliance lead, who owns the quality management system, to keep validation documentation audit-ready and to represent the evaluation process during internal and external audits.


• Scale evaluation without scaling costs linearly. Validation coverage grows with every new feature, model and market, and adding reviewers is not a viable answer. You will coordinate the human review work that sits with our clinical team and find ways to reduce the human load.

What we are looking for

• 2-4 years or more in a role where evaluating quality was the job itself: model or LLM evaluation, applied AI quality, validation, or an equivalent function.


• You have built an evaluation framework from scratch or have managed and evolved one.


• Clear technical writing for a regulatory or audit audience. Validation reports and records are a primary deliverable of this role.


• Comfortable working with data and interpreting quality and performance metrics.


• Accountability as the role carries real authority over what ships, and the judgement to use it well.


• Strong communication skills and the ability to work effectively with AI engineers, clinicians and compliance.


• Collaborative mindset and ability to operate in a fast-paced environment.


• Full professional proficiency in Portuguese and English.

Nice to have

• Experience in a regulated environment, or with audited processes.


• Python or SQL as working tools. Useful, not required.

Healthcare professionals across four countries use our products in their daily work, and two more markets are opening. Tonic AI is a CE-marked medical device, which very few AI products in Europe can say, and it is used on real cases by real clinicians rather than sitting in a pilot that never leaves the slide deck.

If you want to work hands-on with AI that is already being used in real clinical practice and play a key role in making it more reliable, consistent and useful this is the job.