AI Quality & Evaluation Engineer: Production Metrics

ConnexAI; Manchester; On-site; GBP 50,000 - 70,000; Full time

Job summary

ConnexAI in Manchester is looking for a professional to measure and improve AI systems in production. This role focuses on defining performance metrics for LLMs, ASR, and TTS, and requires strong Python skills. Responsibilities include designing evaluation tests, building datasets, and collaborating with product and engineering teams to ensure AI improvements translate into better user experiences. Ideal candidates should have a solid understanding of AI quality measurement and experience in machine learning environments.

Qualifications

• Strong Python skills for scripting and data analysis.
• Experience with machine learning systems, particularly evaluation.
• Understanding of LLMs or speech systems like ASR and TTS.

Responsibilities

• Design and run evaluations for AI systems.
• Build datasets and define metrics for quality evaluation.
• Work with product teams to improve model performance.

Skills

Strong Python

Experience with ML systems

Understanding of LLMs or speech systems

Ability to design test cases

Comfortable working with engineers and product teams

Job description

ConnexAI in Manchester is looking for a professional to measure and improve AI systems in production. This role focuses on defining performance metrics for LLMs, ASR, and TTS, and requires strong Python skills. Responsibilities include designing evaluation tests, building datasets, and collaborating with product and engineering teams to ensure AI improvements translate into better user experiences. Ideal candidates should have a solid understanding of AI quality measurement and experience in machine learning environments.