AI Quality & Evaluation Engineer — Flexible Hours

Telavox; Stockholms kommun; In loco; SEK 900.000 - 1.300.000; Tempo pieno

Descrizione del lavoro

Telavox is seeking an AI Engineer to own evaluation of models used in our transcripts, summaries, and AI agents. You’ll build continuous evaluation pipelines, benchmark models and prompts, and decide which models run in production across our AI products.

You’ll collaborate with backend engineers and the AI domain team to shape priorities with the product manager, ensuring data-driven decisions and measurable quality improvements across regions.

Competenze

• Hands-on work evaluating LLM/machine-learning model output in a real product.
• Strong Python and data-science tooling, and experience analysing data to make product and business recommendations.
• Experience with prompt design and structured outputs for LLMs in production.
• Ability to communicate findings clearly, in writing and in presentations.
• Proficient in English, both spoken and written.

Mansioni

• Build and run evaluation pipelines for transcription, summarisation and conversational agents.
• Benchmark candidate models and decide what runs in production with quality and cost as factors.
• Design and evaluate prompts and structured outputs behind our products.

Conoscenze

LLM evaluation

Python

Data science tooling

Prompt design

Communication

English proficiency

Strumenti

Open models deployment

Self-hosted models

GPU deployment

Descrizione del lavoro

Telavox is seeking an AI Engineer to own evaluation of models used in our transcripts, summaries, and AI agents. You’ll build continuous evaluation pipelines, benchmark models and prompts, and decide which models run in production across our AI products.

You’ll collaborate with backend engineers and the AI domain team to shape priorities with the product manager, ensuring data-driven decisions and measurable quality improvements across regions.