AI Ops Engineer

Sovos

Porto

Presencial

EUR 42 000 - 68 000

Tempo integral

Há 9 dias

Recebe mais respostas dos empregadores

Envia um currículo específico para a oferta em poucos minutos.

Resumo da oferta

Sovos is seeking an AI Ops Engineer to own the end-to-end LLMOps lifecycle for a portfolio of AI products. You will drive AI quality, safety, evaluation, observability, and continuous improvement to ensure reliable, scalable, production-ready AI experiences.

Join a global, remote-first team and help shape AI experiences used by customers worldwide. This role includes working the UK shift and collaborating with AI, Product, and Engineering leaders.

Qualificações

1–3 years in AI engineering or related fields.
Experience with LLM observability and evaluation platforms.
Knowledge of NLP concepts, embeddings, and retrieval.
Strong product mindset and stakeholder management.
Hands-on in building evaluation datasets and benchmarks.

Responsabilidades

Gate new AI agents, prompts, and models before production release.
Design and run evaluation frameworks using deterministic testing.
Build and maintain golden datasets, annotation pipelines, and human-review workflows.
Monitor AI quality, hallucinations, drift, latency, cost, and safety metrics.
Translate customer feedback and production learnings into measurable quality improvements.
Drive AI adoption and quality standards across your assigned product portfolio.

Conhecimentos

LLMOps

AI Quality Eng

Langfuse

LangSmith

Phoenix

NLP concepts

Embeddings

Retrieval

Stakeholder mgmt

Product mindset

QA/Test Automation

Ferramentas

Langfuse

LangSmith

Phoenix

Descrição da oferta de emprego

Build your future with Sovos.

Sovos is transforming tax compliance from a business requirement into a force for growth. Trusted by more than 100,000 customers worldwide, including many of the world's leading organizations, we are leveraging AI to build smarter, more impactful customer experiences.

We're looking for a AI Ops Engineer to own the end-to-end LLMOps lifecycle for a portfolio of Ask Sovi products. You'll drive AI quality, safety, evaluation, observability, and continuous improvement, ensuring our AI-powered experiences are reliable, scalable, and production-ready.

What You'll Do

Gate new AI agents, prompts, and models before production release
Design and run evaluation frameworks using deterministic testing and LLM-as-a-Judge methodologies
Build and maintain golden datasets, annotation pipelines, and human-review workflows
Monitor AI quality, hallucinations, drift, latency, cost, and safety metrics
Convert customer feedback and production learnings into measurable quality improvements
Drive AI adoption and quality standards across your assigned product portfolio

What We're Looking For

1-3 years of experience in AI Engineering, AI Operations, AI Quality Engineering, LLMOps, MLOps, Test Automation, or related disciplines
Experience with Langfuse, LangSmith, Phoenix, or similar LLM observability/evaluation platforms
Strong understanding of LLMOps, AI quality engineering, and evaluation frameworks
Experience with hallucination detection, model drift monitoring, prompt regression management, and AI safety evaluations
Hands‑on experience building evaluation datasets, quality benchmarks, and feedback loops
Familiarity with NLP concepts such as embeddings, semantic similarity, classification, and retrieval
Strong product mindset and stakeholder management skills

Ideal Backgrounds

Applied AI Engineers from AI-native or AI-first organizations
Product‑minded Engineers with strong customer and business awareness
QA/Test Automation Engineers who have transitioned into AI evaluation
AI Quality Analysts with expertise in hallucination detection, drift monitoring, and LLM‑as‑a‑Judge frameworks

Working Hours

UK Shift

Why Join Sovos?