LLM Ops Lead for AI Quality & Governance

Sovos Latinoamérica; Porto; Presencial; EUR 45 000 - 75 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Bi-Weekly Meeting Free Days

Mentoring & Career Development

Globally Recognized Training

Resumo da oferta

Sovos in Porto, Portugal is hiring an LLM Ops Lead with 2–3 years of Applied AI experience. You will drive AI quality, governance, and observability for Generative AI across products, coordinating with Product and Engineering teams to ensure safe, scalable deployments.

The role focuses on release gates, evaluation pipelines, and incident response, with influence across cross‑functional teams and UK business hours support.

Qualificações

• 2–3 years of Applied AI, LLMOps, AI Quality Engineering, AI Operations, Test Automation, or related fields.
• Hands‑on experience with Langfuse, LangSmith, or similar AI observability and evaluation tools
• Strong understanding of AI quality, evaluation pipelines, prompt versioning, and AI governance
• Experience with AI safety, guardrails, content moderation, audit logging, and release management
• Strong stakeholder management and communication skills
• Ability to support teams operating across UK business hours

Responsabilidades

• Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products
• Lead prompt and model versioning, regression detection, and AI governance initiatives
• Coordinate Safety, Guardrails, and Red Team testing programs
• Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse
• Manage Jira epics, ticket triage, and cross‑functional coordination across Product and Engineering teams
• Develop LLMOps playbooks, best practices, and operational standards
• Lead AI incident response, troubleshooting, and root‑cause analysis

Conhecimentos

Langfuse

LangSmith

AIOps

AI governance

Stakeholder mgmt

Ferramentas

Langfuse

LangSmith

Descrição da oferta de emprego

Sovos in Porto, Portugal is hiring an LLM Ops Lead with 2–3 years of Applied AI experience. You will drive AI quality, governance, and observability for Generative AI across products, coordinating with Product and Engineering teams to ensure safe, scalable deployments.

The role focuses on release gates, evaluation pipelines, and incident response, with influence across cross‑functional teams and UK business hours support.