LLMOps Lead - AI Quality, Governance & Observability

Sovos Latinoamérica; Lisboa; Presencial; EUR 45 000 - 75 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Bi-Weekly Meeting Free Days

Mentoring and career development

Globally recognized training opportuni

Health and wellness benefits

Resumo da oferta

Sovos in Lisbon, Portugal, seeks an LLM Ops Lead with 2-3 years of hands-on experience to scale Generative AI across the business, ensuring quality, safety, observability, and continuous improvement. You will own AI quality standards, evaluation frameworks, and release gates for our AI-powered products.

You will lead prompt and model versioning, governance initiatives, safety programs, and AI incident response, coordinating with Product and Engineering.

Qualificações

• 2-3 years of experience in Applied AI, LLMOps, AI Quality Engineering, AI Operations, Test Automation, or related fields.
• Hands-on experience with Langfuse, LangSmith, or similar AI observability and evaluation tools.
• Strong understanding of AI quality, evaluation pipelines, prompt versioning, and AI governance.
• Experience with AI safety, guardrails, content moderation, audit logging, and release management.
• Strong stakeholder management and communication skills.
• Ability to support teams operating across UK business hours.

Responsabilidades

• Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products
• Lead prompt and model versioning, regression detection, and AI governance initiatives
• Coordinate Safety, Guardrails, and Red Team testing programs
• Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse
• Manage Jira epics, ticket triage, and cross-functional coordination across Product and Engineering teams
• Develop LLMOps playbooks, best practices, and operational standards
• Lead AI incident response, troubleshooting, and root-cause analysis

Conhecimentos

AI governance

Stakeholder management

Communication

AI safety

Ferramentas

Langfuse

LangSmith

Descrição da oferta de emprego

Sovos in Lisbon, Portugal, seeks an LLM Ops Lead with 2-3 years of hands-on experience to scale Generative AI across the business, ensuring quality, safety, observability, and continuous improvement. You will own AI quality standards, evaluation frameworks, and release gates for our AI-powered products.

You will lead prompt and model versioning, governance initiatives, safety programs, and AI incident response, coordinating with Product and Engineering.