LLM Ops Lead — AI Quality, Safety & Observability

Sovos; Porto; Presencial; EUR 60 000 - 80 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Bi-Weekly Meeting Free Days

Mentoring and career development

Globally recognized training programs

Health and wellness benefits

Flexible and collaborative work enviro

Resumo da oferta

Sovos is seeking an LLM Ops Lead with 2-3 years of experience to scale, govern, and operationalize Generative AI across the business from Porto, Portugal. You will own AI quality standards, manage model/versioning, and drive AI safety programs in collaboration with Product and Engineering teams.

The role emphasizes building observability for latency, cost, quality, and safety, plus leading incident response and governance initiatives while coordinating with UK business hours across teams.

Qualificações

• 2-3 years of experience in Applied AI, LLMOps, AI Quality Engineering, or related fields.
• Experience with AI safety, guardrails, content moderation, audit logging, and release management.

Responsabilidades

• Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products.
• Lead prompt and model versioning, regression detection, and AI governance initiatives.
• Coordinate Safety, Guardrails, and Red Team testing programs.
• Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse.
• Manage Jira epics, ticket triage, and cross-functional coordination across Product and Engineering teams.
• Develop LLMOps playbooks, best practices, and operational standards.
• Lead AI incident response, troubleshooting, and root-cause analysis.

Conhecimentos

2-3 years experience

Stakeholder management

Ferramentas

Langfuse

LangSmith

Descrição da oferta de emprego

Sovos is seeking an LLM Ops Lead with 2-3 years of experience to scale, govern, and operationalize Generative AI across the business from Porto, Portugal. You will own AI quality standards, manage model/versioning, and drive AI safety programs in collaboration with Product and Engineering teams.

The role emphasizes building observability for latency, cost, quality, and safety, plus leading incident response and governance initiatives while coordinating with UK business hours across teams.