Senior LLM Ops Engineer

InnoTech; Lisboa; Presencial; EUR 50 000 - 70 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Free mental health support

25 vacation days, including birthdays

Petsitting partnerships

Regular company parties

Resumo da oferta

InnoTech is looking for a Senior LLM Ops Engineer based in Lisbon, Portugal, to design and implement pipelines for processing production ML traffic and enhance operational excellence through robust monitoring and evaluation frameworks.

The ideal candidate will possess proven experience in building operational ML platforms, strong Python skills, and a focus on data protection measures. Benefits include mental health support, 25 vacation days, and regular team events.

Qualificações

• Experience building and operating data platforms with ingestion, monitoring, and support.
• Hands-on experience with evaluation frameworks and autorating systems.
• Ability to deliver production-ready solutions in Python.

Responsabilidades

• Design scalable pipelines for ML traffic processing.
• Develop observability and monitoring capabilities for production.
• Implement data protection measures for PII.

Conhecimentos

Production-grade data platform experience

Machine Learning framework maintenance

Python development

LLM observability and troubleshooting

Formação académica

BSc or MSc in Computer Science/Engineering

Ferramentas

Azure

AWS

SQL

TypeScript

Descrição da oferta de emprego

Senior LLM Ops Engineer

Responsibilities

• Design and implement scalable pipelines to collect and process production ML and LLM traffic from multiple sources;
• Define data storage strategies, retention policies, and privacy controls aligned with business and regulatory requirements;
• Develop observability, debugging, and monitoring capabilities to support production issue investigation and operational excellence;
• Build and maintain evaluation frameworks, automated rating systems, and alerting mechanisms to detect model and prompt performance regressions;
• Implement data protection measures, including PII redaction, content safety controls, and end-to-end platform operations.

Requirements

• Proven experience building and operating production-grade data or ML platforms, including ingestion, storage, governance, monitoring, and operational support;
• Hands-on experience creating and maintaining ML/LLM evaluation frameworks, autorating systems, golden datasets, and LLM-as-a-judge workflows;
• Practical experience with LLM observability, tracing solutions, and troubleshooting production AI systems;
• Strong Python development skills, with the ability to deliver production-ready solutions and leverage AI-assisted development practices;
• CPLP nationality or an EU work permit is required to facilitate employment.

Desired Skills

• Experience working with multi-tenant SaaS environments, data isolation strategies, and cloud platforms such as Azure and AWS;
• Knowledge of vector databases, embedding-based analysis, data versioning solutions, and GDPR-related data management practices;
• Experience with Claude Code, GitHub, SQL, TypeScript, and LLM ecosystems including Anthropic, OpenAI, and open-source models;
• Exposure to automotive, Android, AAOS, or other resource-constrained technology environments;
• BSc or MSc in Computer Science/Engineering, Information Technology, or related fields.

Benefits

• Health-focused partnerships, including free mental health support;
• 25 days of vacation, including your birthday, Carnival, and a bridging day between Christmas and New Year;
• Half a vacation day on your children's birthday and a school supplies voucher;
• Petsitting partnerships and a pet bereavement day;
• Regular team buildings, four annual company parties, game nights, and social events;
• InnoSports initiatives (football, marathons, and more);
• Events focused on sustainability and social responsibility actions.