Senior QA Engineer, AI-Driven FinTech | Remote & Growth

Mount Olive College; Lisboa; Híbrido; EUR 70 000 - 110 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

24 days annual leave

Home office stipend

Learning budget

Recharge Week

Resumo da oferta

UMO is building a unified, AI-powered money platform with offices in Lisbon, Dubai, Kyiv, and Lviv, offering remote-ready roles. You will own test automation across backend microservices and mobile apps using TypeScript, Playwright, Appium, or Cucumber, and lead LLM evaluation to ensure robust, safe AI features.

You will collaborate with engineering, product, and AI/ML squads to embed quality into the build pipeline and drive continuous delivery in GitHub Actions or GitLab CI.

Qualificações

• 5+ years of software testing and quality engineering experience.
• Strong TypeScript testing with Playwright, Cucumber, or similar.
• Experience with native iOS/Android test automation (Appium or equivalents).
• Proven REST API testing in data-sensitive environments.
• Experience evaluating LLM/RAG systems with tools like RAGAS or DeepEval.

Responsabilidades

• Own end-to-end test automation architecture across backend microservices and native mobile apps.
• Own quality bar for LLM-powered features; evaluate model behavior and catch hallucinations/regressions.
• Collaborate with engineering, product, and AI/ML squads to embed quality into the build pipeline.
• Integrate automated regression, evaluation, and red-team suites into CI pipelines (GitHub Actions or GitLab CI).
• Design evaluation suites for LLM-powered features and measure groundedness, relevancy, faithfulness, and hallucination rate.
• Perform AI red-teaming and safety testing across LLM and agent systems.
• Support lightweight MLOps tasks: prompt/model versioning, evaluation data management, monitoring.

Conhecimentos

TypeScript

Playwright

Appium

Cucumber

API testing

LLM evaluation

CI/CD

AI tools

English communication

Ferramentas

RAGAS

DeepEval

DeepTeam

PyRIT

LangSmith

Langfuse

Arize

GitHub Actions

GitLab CI

Descrição da oferta de emprego

UMO is building a unified, AI-powered money platform with offices in Lisbon, Dubai, Kyiv, and Lviv, offering remote-ready roles. You will own test automation across backend microservices and mobile apps using TypeScript, Playwright, Appium, or Cucumber, and lead LLM evaluation to ensure robust, safe AI features.

You will collaborate with engineering, product, and AI/ML squads to embed quality into the build pipeline and drive continuous delivery in GitHub Actions or GitLab CI.