QA Engineer – Quality Engineer

Jobtailor; São Paulo; Presencial; BRL 120 000 - 170 000; Tempo integral

Resumo da oferta

Jobtailor in Brazil, São Paulo, seeks an experienced QA/SDET to design and maintain automated test suites for AI-driven products. You will evaluate AI outputs, define quality gates, and ensure CI/CD reliability while triaging defects and guiding product improvements.

Ideal candidates have 3+ years in QA, hands-on automation with Python or TypeScript, and experience with Playwright or Cypress, SAP testing tools, and UAT processes. Proficiency in English is essential.

Qualificações

• 3+ years of experience in QA, SDET, or Test Engineering.
• Hands-on experience with test automation for products in production.
• Strong knowledge of Python and/or TypeScript.
• Experience with modern E2E testing frameworks such as Playwright, Cypress, or equivalent.
• Experience with API testing.
• Solid knowledge of CI/CD, including pipeline creation and maintenance.
• Ability to test systems with non-deterministic responses.
• Experience evaluating LLM or Machine Learning outputs, or ability to develop a consistent approach to doing so.
• Ability to accurately assess defect severity and impact.
• Use of AI tools in day-to-day work.
• Self-directed, with ability to establish processes in unstructured environments.
• Previous experience at early-stage companies, preferably building QA functions from the ground up.
• Experience evaluating LLM-based features or AI agents in production.
• Knowledge of evals, hallucination detection, and HITL review processes.
• Familiarity with enterprise ecosystems, especially SAP, Salesforce, ERP, and CRM.
• Experience with SAP testing tools like Tricentis or Worksoft.
• Experience with SAP testing cycles and UAT.
• Knowledge of performance, load, or security testing.
• Sufficient programming ability to not only identify a bug but also potentially fix it.
• Professional to fluent English.

Responsabilidades

• Build and maintain the platform’s automated test suite, including E2E, integration, and regression tests.
• Create mechanisms to evaluate responses generated by AI agents.
• Develop golden datasets, scoring criteria, and regression tests for LLM-generated artifacts.
• Determine whether changes to prompts or models have improved or degraded results.
• Define Quality Gates and the criteria that may block a release.
• Ensure the reliability of CI pipelines.
• Identify and fix flaky tests and slow pipelines.
• Perform exploratory and release testing.
• Triage defects, assign severity, and track fixes through final validation.
• Turn failure patterns identified during testing into insights for product and roadmap evolution.

Conhecimentos

Test Automation

Python

TypeScript

E2E Testing

API Testing

CI/CD

LLM Evaluation

Performance Testing

Load Testing

Security Testing

Playwright

Cypress

Tricentis

Worksoft

SAP

Salesforce

ERP

CRM

HITL/Revision

Ferramentas

SAP

Salesforce

ERP

CRM

Tricentis

Worksoft

Playwright

Cypress

Descrição da oferta de emprego

• Build and maintain the platform’s automated test suite, including E2E, integration, and regression tests
• Create mechanisms to evaluate responses generated by AI agents
• Develop golden datasets, scoring criteria, and regression tests for LLM-generated artifacts
• Determine whether changes to prompts or models have improved or degraded results
• Define Quality Gates and the criteria that may block a release
• Ensure the reliability of CI pipelines
• Identify and fix flaky tests and slow pipelines
• Perform exploratory and release testing
• Triage defects, assign severity, and track fixes through final validation
• Turn failure patterns identified during testing into insights for product and roadmap evolution

Requirements

• 3+ years of experience in QA, SDET, or Test Engineering
• Hands-on experience with test automation for products in production
• Strong knowledge of Python and/or TypeScript
• Experience with modern E2E testing frameworks such as Playwright, Cypress, or equivalent
• Experience with API testing
• Solid knowledge of CI/CD, including pipeline creation and maintenance
• Ability to test systems with non-deterministic responses
• Experience evaluating LLM or Machine Learning outputs, or the ability to develop a consistent approach to doing so
• Ability to accurately assess defect severity and impact
• Use of AI tools in day-to-day work
• Self-directed, with the ability to establish processes in environments that do not yet have a defined model
• Previous experience at early-stage companies, preferably having helped build the QA function or processes from the ground up
• Experience evaluating LLM-based features or AI agents in production
• Knowledge of evaluation design (evals), hallucination detection, and Human-in-the-Loop (HITL) review processes
• Familiarity with enterprise software ecosystems, especially SAP, Salesforce, ERP, and CRM
• Experience with SAP testing tools such as Tricentis or Worksoft
• Experience with SAP testing cycles and UAT
• Knowledge of performance, load, or security testing
• Sufficient programming ability to not only identify a bug but also potentially fix it
• Professional to fluent English

Core Competencies

Demonstrates expertise in test automation, particularly in building and maintaining automated test suites for AI-driven products. Proficient in evaluating LLM outputs and ensuring the reliability of CI/CD pipelines while effectively managing defect triage and testing processes.

Highest-signal resume keywords

• Test Automation
• Python
• E2E Testing Frameworks
• CI/CD Pipeline Management
• LLM Evaluation

ATS Optimization Keywords

Hard Skills

• Test Automation
• Python
• TypeScript
• E2E Testing
• API Testing
• CI/CD
• Defect Severity Assessment
• Performance Testing
• Load Testing
• Security Testing

Soft Skills

• Self-Directed
• Process Establishment
• Insight Generation

Industry Keywords

• QA
• SDET
• Test Engineering
• Human-in-the-Loop
• Evaluation Design
• Hallucination Detection

Tools & Technologies

• Playwright
• Cypress
• Tricentis
• Worksoft
• SAP
• Salesforce
• ERP
• CRM