AI/ML Software Test Developer

Jobtailor; Manaus; Presencial; BRL 120 000 - 180 000; Tempo integral

Resumo da oferta

Jobtailor in Manaus seeks an AI/ML quality assurance engineer to analyze requirements and acceptance criteria, focusing on probabilistic model behavior and impact on products. You will shape QA strategies across data, training, evaluation, inference, integration, and testing phases.

You will implement test automation with Pytest and other tools while monitoring data and model quality across versions. The role requires strong English, Python skills and a degree pursuit in CS/SE/Data Science, with

Qualificações

• Knowledge of AI/ML development lifecycle and evaluation.
• Understanding of ML fundamentals including NLP and vision.
• Knowledge of data leakage, imbalance and performance evaluation.
• Experience with code versioning (Git/GitLab/GitHub).
• Python proficiency for automation and testing tasks.
• Pursuing a relevant degree and strong English communication.

Responsabilidades

• Participate in analysis of requirements, user stories and acceptance criteria for AI/ML-enabled systems, identifying model limitations within the project's targeted context and the impact of the AI’s probabilistic behavior on the final product.
• Support QA strategies across data, training, evaluation, inference and integration stages of model development.
• Assess quality of dataset splits, including consistency, completeness, balance and leakage; evaluate models using team-defined test strategies and metrics.
• Execute the defined testing strategy for functional and non-functional aspects of systems.
• Validate end-to-end behavior of AI systems.
• Monitor, analyze and report quality metrics for software, data and models across versions.
• Develop and maintain test automation using Pytest, Robot Framework, Selenium, Appium, Cypress, JMeter, Artillery or equivalents.

Conhecimentos

AI/ML lifecycle understanding

Python

LLMs in test automation

Analytical thinking

Git versioning

English - advanced

Formação académica

Pursuing a degree in Computer Science/Software Engineering/Data Science

Ferramentas

Pytest

Robot Framework

Selenium

Appium

Cypress

JMeter

Artillery

Descrição da oferta de emprego

Responsibilities

• Participate in the analysis of requirements, user stories and acceptance criteria for AI/ML-enabled systems, identifying model limitations within the project's targeted context and the impact of the AI’s probabilistic behavior on the final product.
• Support the implementation of defined quality strategies for AI/ML projects, considering all stages of model development (data, training pipelines, evaluation, inference, integration and probabilistic behavior).
• Assess the quality of splits created in datasets, understanding core concepts of consistency, completeness, representativeness, balance, annotations, data leakage and suitability for the model and system objectives. Evaluate AI/ML models based on the test strategies and metrics defined by the team.
• Execute the defined testing strategy considering all functional and non-functional aspects of systems.
• Validate end-to-end behavior of AI systems.
• Monitor, analyze and report quality metrics related to software, data and models across all versions.
• Develop and maintain test automations and automated testing processes using frameworks such as Pytest, Robot Framework, Selenium, Appium, Cypress, JMeter, Artillery or equivalent tools.

Requirements

• Knowledge of the AI/ML development lifecycle, including problem definition, data collection, data preparation, training, validation, testing, deployment, monitoring and retraining.
• Understanding of fundamental concepts in machine learning, NLP, computer vision and multimodal models.
• Knowledge of model training, validation and testing, including overfitting, underfitting, generalization, data leakage, class imbalance, data distribution, data drift and performance evaluation.
• Analytical ability to investigate failures, compare results, identify patterns, propose hypotheses, validate evidence and communicate technical conclusions. Experience with code versioning, preferably with Git, GitLab, GitHub or similar tools.
• Basic knowledge of completeness analysis and safety for autonomous robotic tasks.
• Knowledge of the Python programming language.
• Ability to apply LLMs in the context of test automation for software.
• Currently pursuing a degree in Computer Science, Software Engineering, Computer Engineering, Data Science, Statistics or related fields.
• Advanced/Fluent English for reading, writing and day-to-day communication with the team and external stakeholders.