Principal QA Engineer - AI-Driven Test Automation

Roca Alliances; Medellín; On-site; COP 180,000,000 - 240,000,000; Full time

Benefits offered by this job

Permanent contract

Private health insurance

Job summary

Roca Alliances is seeking a Principal Software Engineer in Test to own product quality across web, mobile, and API. You will lead the test strategy, automation architecture, and AI agents that execute tests, driving release safety and quality at scale.

The role demands deep experience in test automation for large SaaS platforms, hands‑on work with LLMs, strong SQL skills, and the ability to influence engineers, product, and executives. This is a senior IC role with platform focus.

Qualifications

• Experience designing test automation for large, long‑lived web applications.
• Hands‑on experience building with LLMs, agent loops, tool integration and evaluation harnesses.
• Strong SQL and relational data skills for test data in complex schemas.
• Proven CI/CD ownership: pipelines, gating, flake management and run‑time budgets.
• Ability to align quality strategy with engineers, product and executives, and adapt when evidence demands.

Responsibilities

• Own test strategy: decide what to test, data to use, and evidence required before release.
• Design and maintain automation architecture across web, mobile, and API layers, including data strategy and CI fixtures.
• Develop and operate AI agents that derive tests from tickets, explore the app, and generate regression scenarios.
• Set engineering testing standards, guardrails, and review rules to improve product quality.
• Provide clear quality signals: track defect and escape rates, and communicate outcomes to leadership.

Skills

Test automation design

LLM-based agent loops

SQL & data modeling

CI/CD ownership

Cross-functional influence

Advanced English

Tools

Jenkins

Bitbucket

Jira

Confluence

Terraform

Datadog

Grafana

Job description

Roca Alliances is seeking a Principal Software Engineer in Test to own product quality across web, mobile, and API. You will lead the test strategy, automation architecture, and AI agents that execute tests, driving release safety and quality at scale.

The role demands deep experience in test automation for large SaaS platforms, hands‑on work with LLMs, strong SQL skills, and the ability to influence engineers, product, and executives. This is a senior IC role with platform focus.