Senior Quality Engineer (AI) - E-Learning

Truelogic; Brasil; Teletrabalho; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

Truelogic seeks a GenAI Quality Coach to innovate QA practices for GenAI systems. The role involves defining QA strategies and partnering with teams to ensure high-quality AI features. You will thrive in a remote setting, collaborating with top U.S. companies while enjoying competitive pay and flexible working conditions.

Serviços

100% Remote Work

Highly Competitive USD Pay

Paid Time Off

Work with Autonomy

Work with Top American Companies

Qualificações

• Excited by complexities of GenAI testing.
• Think like a product owner, act like a tester.
• Thrive in ambiguity; passionate about safe AI development.

Responsabilidades

• Define GenAI-specific QA strategy and framework.
• Own end-to-end test strategy for GenAI features.
• Lead design of prompt and model evaluation protocols.

Conhecimentos

Test Innovation

Prompt Analysis

User Feedback Insights

Collaboration

Formação académica

Experience in QA Testing

Descrição da oferta de emprego

About Truelogic

At Truelogic we are a leading provider of nearshore staff augmentation services headquartered in New York. For over two decades, we’ve been delivering top-tier technology solutions to companies of all sizes, from innovative startups to industry leaders, helping them achieve their digital transformation goals.

Our team of 600+ highly skilled tech professionals, based in Latin America, drives digital disruption by partnering with U.S. companies on their most impactful projects. Whether collaborating with Fortune 500 giants or scaling startups, we deliver results that make a difference.

By applying for this position, you’re taking the first step in joining a dynamic team that values your expertise and aspirations. We aim to align your skills with opportunities that foster exceptional career growth and success while contributing to transformative projects that shape the future.

Our Client

Our client is committed to building innovative and scalable solutions that drive efficiency and impact. We foster a culture of continuous learning, collaboration, and proactive problem-solving. If you're looking for an environment where you can grow and make a difference, we want to hear from you!

Job Summary

As we advance our AI development efforts, we recognize the need for more than a traditional QA engineer. We are seeking a GenAI Quality Coach—a strategic and hands-on role that blends test innovation, prompt effectiveness analysis, and user feedback insights.

This individual will help shape and evolve our QA practices specifically for GenAI

systems. They will partner closely with developers, product owners, and SMEs to ensure we are building robust, safe, and high-quality AI features

Responsibilities

1. Define GenAI-Specific QA Strategy

Develop a QA framework tailored to GenAI systems and workflows.

Design tests for:

• Prompt behavior across varied inputs and user tasks

• Hallucination detection

• Factual consistency and groundedness

Blend manual and automated test design for both deterministic and stochastic

outputs.

Collaborate with teams to obtain or create sample data with clear target

outputs.

2. Test Plan Ownership

Own end-to-end test strategy and execution for GenAI-powered features.

Ensure coverage across:

• Diverse prompt phrasing, user intents, and failure modes

• Multiple GenAI features (e.g., summarization, generation, classification)

• High-risk, edge-case, and compliance-driven scenarios.

3. Prompt Validation & Evaluation

Lead the design and implementation of prompt and model evaluation protocols:

• Alignment between user input and intended behavior

• Output fluency, tone, and coherence

• Clarity, coverage, and relevance of responses

Use Golden datasets and benchmark prompts to establish evaluation

baselines.

4. Human-in-the-Loop (HITL) Evaluation

Design and manage SME-driven review workflows.

Facilitate structured reviews focused on:

• Correctness/accuracy based on metrics and SME feedback

• Capturing edge-case failures

5. Reporting and KPIs

Define and track QA effectiveness using metrics such as:

• Pass rate for high-risk use-cases

• HITL reviewer agreement rates and flagging critical issues

• Use-case specific measures of “quality”

Deliver clear, actionable dashboards and reports to leadership on AI quality,

safety, and readiness

Qualifications and Job Requirements

You might be a great fit if you:

• Are excited by the complexities and challenges of GenAI testing.

• Think like a product owner, act like a tester, and communicate like a coach.

• Thrive in ambiguity and enjoy shaping new standards.

• Are passionate about safe, responsible AI development.

What We Offer

• 100% Remote Work: Enjoy the freedom to work from the location that helps you thrive. All it takes is a laptop and a reliable internet connection.

• Highly Competitive USD Pay: Earn an excellent, market-leading compensation in USD, that goes beyond typical market offerings.

• Paid Time Off: We value your well-being. Our paid time off policies ensure you have the chance to unwind and recharge when needed.

• Work with Autonomy: Enjoy the freedom to manage your time as long as the work gets done. Focus on results, not the clock.

• Work with Top American Companies: Grow your expertise working on innovative, high-impact projects with Industry-Leading U.S. Companies.

Why You’ll Like Working Here

• A Culture That Values You: We prioritize well-being and work-life balance, offering engagement activities and fostering dynamic teams to ensure you thrive both personally and professionally.

• Diverse, Global Network: Connect with over 600 professionals in 25+ countries, expand your network, and collaborate with a multicultural team from Latin America.

• Team Up with Skilled Professionals: Join forces with senior talent. All of our team members are seasoned experts, ensuring you're working with the best in your field.

Apply now!