Software QA Analyst

Jobtailor; Zionsville (IN); On-site; USD 65,000 - 95,000; Full time

Job summary

Jobtailor in the United States (Indiana) is seeking a QA engineer to test our services, APIs, and AI-driven workflows. You will design and run test plans, review agent outputs, and help define what done well looks like before new work ships.

The role emphasizes API test automation with Postman, pytest, and httpx, plus Python or TypeScript. Strong communication and attention to detail are essential, as is comfort with Git, SQL basics, and Linux shell.

Qualifications

• Experience in software testing, QA tooling, or API test automation.
• Ability to read code across Python or TypeScript.
• Familiarity with Git, HTTP/REST APIs, basic SQL, and Linux shell.
• Ability to document test cases, failure patterns, and improvement proposals.
• Proactive communication about issues and questions with minimal prompting.

Responsibilities

• Test services, APIs, and integration flows across control plane and webhook components.
• Review agent outputs and assess accuracy, relevance, and quality.
• Write and maintain test cases and evaluation criteria for APIs and end-to-end flows.
• Identify failure patterns and log them clearly.
• Collaborate with platform team to implement fixes and guardrails.
• Own tickets from test plan to verified improvements.
• Participate in design reviews to define done-well criteria.

Skills

Software Testing

API Test Automation

Python

TypeScript

Basic SQL

HTTP/REST APIs

Linux Shell

Test Case Writing

Failure Pattern Identification

CI/CD Exposure

Soft Skills: Clear Communication

Soft Skills: Attention To Detail

Soft Skills: Proactive Problem Solving

Tools

Postman

pytest

httpx

Git

Job description

Responsibilities

• Test the services we build — control plane APIs, intake registry, agent dispatch flows, webhook integrations — at the API and integration level.
• Review agent output — code review comments, generated tests, draft MRs — and assess accuracy, relevance, and quality against the codebase and context.
• Write and maintain test cases and evaluation criteria: API contracts, end-to-end flows, agent output quality for each agent type (reviewer, tester, builder).
• Identify failure patterns across the stack: broken API responses, dropped state, hallucinations, missed issues, bad diffs. Log them clearly.
• Work with the platform team to translate findings into fixes, guideline improvements, and guardrail changes.
• Own tickets end-to-end: test plan → findings → documented change → verified improvement.
• Sit in on design reviews; help define what "done well" looks like before work ships.

Requirements

• 1-3 years of professional experience with software testing, QA tooling, or API test automation (Postman, pytest, httpx, or similar).
• Attention to detail and the ability to read code across at least one language (Python or TypeScript preferred).
• Comfortable with Git, HTTP/REST APIs, basic SQL, and Linux shell.
• Writes things down clearly — test cases, failure patterns, improvement proposals.
• Asks questions and communicates what's wrong without waiting to be asked.
• Familiarity with LLMs or AI agents (Nice-to-haves).
• Python or TypeScript & any CI/CD exposure (Nice-to-haves).
• AWS basics (ECS, CloudWatch, Secrets Manager) (Nice-to-haves).

Hard Skills

• Software Testing
• API Test Automation
• Python
• TypeScript
• Basic SQL
• HTTP/REST APIs
• Linux Shell
• Test Case Writing
• Failure Pattern Identification
• CI/CD Exposure

Soft Skills

• Clear Communication
• Attention To Detail
• Proactive Problem Solving