Remote Python QA Lead — Drive AI Training Quality

YO IT Consulting; Indiana (PA); On-site; USD 82,656 - 165,312; Part time

Job summary

YO IT Consulting seeks a Python Quality Assurance Lead for a remote, hourly contractor role. You will oversee quality across Python AI training projects, review AI-generated Python code and trainer work, and provide precise written feedback to ensure adherence to project rubrics and guidelines.

As a lead, you will spot quality issues, document improvements, collaborate with remote trainers and QAs, maintain documentation and onboarding materials, and help escalate blockers.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Science, Information Technology, or equivalent professional software engineering experience.
• Strong grasp of English to follow guidelines and provide clear technical feedback.
• 3+ years of professional experience in Python development, backend engineering, automation, scripting, data workflows, code review, QA, teaching, or mentoring.
• Strong understanding of Python fundamentals: data structures, functions, classes, modules, exceptions, and testing.
• Ability to evaluate Python content against rubrics and identify issues.
• Familiarity with pytest, unittest, typing/mypy, pip/poetry, virtualenv, FastAPI/Flask/Django, requests, asyncio, pandas, SQLAlchemy, GitHub, Docker, and CI/CD is preferred.
• Experience leading or supporting remote teams of trainers, engineers, reviewers, annotators, educators, or QAs is preferred.
• Comfortable with Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Highly organized and able to maintain style guides, onboarding materials, and documentation.
• Experience with AI training, data annotation, LLM evaluation, code QA, or rubric-based code review is a strong plus.

Responsibilities

• Spot-check Python items, identify quality issues, provide feedback, and elevate recurring or critical issues.
• Review AI-generated Python code, debugging responses, algorithms, backend snippets, automation scripts, tests, and explanations.
• Update trainers/QAs on Discord about guidelines, workflow updates, and Python-specific quality expectations.
• Respond to questions around Python syntax, runtime behavior, exceptions, package usage, testing, typing, security, performance, and rubric interpretation.
• DM inactive contributors, encourage activation, track follow-ups, and flag availability issues.
• Create and maintain Python documentation, style guides, trackers, FAQs, examples, and onboarding materials.
• Run onboarding/training calls for Python contributors.
• Flag misleading, non-executable, insecure, or non-production-ready Python recommendations.
• Identify recurring quality gaps and improve Python QA workflows.

Skills

Python development

QA / code review

Remote team leadership

Rubric-based evaluation

Automation / scripting

English communication

Organization & documentation

Mentoring remote teams

Education

Bachelor’s or Master’s degree in CS/SE/Data Science/IT

Tools

pytest

unittest

typing / mypy

pip / poetry

virtualenv

FastAPI

Flask

Django

requests

asyncio

pandas

SQLAlchemy

GitHub

Docker

CI/CD

Job description

YO IT Consulting seeks a Python Quality Assurance Lead for a remote, hourly contractor role. You will oversee quality across Python AI training projects, review AI-generated Python code and trainer work, and provide precise written feedback to ensure adherence to project rubrics and guidelines.

As a lead, you will spot quality issues, document improvements, collaborate with remote trainers and QAs, maintain documentation and onboarding materials, and help escalate blockers.