(UNPAID - Volunteer role ) Sr. Testing AI Manager

Fact Finders Pro; Seattle (WA); Remote; USD 80,000 - 120,000; Part time

Job summary

Ein innovatives Unternehmen sucht einen Senior AI Testing Manager, um ein Team von Testingenieuren zu leiten und Strategien für generative AI-Tests zu entwickeln. Diese Rolle erfordert umfangreiche Erfahrung in der Qualitätssicherung und der Leitung von Teams. Sie werden für die Gestaltung von Testmethoden verantwortlich sein, die für probabilistische AI-Anwendungen geeignet sind. In einem dynamischen Umfeld haben Sie die Möglichkeit, entscheidende Beiträge zur Verbesserung der Testpraktiken zu leisten und innovative Lösungen zu entwickeln. Wenn Sie eine Leidenschaft für Technologie und AI haben, ist dies die perfekte Gelegenheit für Sie.

Qualifications

• 7+ Jahre Erfahrung in der Qualitätssicherung.
• Fähigkeit, Teststrategien für AI-Umgebungen zu entwerfen.

Responsibilities

• Erstellen von Testmethoden für probabilistische AI-Systeme.
• Zusammenarbeit mit Ingenieuren zur Implementierung von Testautomatisierung.

Skills

Qualitätssicherung

AI Testing

Testautomatisierung

Analytische Fähigkeiten

Führungskompetenzen

Education

Bachelor in Informatik

Tools

Selenium

Job description

3 days ago Be among the first 25 applicants

Direct message the job poster from Fact Finders Pro

SHRM CP / People Analytics / Recruitment Specialist

Job Title: Senior AI Testing Manager

Location: Remote

Commitment: Part-time (15-20 hours per week)

About Fact Finders Pro (FFP):

Fact Finders Pro (FFP) is dedicated to combating misinformation and fostering a trustworthy digital landscape. We empower individuals, communities, and organizations with tools to discern facts from fake news.

Role Overview:

We are seeking a Senior AI Testing Manager responsible for managing the test engineering team and designing a generative AI-oriented test strategy and plan. The ideal candidate must have experience with traditional testing practices and applications with hard-coded business rules, as well as the ability to test AI-driven, probabilistic applications (AI agents). This includes designing tests that can identify correct application behavior when outputs vary probabilistically across repeated tests with the same inputs, and understanding how to specify expected results when applications emit probabilistic outputs.

Responsibilities:

• Create testing methodologies suited for probabilistic, LLM-driven, agentic AI systems, including:
• Determining output correctness when application outputs differ on repeated executions.
• Assessing output correctness when exact expected outputs cannot be specified in advance.
• Collaborate with application and machine learning engineers to design and implement test automation tools for testing AI agents.
• Review requirements and technical design documents to provide feedback.
• Identify testable requirements and prepare test strategies.
• Write comprehensive test plans for functional, regression, acceptance, and stress testing.
• Perform functional, regression, and performance testing.
• Estimate, prioritize, plan, and coordinate testing activities.
• Assist in post-production support, such as end-user support and troubleshooting.

Qualifications:

• Bachelor’s degree in computer science, information technology, or related field.
• 7+ years of quality assurance engineering experience.
• 3+ years of quality assurance team management experience.
• Familiarity with conventional statistics and probability theory relevant to testing AI systems.
• Experience designing and implementing test strategies, plans, and cases for AI environments.
• Experience with test automation tools like Selenium, adapted for AI testing.
• Experience in experimentation within private sandboxes.
• Excellent organizational, time management, analytical, and problem-solving skills.
• Strong supervisory and leadership skills.
• Comfort working in a start-up environment is a plus.

Seniority level

• Mid-Senior level

Employment type

Job function

• Quality Assurance

Industries

• Technology, Information and Internet

Referrals increase your chances of interviewing at Fact Finders Pro by 2x

Get notified about new Senior Test Manager jobs in Seattle, WA.

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.