Staff Quality Systems Engineer — Daily Release Gatekeeper

Comfy; San Francisco (CA); On-site; USD 180,000 - 220,000; Full time

Job summary

Comfy, the AI creation engine for visual professionals, seeks a Senior QA Engineer to build the infrastructure that validates daily releases. You’ll define what “correct” means, implement gates, and automate testing across 60,000+ community nodes to keep shipping safely.

In San Francisco, you’ll help move from weekly to daily releases, developing scalable evaluation Harnesses with property-based testing, perceptual checks, and AI-assisted validation.

Qualifications

• Have built testing or verification infrastructure that other engineers depended on.
• Experience with property-based testing, fuzzing, or evaluation harnesses.
• Experience gating releases and measuring flaky tests.

Responsibilities

• Own what "correct" means for ComfyUI and release gates.
• Design property-based and generative testing.
• Solve the combinatorial problem with many community nodes and ensure coverage of real user journeys.
• Solve the visual correctness problem with canvas rendering, layout, and linking; implement perceptual diffing.
• Address non-determinism by modeling outputs with seeds and diverse hardware.
• Build custom node CI to verify third-party nodes continuously.
• Use AI agents in testing and verify their outputs with safeguards.
• Publish metrics to show quality trends and own them in the team.

Skills

TypeScript

Python

CI/CD

Testing infra

Tools

Playwright

Vitest

Storybook

Codecov

Job description

Comfy, the AI creation engine for visual professionals, seeks a Senior QA Engineer to build the infrastructure that validates daily releases. You’ll define what “correct” means, implement gates, and automate testing across 60,000+ community nodes to keep shipping safely.

In San Francisco, you’ll help move from weekly to daily releases, developing scalable evaluation Harnesses with property-based testing, perceptual checks, and AI-assisted validation.