Pioneering AI Quality & Release Engineer (Remote)

Wintermeyer Ventures; New York (NY); Remote; USD 170,000 - 200,000; Full time

Job summary

Wintermeyer Ventures is seeking a senior software quality engineer to own the full verification and release system for an AI/data startup. You will design agentic test harnesses, LLM graders, and CI pipelines, ensuring every build is trustworthy and releases are smooth.

With 7–10+ years in software or quality engineering and hands-on work with agentic AI, you will automate testing, improve gate controls, and drive reliable staging and production environments.

Qualifications

• 7 to 10+ years of software or quality engineering experience.
• 1 year hands-on building/testing evaluating agentic AI systems and LLM apps in the last year.
• Deep proficiency with GitHub Actions and GitHub CI/CD, including container builds and pipelines.
• Strong Python and TypeScript; comfortable with Docker, Kubernetes, and Helm.
• Familiarity with AWS, Snowflake, Databricks or Azure.
• Background in early-stage startups or modernizing QA/DevOps in larger orgs.
• Demonstrated ability to own systems end-to-end with minimal guidance.
• Authorized to work in the US or Canada without visa sponsorship now or in the future.

Responsibilities

• Design, build, and maintain agentic test harnesses and LLM-based evaluation systems that run tests automatically.
• Own regression coverage and release gates, stopping unsafe builds with evidence.
• Develop and improve CI pipelines, artifact publication workflows, and container build automation.
• Maintain reliable demo and staging environments that reflect production.
• Reduce bug-escape rates by improving automated signals rather than manual testing.
• Participate in business-hours incident escalations to reduce incidents over time.

Skills

GitHub Actions

Python

TypeScript

Docker

Kubernetes

Helm

AWS

Azure

Snowflake

Databricks

Job description

Wintermeyer Ventures is seeking a senior software quality engineer to own the full verification and release system for an AI/data startup. You will design agentic test harnesses, LLM graders, and CI pipelines, ensuring every build is trustworthy and releases are smooth.

With 7–10+ years in software or quality engineering and hands-on work with agentic AI, you will automate testing, improve gate controls, and drive reliable staging and production environments.