Software Development Engineer in Test

Ease Health; New York (NY); On-site; USD 120,000 - 160,000; Full time

Job summary

Ease Health is seeking a highly technical, product-minded engineer to own testing of an AI-native behavioral health platform at scale while maintaining speed of delivery. You will design and maintain scalable automated testing frameworks across the stack and craft strategies for non-deterministic AI systems and autonomous agents in clinical workflows.

You’ll embed automation into CI/CD pipelines, partner with product and engineering on testability, and identify edge cases with a focus on billing

Qualifications

• No CS degree required; no fixed years of experience.
• Strong testing mindset with focus on AI/LLM evaluation.
• Ability to learn behavioral health workflows quickly.

Responsibilities

• Design and maintain scalable automated testing frameworks across the entire stack
• Develop testing strategies for non-deterministic AI systems, LLM outputs, and autonomous agents in clinical workflows
• Embed automation into CI/CD pipelines to ship quickly with confidence
• Partner with product and engineering on testability during feature development
• Identify edge cases, race conditions, and compliance gaps in billing and scheduling systems
• Help define testing philosophy and influence product direction

Skills

Engineering fundamentals

AI/LLM testing

Automation tooling

CI/CD integration

Edge case analysis

Ownership mindset

Tools

CI/CD tools

Test automation frameworks

Job description

This is a highly technical, product-minded engineering role. You will own how we test an AI-native behavioral health platform at scale, without sacrificing the speed we ship at.

What you'll do

• Design and maintain scalable automated testing frameworks across the entire stack
• Develop testing strategies for non-deterministic AI systems, LLM outputs, and autonomous agents in clinical workflows
• Embed automation into CI/CD pipelines so we can ship quickly with confidence
• Partner with product and engineering on testability while features are being built
• Identify edge cases, race conditions, and compliance gaps in billing and scheduling systems
• Help define our testing philosophy and influence product direction

What we're looking for

• Strong engineering fundamentals and a quality-first mindset
• Interest or experience in AI/LLM evaluation and testing autonomous agents
• Ability to learn behavioral health workflows quickly
• An automation bias and the initiative to build tools the whole team uses
• Comfort with ambiguity in an emerging area of testing
• An ownership mentality for defining testing philosophy at a startup

No CS degree required. No minimum years of experience. No brand-name résumé needed.