Ekko is an AI-powered healthcare product that turns clinical conversations into structured documentation. Our pipeline transcribes audio, generates clinical notes, and extracts structured data using LLMs. We ship across three surfaces — a web app, a backend service, and a browser extension — used by clinicians who depend on accuracy. We're a small, early-stage team.
This is Ekko's inaugural Senior QA Engineer position. You won't be walking into a mature QA process — you'll define it. From week one you'll be hands-on writing tests, not drawing up strategy decks. You'll report directly to the CTO and work alongside a junior QA engineer, setting the quality bar for a fast-moving clinical AI product.
Architect QA Processes
- Define testing strategy across unit, integration, end-to-end, and AI output validation
- Establish standards for test design, defect triage, release gates, and regression coverage
- Set up CI/CD-integrated test pipelines
Build Automation Suite
- Write Playwright-based end-to-end tests for web app and browser extension
- Develop API test coverage for backend services
- Implement SQL-based data validation for clinical workflows
Validate AI Output Quality
- Design evaluation frameworks for LLM pipeline performance
- Build automated comparisons between AI outputs and ground-truth references
- Define and track clinically-relevant quality metrics
Mentor & Partner
- Pair with a junior QA engineer on test design and automation
- Review test scripts and support team development
- Embed quality earlier in development cycles
- Conduct defect triage, root cause analysis, and advocate for quality in sprint planning
What You Bring
- **5+ years in QA,** with at least 2 years in a senior or lead role
- **Strong hands-on automation** with Playwright (preferred) or Selenium, in TypeScript or Python
- **API and database testing**: REST APIs, JSON schema validation, SQL backend checks
- **AI/LLM output validation experience** — you've tested generative AI outputs and know the difference between "passes the test" and "is actually correct." Familiarity with techniques like semantic similarity, structured output validation, RAGAS, or DeepDiff-style comparison
- **Experience defining QA processes from scratch**, not just executing within an existing one
- **Track record of mentoring junior QAs** or owning quality across a small team
- **Comfort with a startup pace** — ambiguity, ownership, and shipping over perfection
Nice to Have
- Healthcare or clinical software background
- Compliance knowledge (HIPAA, NABH, DPDP)
- Browser extension testing experience
- Collaboration with AI/ML engineers
- CI/CD experience (GitHub Actions)
Why join Ekko
- Real ownership. You're Ekko's inaugural QA leader — you define the quality bar, not inherit it.
- AI product depth. You'll tackle testing problems most QA roles never touch — LLM output validation in a live clinical setting.
- Direct impact. Small team, fast feedback, decisions made in days not quarters.
- Good people. We take the work seriously and ourselves a lot less so.