Senior QA Engineer — ML/LLM End-to-End Testing & Eval

Solstice; New York (NY); On-site; USD 160,000 - 300,000; Full time

Benefits offered by this job

Health, dental, and vision insurance

Ground-floor equity opportunity

401(k) w/ match

Visa sponsorship available

Professional growth stipend

Relocation support

Job summary

Solstice is seeking a dedicated QA lead to manage quality for AI-powered pharmaceutical marketing content. This senior engineering role demands expertise in building test infrastructure and conducting both automated and manual tests.

You will work on developing evaluation systems, ensuring product reliability, and implementing CI/CD practices. Other responsibilities include designing tests for non-deterministic systems and running exploratory QA.

Benefits include health insurance, equity opportunities, 401(k) with match, and competitive NYC compensation ranging from $160,000 to $300,000.

Qualifications

• Experience building testing and evaluation tools in code.
• Capability in hands-on manual testing.
• Strong experience with CI/CD.

Responsibilities

• Build and maintain testing and evaluation systems.
• Design tests to ensure quality in a probabilistic system.
• Conduct manual testing for edge cases and release sign-off.

Skills

Strong Python

End-to-end and UI test automation

Manual and exploratory QA discipline

Experience testing non-deterministic systems

Evaluation methods

Statistical thinking about quality

Error analysis instinct

Clear communication

Work ethic

Tools

Playwright

TypeScript

React

Job description

Solstice is seeking a dedicated QA lead to manage quality for AI-powered pharmaceutical marketing content. This senior engineering role demands expertise in building test infrastructure and conducting both automated and manual tests.

You will work on developing evaluation systems, ensuring product reliability, and implementing CI/CD practices. Other responsibilities include designing tests for non-deterministic systems and running exploratory QA.

Benefits include health insurance, equity opportunities, 401(k) with match, and competitive NYC compensation ranging from $160,000 to $300,000.