Software quality engineer (US)

WRITER; San Francisco (CA); Hybrid; USD 132,100 - 258,400; Full time

Benefits offered by this job

Generous PTO

Medical, dental, and vision coverage

Paid parental leave

Flexible spending account

Health savings account

Annual wellness stipend

Company stock options

Job summary

WRITER is looking for an experienced Software Quality Engineer to ensure the highest quality for our AI-powered work orchestration platform. This role involves defining quality assurance strategies and developing automation frameworks while monitoring AI model performance.

The ideal candidate will have over 5 years of software quality assurance experience in AI/ML applications, proficiency in Python or TypeScript, and familiarity with cloud services and containers. Benefits include generous PTO, medical coverage, and competitive compensation.

Qualifications

• 5+ years of experience in software quality assurance focused on AI/ML applications.
• Experience with scalable automation frameworks and modern testing practices.
• Understanding of the challenges of testing generative AI outputs.

Responsibilities

• Define and implement quality assurance strategies for AI applications.
• Develop automation frameworks from scratch for testing.
• Monitor AI performance and contribute to post-release validation.

Skills

Quality assurance

Test automation

Python

TypeScript

GitHub Actions

AI/ML concepts

Analytical skills

Tools

Playwright

Docker

Kubernetes

AWS

Azure

GCP

Job description

About the Role

Are you passionate about ensuring the highest quality for cutting‑edge generative AI applications? As a software quality engineer at WRITER, you'll play a critical role in shaping the reliability, performance, and trustworthiness of our AI‑powered work orchestration platform. You will define and implement rigorous quality strategies for our enterprise‑grade LLMs and AI agents, directly impacting how hundreds of global companies unlock transformational value through AI. This is a unique chance to dive deep into the unique challenges of AI quality assurance and make a tangible difference in a rapidly evolving field.

This is a hybrid role based out of our London, San Francisco, Seattle, and New York City hubs. You will report directly to the director of engineering.

What You'll Do

• Define and implement comprehensive quality assurance strategies and test plans for our AI agents and LLM‑powered applications, ensuring exceptional product reliability and performance.
• Design and develop automation frameworks: create robust, scalable, and maintainable automated test frameworks from scratch or enhance existing ones, with proficiency in at least one language such as TypeScript or Python.
• Collaborate closely with product managers, machine learning engineers, and data scientists to understand complex AI features and model behaviors, translating them into effective test cases and validation criteria.
• Drive continuous improvement of our testing processes and infrastructure, integrating automated checks within our CI/CD pipelines to ensure rapid, high‑quality releases.
• Identify, document, and track software defects and inconsistencies, performing root cause analysis to provide actionable feedback to development teams.
• Monitor production systems and AI model performance, proactively identifying potential issues and contributing to post‑release quality validation.
• Champion quality best practices across engineering teams, fostering a culture of ownership and continuous improvement in delivering world‑class AI solutions.
• Design and maintain test data strategies and mock services to ensure stable, isolated, and repeatable test execution.
• Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol (MCP).
• Manage the full defect lifecycle by analyzing customer feedback and debugging logs to identify, prioritize, and track software bugs, collaborating closely with development teams to ensure timely resolution.

What You Need

• 5+ years of hands‑on experience in software quality assurance or engineering, focusing on testing complex distributed systems or AI/ML applications.
• 5+ years of experience working with GitHub Actions.
• Proficiency in programming languages such as Python or TypeScript for test automation, and experience with modern testing frameworks such as Playwright.
• Solid understanding of AI/ML concepts, including model evaluation metrics, data pipelines, and the unique challenges of testing generative AI outputs.
• Experience with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) in a CI/CD environment is a plus.
• Exceptional analytical skills to dissect complex problems and a keen eye for detail, ensuring no bug goes unnoticed.
• A collaborative spirit that helps connect cross‑functional teams to common quality goals, challenge existing assumptions, and own the end‑to‑end quality of our products.
• Demonstrated ability to drive initiatives independently and thrive in a fast‑paced, evolving startup environment.

Benefits & Perks (US Full‑time Employees)

• Generous PTO, plus company holidays
• Medical, dental, and vision coverage for you and your family
• Paid parental leave for all parents (16 weeks)
• Fertility and family planning support
• Early‑detection cancer testing through Galleri
• Flexible spending account and dependent FSA options
• Health savings account for eligible plans with company contribution
• Annual work‑life stipends for:
• Wellness stipend for gym, massage/chiropractor, personal training, etc.
• Learning and development stipend
• Company‑wide off‑sites and team off‑sites
• Competitive compensation, company stock options, and 401k

WRITER is an equal‑opportunity employer and is committed to diversity. We don't make hiring or employment decisions based on race, color, religion, creed, gender, national origin, age, disability, veteran status, marital status, pregnancy, sex, gender expression or identity, sexual orientation, citizenship, or any other basis protected by applicable local, state or federal law. Under the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

By submitting your application on the application page, you acknowledge and agree to WRITER's Global Candidate Privacy Notice.

Compensation Range: $132,100 - $258,400