Staff Software Engineer, Product Quality

Recruiting from Scratch; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive equity package

Onsite in San Francisco

Job summary

Recruiting from Scratch is representing a fast-growing AI infrastructure company behind a widely used LLM gateway. This Staff Software Engineer will own quality across testing, CI, release infrastructure, and provider integrations—building systems that catch regressions without slowing velocity.

You'll stay hands-on, shaping testing strategies, tooling, and release workflows for 100+ LLM APIs, working onsite in San Francisco with a deeply technical IC mindset and a mission to raise the quality

Qualifications

• Senior-to-Staff level software engineer with production systems ownership.
• Experience in a Seed through Series D startup environment.
• Proven track record building test, CI, or release infrastructure.
• Experience with large external API surfaces and provider integrations.
• Strong debugging and root-cause analysis skills.
• Experience designing automated test suites trusted to gate production releases.
• Understanding of CI/CD, release engineering, and production reliability.
• Open-source contributions are a plus.

Responsibilities

• Own product quality end-to-end across a platform integrating with 100+ LLM providers.
• Design and build provider conformance testing (streaming, tool calling, structured outputs, auth, error mapping, cost tracking).
• Build and own CI infrastructure that serves as a release gate.
• Develop automated testing strategies for high open-source contribution volume.
• Build canary testing and synthetic traffic systems to identify upstream changes.
• Design release infrastructure enabling rapid daily shipping with enterprise stability.
• Debug complex failures across unfamiliar code paths, third-party integrations, and provider APIs.
• Perform deep root-cause analysis and convert defects into regression coverage.
• Improve testing architecture, tooling, and release workflows without slowing velocity.
• Raise the technical bar through hands-on coding, architecture decisions, and code reviews.

Skills

Quality ownership

CI/CD

Testing infrastructure

Open-source experience

API integration

Debugging

Release engineering

Platform reliability

Technical ownership

Multi-provider testing

Tools

CI tools

Job description

Recruiting from Scratch is a specialized talent firm dedicated to helping companies build exceptional teams. We partner closely with our clients to deeply understand their needs, then connect them with top-tier candidates who are not only highly skilled but also the right fit for the company’s culture and vision. Our mission is simple: place the best people in the right roles to drive long-term success for both clients and candidates.

https://www.recruitingfromscratch.com/

Staff Software Engineer, Product Quality

Company Stage of Funding: Startup

Office Type: Onsite

Company Description

We're representing a fast-growing AI infrastructure company behind one of the world's most widely used LLM gateways. Its open-source platform provides a standardized interface for accessing 100+ LLM providers and is trusted by major organizations across technology, media, government, and enterprise environments.

With such a broad integration surface and significant open-source contribution volume, product quality is a core infrastructure problem. The company is hiring a Staff Engineer to own quality across testing, CI, release infrastructure, and provider integrations—building systems that automatically catch regressions and upstream model-provider changes without slowing engineering velocity.

This is a deeply technical individual contributor role for an engineer who wants to remain in the code while setting the quality bar across a high-scale AI developer platform.

What You Will Do

• Own product quality end-to-end across a platform integrating with more than 100 LLM providers.
• Design and build comprehensive provider conformance testing covering streaming, tool calling, structured outputs, authentication, error mapping, and cost tracking.
• Build and own CI infrastructure that is fast, deterministic, and reliable enough to serve as a true release gate.
• Develop automated testing strategies that accommodate a large volume of external and open-source contributions.
• Build canary testing and synthetic traffic systems that identify upstream provider changes before they impact customers.
• Design release infrastructure that enables rapid daily shipping while maintaining stable release channels for enterprise customers.
• Debug complex failures across unfamiliar code paths, third-party integrations, and provider APIs.
• Perform deep root-cause analysis and convert escaped production defects into permanent regression coverage.
• Improve testing architecture, developer tooling, and release workflows without introducing unnecessary process or slowing engineering velocity.
• Raise the technical bar through hands‑on coding, architecture decisions, debugging, and code reviews.

Ideal Background

• Senior-to-Staff level software engineering experience with deep technical ownership of production systems.
• Experience working at a Seed through Series D startup and comfort operating across multiple areas as priorities change.
• Proven experience building test, CI, release, or developer infrastructure for complex production software.
• Experience supporting products with a large external integration or API surface.
• Strong debugging ability with a track record of identifying underlying root causes rather than patching symptoms.
• Experience designing automated test suites engineers trust enough to gate production releases.
• Strong understanding of CI/CD, release engineering, integration testing, and production reliability.
• Track record of improving engineering quality without becoming a bottleneck to shipping.
• Experience contributing to or maintaining open-source software.
• Comfortable taking organization-level technical ownership as an individual contributor.

Preferred

• Experience maintaining CI or release infrastructure for a large open-source project with significant outside contribution volume.
• Experience building infrastructure around third-party APIs where upstream behavior can change unexpectedly.
• Background in AI infrastructure, LLM platforms, developer tools, API gateways, or cloud infrastructure.
• Experience testing LLM functionality including streaming, tool/function calling, structured outputs, and model-provider error handling.
• Experience designing canary releases, synthetic monitoring, or automated production validation systems.
• Familiarity with multi-channel release strategies balancing rapid iteration with enterprise stability.
• Meaningful GitHub or other public open-source contributions.
• Strong opinions about testing architecture, engineering quality, and what makes automated test infrastructure genuinely trustworthy.

Compensation and Benefits

• Competitive equity package.
• Onsite position in San Francisco.
• Deep technical IC role with ownership over product quality rather than a process-heavy QA function.
• Opportunity to build quality and release infrastructure for an AI gateway supporting 100+ LLM APIs.
• Work on infrastructure used by major organizations while contributing to a highly visible open-source ecosystem.
• High-ownership startup environment where engineering quality is driven through code, automation, and strong technical judgment rather than additional process.