Staff Software Engineer: AI Testing & Release Infrastructure

Recruiting From Scratch; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity package

Job summary

Recruiting From Scratch in San Francisco seeks a senior-to-staff software engineer to own end-to-end product quality by designing provider conformance testing and reliable CI infrastructure for an AI gateway.

You will develop automated testing strategies and release workflows to ensure stability across 100+ LLM providers without slowing engineering velocity. Preference for Python, AI infrastructure, and open-source contributions.

Qualifications

• Requires senior-to-staff level software engineering experience with proven track record building test and release infrastructure for complex production systems.
• Preference for candidates with strong Python skills and experience in AI infrastructure or maintaining large open-source projects.

Responsibilities

• Own end-to-end product quality by designing provider conformance testing and reliable CI infrastructure.
• Develop automated testing strategies and release workflows to ensure stability across 100+ LLM providers without slowing engineering velocity.

Skills

Python

CI/CD

Release Eng.

Integration

AI Infra

LLM Platforms

API Gateways

Cloud Infra

Automated Testing

Canary Testing

Synthetic Monitoring

Open Source

Root Cause

Developer Tooling

Production Reliability

System Architecture

Job description

Recruiting From Scratch in San Francisco seeks a senior-to-staff software engineer to own end-to-end product quality by designing provider conformance testing and reliable CI infrastructure for an AI gateway.

You will develop automated testing strategies and release workflows to ensure stability across 100+ LLM providers without slowing engineering velocity. Preference for Python, AI infrastructure, and open-source contributions.