AI-Driven SDET: Testing & Automation Architect

Socket.dev; San Francisco (CA); On-site; USD 149,000 - 224,000; Full time

Job summary

Salesforce in San Francisco is seeking a Software Quality Senior Member of Technical Staff to lead AI-first testing, validate LLM integrations, and push quality across the agile delivery lifecycle. You will design scalable testing frameworks, translate strategies into automation plans, perform white-box testing, and collaborate with engineers, product managers, and data scientists to raise product quality.

This role offers strong career growth, a competitive salary in the SF Bay Area, and a

Qualifications

• Strong Java fundamentals in data structures and algorithms.
• Hands-on evaluation of LLMs and AI validation frameworks.
• Proven ability to deliver automation in a fast-paced CI/CD environment.
• Excellent cross-functional collaboration and communication.
• Analytical problem-solving to identify quality gaps.

Responsibilities

• Design and maintain scalable AI-first testing frameworks.
• Translate quality strategies into automation plans targeting LLMs.
• Perform white-box testing and fix code/test flakiness.
• Implement evaluation processes for subjective AI quality.
• Collaborate with developers, PMs, engineers to embed quality early.

Skills

Java

AI/LLM Validation

CI/CD

Collaboration

Problem-Solving

Job description

Salesforce in San Francisco is seeking a Software Quality Senior Member of Technical Staff to lead AI-first testing, validate LLM integrations, and push quality across the agile delivery lifecycle. You will design scalable testing frameworks, translate strategies into automation plans, perform white-box testing, and collaborate with engineers, product managers, and data scientists to raise product quality.

This role offers strong career growth, a competitive salary in the SF Bay Area, and a