Senior AI Testing &Automation Engineer

Salesforce; San Francisco (CA); On-site; USD 179,000 - 246,000; Full time

Job summary

Salesforce is pursuing a Senior Member of Technical Staff (SMTS) to advance AI-first quality in the Revenue Action Orchestration domain. You will design and scale QA frameworks, validate AI-powered features, and partner with product, data science, and engineering teams to ensure enterprise-grade reliability.

The role emphasizes strong Java fundamentals, LLM evaluation, and CI/CD automation, with collaboration across a broad AI/ML ecosystem to drive velocity and accuracy. On-site in SF is typical.

Qualifications

• Strong Java fundamentals, data structures and algorithms.
• Experience evaluating Large Language Models (LLMs) and AI evaluation frameworks.
• Proven CI/CD automation experience in a fast-paced environment.
• Excellent collaboration and communication across functions.
• Analytical problem-solving with focus on quality gaps.

Responsibilities

• Design, build, and maintain AI-first testing frameworks and automation for RAO workflows.
• Translate quality strategies into automation plans, focusing on LLM integrations and prompt accuracy.
• Perform white-box testing and contribute to fixing code and test flakiness.
• Implement evaluation processes for subjective AI quality dimensions.
• Collaborate with product, data science, and AI teams to ensure early quality integration.

Skills

Java

AI/LLM validation

Agile delivery

Collaboration & communication

Problem-solving

Tools

AWS

Azure

GCP

Job description

Salesforce is pursuing a Senior Member of Technical Staff (SMTS) to advance AI-first quality in the Revenue Action Orchestration domain. You will design and scale QA frameworks, validate AI-powered features, and partner with product, data science, and engineering teams to ensure enterprise-grade reliability.

The role emphasizes strong Java fundamentals, LLM evaluation, and CI/CD automation, with collaboration across a broad AI/ML ecosystem to drive velocity and accuracy. On-site in SF is typical.