Senior AI Testing Engineer - Automation & LLM QA

Salesforce.com, inc.; San Francisco, Northern (CA, KY); Hybrid; USD 149,000 - 224,000; Full time

Job summary

Salesforce, a leader in AI-powered CRM, is seeking a Senior Software Development Engineer in Test to advance AI tooling and automation within the RAO team. You will design scalable testing frameworks, validate LLM-driven features, and drive quality across enterprise-grade sales forecasting and automation tools.

The role emphasizes AI/ML-powered product validation, collaboration with product managers, data scientists, and engineers, and delivering robust, reliable software in a fast-paced

Qualifications

• Strong Java fundamentals: data structures and algorithms.
• Hands-on experience evaluating Large Language Models (LLMs) and AI evaluation frameworks.
• Proven track record delivering automation in fast-paced CI/CD environments.
• Excellent collaboration and communication across product, AI, and engineering teams.
• Strong analytical and problem-solving abilities to identify quality gaps.

Responsibilities

• Design, build, and maintain scalable, AI-first testing frameworks and automation suites for RAO workflows.
• Translate quality strategies into actionable automation plans, validating LLM integrations and prompt accuracy.
• Perform white-box testing and help fix code and test flakiness.
• Support evaluation of subjective AI quality dimensions like relevance and safety.
• Collaborate with developers, product managers, and AI/ML teams to bake quality in early delivery.

Skills

Java fundamentals

AI/LLM validation

CI/CD automation

Cross-functional collaboration

Analytical problem-solving

Job description

Salesforce, a leader in AI-powered CRM, is seeking a Senior Software Development Engineer in Test to advance AI tooling and automation within the RAO team. You will design scalable testing frameworks, validate LLM-driven features, and drive quality across enterprise-grade sales forecasting and automation tools.

The role emphasizes AI/ML-powered product validation, collaboration with product managers, data scientists, and engineers, and delivering robust, reliable software in a fast-paced