Senior AI Tooling & Automation Engineer

salesforce.com, inc.; San Francisco (CA); On-site; USD 178,900 - 246,000; Full time

Job summary

Salesforce is seeking a Senior Software Quality SMTS to drive AI-first quality for Product Led Growth initiatives. You will define quality strategies, collaborate with product and AI teams, and ensure enterprise-grade testing across self-service and growth features.

Bring expertise in validating LLMs, building scalable automation, and improving code quality in a fast-paced CI/CD environment. This role emphasizes cross-functional collaboration and driving testing maturity.

Qualifications

• Strong Java skills with fundamentals of data structures and algorithms.
• AI/LLM validation mindset with experience evaluating Large Language Models (LLMs) and familiarity with GPT-like models and AI evaluation frameworks.
• Agile delivery in CI/CD environments, enabling rapid deployment without compromising quality.
• Excellent collaboration and communication across product managers, engineers, AI/ML engineers, and data scientists.
• Strong analytical and problem-solving abilities focused on identifying and closing quality gaps.

Responsibilities

• Design, build, and maintain scalable AI-first testing frameworks and automation to validate PLG workflows and agentic capabilities.
• Translate quality strategies into actionable automation plans, emphasizing LLM integrations, prompt accuracy, and deterministic behaviors.
• Perform white-box testing and contribute to fixing code and test flakiness.
• Implement evaluation processes for AI features, including relevance, safety, and contextual accuracy.
• Collaborate with cross-functional teams to ensure quality is integrated early in the agile delivery lifecycle.

Skills

Java

AI/LLM validation

Agile delivery

Collaboration & communication

Problem solving & analytics

Tools

Public cloud (AWS/Azure/GCP)

Job description

Salesforce is seeking a Senior Software Quality SMTS to drive AI-first quality for Product Led Growth initiatives. You will define quality strategies, collaborate with product and AI teams, and ensure enterprise-grade testing across self-service and growth features.

Bring expertise in validating LLMs, building scalable automation, and improving code quality in a fast-paced CI/CD environment. This role emphasizes cross-functional collaboration and driving testing maturity.