AI Quality & Agent Operations Specialist – Production Ready

Nvidia Corporation in; Santa Clara (CA); On-site; USD 124,000 - 196,000; Full time

Job summary

NVIDIA is seeking an AI Quality and Agent Operations Analyst in AI Solutions (Finance) to strengthen AI evaluation and operational practices across WWFO. You will work with cross-functional teams to replace ad hoc testing with standardized AI QA workflows.

Responsibilities include developing test scenarios, metrics, and reports; evaluating AI outputs for accuracy and safety; and supporting production readiness, launches, and continuous improvement.

Qualifications

• Experience testing, evaluating, monitoring, or supporting applications in development or production.
• Familiarity with generative AI, ML concepts, APIs, data workflows, or enterprise software.
• Experience analyzing test results, metrics, user feedback, or other data to identify issues.
• Ability to develop structured test cases, evaluation criteria, and quality reports.
• Strong collaboration with Engineering, Product, Data and business teams.

Responsibilities

• Collaborate with teams to replace ad hoc testing with consistent AI evaluation standards.
• Establish test scenarios, quality metrics, and acceptance thresholds.
• Evaluate AI outputs for accuracy, relevance, and safety against workflows.
• Analyze production data, service metrics, and user feedback to monitor quality.
• Investigate failures and communicate findings to technical and business teams.
• Support production readiness and launches with testing, validation, and documentation.

Skills

QA testing

Data analysis

AI/ML concepts

Python

APIs

SQL

Dashboard tools

Education

Bachelor's degree in CS or related field

Tools

Python

SQL

APIs

Dashboard tools

Job description

NVIDIA is seeking an AI Quality and Agent Operations Analyst in AI Solutions (Finance) to strengthen AI evaluation and operational practices across WWFO. You will work with cross-functional teams to replace ad hoc testing with standardized AI QA workflows.

Responsibilities include developing test scenarios, metrics, and reports; evaluating AI outputs for accuracy and safety; and supporting production readiness, launches, and continuous improvement.