Senior AI Engineer (AI evaluation and agentic workflows) role at NVIDIA in Santa Clara, CA (onsite).
Responsibilities
- Evaluate emerging and established AI tools, agent harnesses, and models on representative NVIDIA software engineering workflows
- Compare task success, output quality, reliability, latency, cost, and enterprise readiness
- Identify where AI is the best tool; surface gaps and recommend AI-first approaches based on hands-on evaluation of modern AI-native tools
- Design, implement, and validate reference workflows for coding, testing, code review, and troubleshooting using tools such as Codex, Claude, Cursor, Greptile, and CodeRabbit
- Introduce technologies that enable massively parallel systems to improve turnaround time by an order of magnitude
- Optimize performance and cost by locating bottlenecks across training, evaluation, and testing; improve throughput, latency, and efficiency
- Partner with AI product vendors to gather deep industry insights and share findings with leaders and developers internally
- Work with core and emerging tool vendors to evaluate new capabilities, influence product roadmaps, resolve adoption issues, and add useful features into NVIDIA’s AI ecosystem
- Provide AI tool and model selection guidance, recommended configurations, workflow examples, onboarding materials, and hands-on training
Requirements
- MS in EE/CS or equivalent experience
- 12+ years of work experience
- Strong understanding of LLMs, machine learning, and agentic AI, including how models, agent harnesses, context, and tool use affect end-to-end task outcomes
- Hands-on experience evaluating AI tools or models and applying LLMs to software engineering workflows, including:
- experiment design
- benchmark development
- reproducibility
- failure analysis
- Strong software engineering skills in Python; experience with Java or Go and extensive scripting and automation experience
- Experience building evaluation pipelines, developer tooling, or full-stack applications, including:
- API integration
- data management
- deployment in cloud or enterprise environments
- Experience with CI/CD tooling such as Jenkins, Gitlab CI, Packer, Terraform, Artifactory, Ansible, Chef or similar tools
- Good understanding of distributed systems, including microservice architecture and REST APIs
- Familiarity with software development, testing, code review, and build workflows; ability to translate evaluation results into clear technical recommendations and practical guidance
- Ability to work across organizational boundaries to improve alignment and productivity between teams
Technologies
- AI
- Large language models (LLMs)
- Machine learning
- Agentic AI
- Codex
- Claude
- Cursor
- Greptile
- CodeRabbit
- Python
- Java
- Go
- Jenkins
- Gitlab CI
- Packer
- Terraform
- Artifactory
- Ansible
- Chef
- CI/CD
- Microservice architecture
- REST APIs
- Cloud
- API integration
Benefits
- Eligible for equity and benefits
Ways to stand out from the crowd
- Industry thought leader in AI, influenced AI ecosystem to deliver forward-looking solutions
- Expertise in agent harnesses, tool calling, MCP, retrieval-augmented generation (RAG), context management, or fine-tuning, and experience evaluating their impact on complex engineering workflows
- Experience rolling out AI tools across engineering teams, collaborating with vendors, and turning emerging capabilities into measurable adoption and productivity improvements
- Experience developing large-scale distributed tooling, evaluation services, or observability systems with real-time constraints
- Strong collaborative and interpersonal skills, with a record of guiding and influencing others in dynamic environments
Compensation: USD 224,000 - 356,500 per year.
Location: Santa Clara, CA (onsite).