Relational AI Product Analyst

Sproutai; Raleigh (NC); On-site; USD 70,000 - 95,000; Full time

Job summary

Sprout AI is seeking a hands-on AI Product Analyst to dive into building AI systems across multiple ventures. You'll be the go-to expert for prompt engineering, evaluations, and performance optimization, crafting prompts, testing tools, and designing evaluation pipelines.

You’ll work with Cursor, Claude Code, v0, or Replit, expanding your technical skills while helping our ventures refine AI products in a fast-paced studio environment.

Qualifications

• 1-3 years of experience with AI systems and data analysis tools.
• Strong prompt engineering experience with large language models.
• Understanding of testing, performance measurement, and AI evaluation platforms.
• Willingness to learn in a fast-paced venture studio.
• Good problem-solving and multi-project context-switching abilities.
• Excitement for using AI to enhance human relationships and communities.
• Collaborative communication across technical and non-technical stakeholders.

Responsibilities

• Build and maintain evaluation frameworks to measure AI model effectiveness across multiple ventures
• Design and manage prompt testing systems with comprehensive performance tracking
• Craft and iterate on prompt templates through systematic testing and optimization
• Analyze fine-tuning results and optimization strategies including RLHF and other techniques
• Design A/B testing frameworks and automated testing pipelines for different AI configurations
• Partner with Product Leads and Founders to translate product requirements into high-performing AI models
• Build reusable evaluation frameworks and testing approaches that accelerate development

Skills

Prompt engineering

Large language models

AI evaluation

Data analysis tools

Basic programming

Tools

Arize

Langsmith

Braintrust

Job description

Sprout AI is seeking a hands-on AI Product Analyst to dive into building AI systems across multiple ventures. You'll be the go-to expert for prompt engineering, evaluations, and performance optimization, crafting prompts, testing tools, and designing evaluation pipelines.

You’ll work with Cursor, Claude Code, v0, or Replit, expanding your technical skills while helping our ventures refine AI products in a fast-paced studio environment.