AI Model Benchmarking Intern (Remote, Iowa-Based)

Braintrust; United States; Remote; USD 30,000 - 39,000; Part time

Job summary

SureHost, a Des Moines, Iowa-based hospitality tech company, seeks an AI Model Benchmarking Intern. This remote, Iowa-focused internship involves evaluating LLMs and VLMs, building benchmarks, and reporting findings to guide model choices and roadmap.

You will work closely with the AI Chief Architect, perform repeatable experiments, and document methods and results. This role offers real responsibility on a live product and mentorship on applied model evaluation.

Qualifications

• Interest in applied AI and familiarity with LLMs, VLMs, generative AI tools, or model APIs through coursework or projects.
• Logical, evidence-based thinking and comfort working with structured data and evaluation criteria.
• Basic Python or data-analysis skills.
• The ability to document methods and results clearly.
• Patience to run repeatable experiments, investigate edge cases, and separate a compelling demo from dependable performance.

Responsibilities

• Research and compare evolving LLM and VLM models that may support SureHost's product use cases.
• Create representative evaluation datasets, test cases, scoring criteria, and repeatable benchmark workflows.
• Measure model quality across accuracy, reliability, latency, cost, and other product-relevant factors.
• Analyze failure patterns and trade-offs, including false positives and false negatives on issues operators care about.
• Present clear findings and recommendations to inform model strategy and the product roadmap.
• Maintain benchmark results and documentation as models, APIs, and requirements change.

Skills

Applied AI

Python basics

Data analysis

Documentation

Tools

Python

Job description

• Rate: $22 – $28/hr
• Hours: 20 hours / week
• Experience: 1 - 3 years
• Location: United States | Canada + 23 more

AI Model Benchmarking Intern (Remote, Iowa-Based)

Part-time, 15–20 hours/week · Remote, Iowa-based · 6-month internship · $22–$28/hour

About SureHost

SureHost is an early-stage hospitality technology company based in Des Moines, Iowa. We help growing short-term rental operators confirm each property is guest-ready after a cleaning turn, using documented property standards, photo and video evidence, AI-assisted review, and human review where it matters.

We are working with an initial paying customer and preparing for broader growth in the U.S. market. Our customer and product work happens in Des Moines, and our technical team is based in Pune, India. Our stack is Python and Flutter, deployed on GCP.

The role

SureHost uses AI narrowly, where it can be measured and trusted. Language and vision models change quickly, and a model that looks strong in a demo can still fail on real property footage, cost too much to run, or make decisions an operator cannot follow.

In this role, you will help us tell the difference. You will build repeatable benchmarks that compare LLMs and VLMs on SureHost use cases, and your findings will inform model choices and the product roadmap. You will work with our AI Chief Architect.

What you will do

• Research and compare evolving LLM and VLM models that may support SureHost's product use cases.
• Create representative evaluation datasets, test cases, scoring criteria, and repeatable benchmark workflows.
• Measure model quality across accuracy, reliability, latency, cost, and other product-relevant factors.
• Analyze failure patterns and trade-offs, including false positives and false negatives on issues operators care about.
• Present clear findings and recommendations to inform model strategy and the product roadmap.
• Maintain benchmark results and documentation as models, APIs, and requirements change.

What you bring

• Interest in applied AI and familiarity with LLMs, VLMs, generative AI tools, or model APIs through coursework or projects.
• Logical, evidence-based thinking and comfort working with structured data and evaluation criteria.
• Basic Python or data-analysis skills.
• The ability to document methods and results clearly.
• Patience to run repeatable experiments, investigate edge cases, and separate a compelling demo from dependable performance.
• Clear communication, willingness to ask questions early, and responsible handling of customer, property, and company information.

Required

• Based in Iowa.

Details

• Pay: $22–$28/hour, based on experience and how independently you can work.
• Hours: 15–20 hours per week, with flexible scheduling.
• Location: Remote, for Iowa-based candidates.
• Term: 6-month internship.
• Start date: Flexible. We are onboarding now and can work around school schedules, finals, and the holidays.
• Works with: AI Chief Architect.
• Team time: A monthly in-person check-in over coffee in Des Moines.

What you get

• Real responsibility on a live hospitality technology product, with few layers between customer needs and product decisions.
• Direct mentorship from the AI Chief Architect on applied model evaluation.
• Work you can point to: benchmark methods, results, and recommendations to reference in future roles.

Company: SureHost