Technical Experts needed to stump AI Models

Lifted (an Upwork Company); United States; On-site; USD 80,000 - 160,000; Full time

Job summary

Lifted (an Upwork Company) is looking for senior professionals to help build a benchmark of expert-level AI problems that today’s best agents still can’t solve. You will craft tasks in a Linux terminal that reveal the limits of current models for a leading enterprise client.

The project runs ~7 weeks with a brief onboarding, and requires rigorous technical expertise and proven publication experience.

Qualifications

• Masters degree or higher in a technical field.
• 10+ years of professional experience in a technical domain.
• At least one publication (academic or professional).

Responsibilities

• Design and curate benchmark problems that challenge AI agents.
• Collaborate with domain experts to ensure problem relevance.
• Document methodology and results for publication and review.

Skills

Experience in technical domains

Publication

Education

Master's degree or higher

Job description

Help build a benchmark of real, expert-level problems that today’s best AI agents still can’t solve.

Experts take a genuinely hard problem from their own field and turn it into a task an AI must solve from scratch in a Linux terminal.

The goal: stump the model. A task counts only if, when three frontier AI models each try it three times, they mostly fail. It’s not annotation - contributors design the problems that expose where AI still breaks.

Who we need:

• Masters degree or higher
• 10+ years of professional experience in a technical domain
• At least one publication (academic or professional)

Domains in demand:

• Coding
• Machine Learning
• Systems
• Security
• Hardware

This is an ongoing project - at least 7 weeks.

Onboarding process is approximately 90 minutes total.

This enterprise client helps the world’s most innovative companies improve their AI models by providing human feedback.