AI Benchmark Architect: Expert Challenge Designer

Lifted (an Upwork Company); United States; On-site; USD 80,000 - 160,000; Full time

Job summary

Lifted (an Upwork Company) is looking for senior professionals to help build a benchmark of expert-level AI problems that today’s best agents still can’t solve. You will craft tasks in a Linux terminal that reveal the limits of current models for a leading enterprise client.

The project runs ~7 weeks with a brief onboarding, and requires rigorous technical expertise and proven publication experience.

Qualifications

• Masters degree or higher in a technical field.
• 10+ years of professional experience in a technical domain.
• At least one publication (academic or professional).

Responsibilities

• Design and curate benchmark problems that challenge AI agents.
• Collaborate with domain experts to ensure problem relevance.
• Document methodology and results for publication and review.

Skills

Experience in technical domains

Publication

Education

Master's degree or higher

Job description

Lifted (an Upwork Company) is looking for senior professionals to help build a benchmark of expert-level AI problems that today’s best agents still can’t solve. You will craft tasks in a Linux terminal that reveal the limits of current models for a leading enterprise client.

The project runs ~7 weeks with a brief onboarding, and requires rigorous technical expertise and proven publication experience.