Technical Experts needed to stump AI Models

Lifted, an Upwork Company™; Florida City (FL); On-site; USD 90,000 - 130,000; Full time

Job summary

SmartRecruiters, Inc. seeks a seasoned technical expert to design and present hard problems for AI benchmarking.

You will craft tasks in a Linux terminal drawn from your domain, ensuring they resist current frontier models when tested by multiple AI systems. Ideal candidates hold a Masters or higher, 10+ years of experience in a technical field, at least one publication, and demonstrated coding, systems, and security expertise.

Qualifications

• Masters degree or higher required.
• 10+ years of professional experience in a technical domain.
• At least one publication (academic or professional).
• Strong coding, systems, and security capabilities.

Responsibilities

• Design hard problems from your field to benchmark AI models.
• Create tasks in a Linux terminal that challenge current frontier systems.
• Contribute to a 7-week contract engagement and support onboarding.

Skills

Coding

Systems

Security

Publications

Education

Masters degree or higher

Job description

Technical Experts needed to stump AI Models

• Contract

This enterprise client helps the world’s most innovative companies improve their AI models by providing human feedback.

Help build a benchmark of real, expert-level problems that today’s best AI agents still can’t solve. Experts take a genuinely hard problem from their own field and turn it into a task an AI must solve from scratch in a Linux terminal.

The goal: stump the model. A task counts only if, when three frontier AI models each try it three times, they mostly fail. It’s not annotation — contributors design the problems that expose where AI still breaks.

Who we need:

• Masters degree or higher
• 10+ years of professional experience in a technical domain
• At least one publication (academic or professional)
• Coding
• Systems
• Security

• This is an ongoing project — at least 7 weeks
• Onboarding process is approximately90 minutes total