Senior Coding, ML & Security Experts - AI Benchmarking

Outlier; Norway; Remote; NOK 2,474,000 - 2,995,000; Full time

Benefits offered by this job

Remote work

International collaborators

Job summary

Outlier is seeking experienced professionals in coding, machine learning or cybersecurity to craft challenging tasks for AI benchmarks. You will identify hard problems from your field and translate them into Linux-terminal tasks tested against leading models.

The role requires 8+ years in relevant fields, a Master’s or higher, and at least one publication. Remote work from multiple eligible countries is supported, with onboarding across courses and assessments.

Qualifications

• At least 8 years of relevant professional experience in coding, machine learning or cybersecurity.
• Master’s degree or higher; or bachelor’s degree with at least 10 years of experience and a publication.
• Degrees completed and not currently enrolled.
• At least one academic or professional publication.
• Comfort working in a Linux terminal.
• Residence in one of the listed countries.

Responsibilities

• Identify difficult, meaningful problems in your area of expertise.
• Translate those problems into tasks solvable in a Linux terminal.
• Design challenges that reveal limitations in advanced AI agents; each task tested against models.

Skills

Coding

Machine Learning

Cybersecurity

Education

Master's degree or higher

Bachelor's degree + 10+ years exp with publication

Tools

Linux terminal

Job description

Bring your expertise to the problems today’s most advanced AI agents still struggle to solve.

We’re looking for experienced coding, machine learning or cybersecurity professionals to develop challenging tasks for an AI benchmark. You’ll draw on problems from your own field and turn them into clearly defined challenges that an AI agent must solve from scratch in a Linux terminal.

What you’ll do

• Identify difficult, meaningful problems in your area of expertise.
• Translate those problems into tasks that can be solved in a Linux terminal.
• Design challenges that expose limitations in advanced AI agents. Each task is tested against three leading models, with three attempts per model, and must prove difficult enough that the models mostly fail.

Who we’re looking for

• At least 8 years of relevant professional experience in coding, machine learning or cybersecurity.
• A completed master’s degree or higher. Candidates with a completed bachelor’s degree, at least 10 years of relevant professional experience and a publication may also be considered.
• Applicants must have completed their degrees and must not currently be enrolled in a degree program.
• At least one academic or professional publication.
• Practical comfort working in a Linux terminal.
• Current residence in Argentina, Mexico, Puerto Rico, New Zealand, Canada, the United Kingdom, the United States, Australia, Peru, Colombia, Austria, Ireland, Spain, Luxembourg, Italy, France, Germany, Belgium, the Netherlands, Sweden, Finland, Singapore, Denmark, Norway and Switzerland.

Project details

• Task-based project expected to run for the next weeks.
• Onboarding includes two courses and a graded assessment, taking approximately 90 minutes.
• A live onboarding webinar is also required before moving to tasking.

Compensation

Earn up to $1,350 per task, with opportunities for higher rates based on task quality and submission volume. Tasks take six to seven hours on average, although initial tasks may take longer as you get familiar with the process.