Linux AI Benchmark Architect

Joinkeywork; United States; Remote; Full time

Job summary

Joinkeywork seeks experienced professionals to craft challenging AI benchmark tasks. You’ll convert expert-domain problems into clearly defined Linux-terminal tasks, then design tests to probe AI agents’ limits across multiple models and attempts.

The role requires a master’s or higher, 10+ years in a technical field, and publications. You must be comfortable in a Linux terminal and based in the United States.

Qualifications

• Completed master’s degree or higher; candidates with bachelor’s degree and 10+ years experience may be considered.
• At least 10 years of professional experience in a technical domain.
• Expertise in coding, machine learning, systems, cybersecurity or hardware.
• At least one academic or professional publication.
• Practical comfort working in a Linux terminal.
• Currently based in the United States.

Responsibilities

• Identify difficult, meaningful problems in your area of expertise.
• Translate those problems into tasks that can be solved in a Linux terminal.
• Design challenges that expose limitations in advanced AI agents; each task tested against three models with three attempts per model.

Skills

Coding

Machine learning

Systems

Cybersecurity

Hardware

Education

Master’s degree or higher

Bachelor’s degree + 10 years experience

Publication

Tools

Linux terminal

Job description

Joinkeywork seeks experienced professionals to craft challenging AI benchmark tasks. You’ll convert expert-domain problems into clearly defined Linux-terminal tasks, then design tests to probe AI agents’ limits across multiple models and attempts.

The role requires a master’s or higher, 10+ years in a technical field, and publications. You must be comfortable in a Linux terminal and based in the United States.