AI Benchmark Engineer (Remote Contract)

LinkedIn; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

LinkedIn is seeking part-time, project-based experts to design and evaluate Terminal-Bench 4.0 tasks for AI systems. You will create end-to-end tasks, build Docker environments, and author precise instructions plus robust reference solutions.

You will also develop hidden tests to ensure genuine correctness and resist shortcut solutions, all while maintaining reproducible setups. The role emphasizes hands-on software engineering, Python proficiency, and Linux/Docker fluency, with additional value

Qualifications

• Strong hands-on software engineering ability, especially Python.
• Experience in ML engineering, research engineering, AI systems, or related fields.
• Fluency with Linux/terminal environments and Docker for reproducible setups.
• Experience with GPU programming and performance optimization is a plus.

Responsibilities

• Design challenging, technically substantive terminal-based engineering tasks.
• Build reproducible Docker environments for each task.
• Write clear, self-contained task specifications and instructions.
• Implement correct, working reference solutions.
• Develop hidden test suites that evaluate genuine functional correctness.
• Design evaluations that resist hard-coded answers, shortcuts, and incomplete solutions.
• Verify that the reference solution passes the evaluation and that the untouched task fails.

Skills

Python

Linux/terminal

Software testing

Debugging

Independent problem solving

Tools

Docker

CUDA

Job description

LinkedIn is seeking part-time, project-based experts to design and evaluate Terminal-Bench 4.0 tasks for AI systems. You will create end-to-end tasks, build Docker environments, and author precise instructions plus robust reference solutions.

You will also develop hidden tests to ensure genuine correctness and resist shortcut solutions, all while maintaining reproducible setups. The role emphasizes hands-on software engineering, Python proficiency, and Linux/Docker fluency, with additional value