AI Benchmark Engineer: Docker Labs & Terminal Tasks

Clorvin Labs; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Clorvin Labs in the United States is seeking a developer to design terminal-based tasks that mirror real-world software engineering challenges. You will build and maintain Docker environments, creating reference solutions and validation tests to assess AI models.

You will run tasks against state-of-the-art models, analyze results, and refine tasks to improve evaluation accuracy. Candidates should have a CS/engineering degree, strong Linux and Python skills, and a track record solving genuine

Qualifications

• Degree in Computer Science, Engineering, or related discipline (B.Tech, M.Tech, or PhD).
• Proficiency in Linux and hands-on experience with Docker.
• Fluency in Python or another programming language.
• Track record of tackling and solving genuine technical problems.
• Familiarity with Terminal-Bench or comparable AI evaluation frameworks is valued, though not required.

Responsibilities

• Design complex terminal-based tasks that mirror challenges software engineers face in practice.
• Build and maintain Docker environments that reliably reproduce the problems these tasks present.
• Create reference solutions and validation tests that catch when an AI model's answer falls short.
• Run tasks against state-of-the-art AI models, analyze their performance, and refine based on the results.

Skills

Linux

Docker

Python

Problem solving

Test automation

Education

B.Tech / M.Tech / PhD in CS/Engineering

Tools

Terminal-Bench

Job description

Clorvin Labs in the United States is seeking a developer to design terminal-based tasks that mirror real-world software engineering challenges. You will build and maintain Docker environments, creating reference solutions and validation tests to assess AI models.

You will run tasks against state-of-the-art models, analyze results, and refine tasks to improve evaluation accuracy. Candidates should have a CS/engineering degree, strong Linux and Python skills, and a track record solving genuine