AI Agent Evaluation Engineer — Linux & Edge-Case Debugging

Workada

United States

Remote

USD 90,000 - 140,000

Full time

37 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Workada is seeking a contributor who can convert real-world problems into self-contained Linux-terminal tasks for AI agents. You will craft briefs, set up environments with necessary files and tooling, build reference solutions, and implement automated checks to ensure correctness.

You’ll leverage deep knowledge of algorithms, OS, and data processing to identify failure modes and edge cases, communicating clearly where a workflow breaks and how to fix it.

Qualifications

  • PhD in CS or software engineering or 5+ years of professional experience in related field.
  • Strong Linux terminal and scripting skills (Python or Bash).
  • Experience with Docker and distributed/OS concepts is a plus.

Responsibilities

  • Translate problems into self-contained tasks for AI agents in a Linux terminal.
  • Create briefs, environment setups, reference solutions, and automated checks for each Task.
  • Identify and explain failure modes and edge cases in workflows.

Skills

Linux terminal
Python/Bash scripting
Concurrency and debugging
Research and problem solving

Education

PhD in computer science or software engineering

Tools

Docker
CI/CD pipelines

Job description

Workada is seeking a contributor who can convert real-world problems into self-contained Linux-terminal tasks for AI agents. You will craft briefs, set up environments with necessary files and tooling, build reference solutions, and implement automated checks to ensure correctness.

You’ll leverage deep knowledge of algorithms, OS, and data processing to identify failure modes and edge cases, communicating clearly where a workflow breaks and how to fix it.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Evaluation Architect for Linux Task Challenges
AI Evaluation Architect for Linux Task Challenges

Workada • United States

Remote
USD 120,000 - 180,000
Software Engineer – AI Agent Evaluation
Software Engineer – AI Agent Evaluation

Workada • United States

Remote
USD 90,000 - 140,000
Security Engineer - AI Agent Task Architect
Security Engineer - AI Agent Task Architect

Workada • United States

Remote
USD 120,000 - 180,000
Security Engineer – AI Agent Evaluation
Security Engineer – AI Agent Evaluation

Workada • United States

Remote
USD 120,000 - 180,000
AI Agent Evaluation Expert
AI Agent Evaluation Expert

Workada • United States

Remote
USD 120,000 - 180,000
AI Benchmark Challenge Architect (Coding/ML/Cyber)
AI Benchmark Challenge Architect (Coding/ML/Cyber)

Joinkeywork • United States

Remote
USD 262,000 - 317,000
Remote Agent Engineer: Code Review & Debugging Expert
Remote Agent Engineer: Code Review & Debugging Expert

Auraone • United States

Remote
USD 138,000 - 689,000
Remote Software Engineer: AI Task Expert & Optimizer
Remote Software Engineer: AI Task Expert & Optimizer

Edatabae • United States

Remote
USD 120,000 - 200,000
Finance & Operations Research Expert – AI Agent Evaluation
Finance & Operations Research Expert – AI Agent Evaluation

Workada • United States

Remote
USD 120,000 - 180,000
Remote AI Code Auditor & Red-Teaming Engineer
Remote AI Code Auditor & Red-Teaming Engineer

DataAnnotation • Washington

Remote
USD 100,000 - 200,000
Flexible work arrangement
Remote work