Senior AI Benchmark Engineer (Remote)

HUMAN-AI-Z Nexus Technology; Dubai; On-site; AED 441,000 - 661,000; Full time

Job summary

HUMAN-AI-Z Nexus Technology is seeking a Senior AI Engineer to design and implement multi-agent benchmarks for real-world open-source code changes. This remote, full-time role focuses on evaluating AI agents across software projects and requires deep Python/JavaScript proficiency, Docker expertise, and familiarity with modern web frameworks.

You will work within the Harbor evaluation framework, write precise task instructions, and build verification scripts to ensure correct agent outputs in

Qualifications

• 5+ years of experience in Python and JavaScript development.
• Experience with AI coding benchmarks such as SWE-bench or Terminal-bench.
• Strong ability to read and navigate large open-source codebases, including frameworks like Django, Flask, FastAPI, or Node.js.
• Familiarity with Git workflows, including pull requests, diffs, cherry-picking, and working with specific commits.
• Experience with Docker, including writing Dockerfiles and managing builds.

Responsibilities

• Design and build multi-agent benchmark tasks based on real-world open-source code changes, including bug fixes, migrations, and refactors.
• Work within the Harbor evaluation framework to run and validate tasks inside Docker environments.
• Write clear, precise task instructions specifying file paths, function signatures, expected behavior, and constraints.
• Develop Python-based verification scripts to validate correctness of agent-generated code changes.
• Decompose complex engineering problems across multiple specialized agents and refine tasks within containerized environments.

Skills

Python

JavaScript

Open-source codebase navigation

Git workflows

Tools

Docker

Django

Flask

FastAPI

Node.js

Job description

HUMAN-AI-Z Nexus Technology is seeking a Senior AI Engineer to design and implement multi-agent benchmarks for real-world open-source code changes. This remote, full-time role focuses on evaluating AI agents across software projects and requires deep Python/JavaScript proficiency, Docker expertise, and familiarity with modern web frameworks.

You will work within the Harbor evaluation framework, write precise task instructions, and build verification scripts to ensure correct agent outputs in