LLM Evaluation Engineer - Open-Source Code Vetting(Remote)

HUMAN-AI-Z Nexus Technology; Dubai; On-site; AED 279,000 - 502,000; Part time

Job summary

HUMAN-AI-Z Nexus Technology is seeking a Senior Software Engineer - LLM Evaluation & Repository Validation to work on a part-time basis. You will help build evaluation datasets, automate environments, and assess code quality across real-world projects.

You will analyze open-source code, configure repositories with Docker, review unit tests, and run experiments to measure LLM performance. This remote role offers flexible hours and competitive compensation.

Qualifications

• Proficient in at least one modern language (Python/JavaScript/Java/Go/Rust/C/C++/C#/Ruby).
• Familiar with Git, Docker, and basic software pipeline setup.
• Ability to navigate and modify complex codebases and run tests locally.

Responsibilities

• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment setup.
• Evaluate unit test coverage and quality in real-world codebases.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to identify challenging repositories for LLMs.

Skills

Python

JavaScript

Java

Go

Rust

C/C++

C#

Ruby

Tools

Git

Docker

CI/CD basics

Job description

HUMAN-AI-Z Nexus Technology is seeking a Senior Software Engineer - LLM Evaluation & Repository Validation to work on a part-time basis. You will help build evaluation datasets, automate environments, and assess code quality across real-world projects.

You will analyze open-source code, configure repositories with Docker, review unit tests, and run experiments to measure LLM performance. This remote role offers flexible hours and competitive compensation.