AI-Powered Dev Workflow Auditor

Remote DXB; United States; Remote; USD 55,000 - 96,000; Full time

Job summary

Remote DXB is seeking experienced evaluators to audit tasks used to train and evaluate AI systems. You will assess developer workflow tasks for accuracy, realism, and reproducibility, and provide actionable feedback on AI-assisted coding processes.

The role focuses on testing complex technical scenarios, evaluating AI training workflows, and collaborating with a distributed team to ensure rigor and practicality of tooling.

Qualifications

• Professional experience and deep technical knowledge in modern developer tooling.
• Expertise in AI coding assistants, telemetry, and trace analysis.
• Strong analytical and problem-solving skills for evaluating complex technical scenarios.

Responsibilities

• Assess developer workflow tasks for technical accuracy, realism, and reproducibility
• Provide actionable feedback on IDE integration faults and AI-assisted coding inefficiencies
• Rigorously test and troubleshoot complex technical scenarios to evaluate AI training workflows

Skills

Developer tooling expertise

AI coding assistants

Telemetry & trace analysis

Tools

Telemetry analysis tools

AI debugging tooling

Job description

Remote DXB is seeking experienced evaluators to audit tasks used to train and evaluate AI systems. You will assess developer workflow tasks for accuracy, realism, and reproducibility, and provide actionable feedback on AI-assisted coding processes.

The role focuses on testing complex technical scenarios, evaluating AI training workflows, and collaborating with a distributed team to ensure rigor and practicality of tooling.