AI Evaluation Engineer — Remote Task Design & Model Eval

Patronusaiinc; United States; Remote; USD 41.000 - 138.000; Teilzeit

Benefits dieser Stelle

Remote work

Flexible hours

Zusammenfassung

Patronus AI in the United States seeks experienced software engineers for task-based contracts to design and judge complex AI evaluation tasks. You will create multi-step coding challenges, review model outputs, and provide structured feedback to advance real-world engineering practice.

Remote, hourly contract with flexible 15-20 hours per week, suitable alongside a full-time role. Must have strong coding skills in Python/Java/C++ and a background in software engineering, data science, or

Qualifikationen

• 2+ years of professional experience in software engineering, data science or analytics.
• Experience with system design, data pipelines, ML or LLM-based applications.
• Prior work in code review, technical writing, AI training or model evaluation.
• Bachelor's degree in computer science, engineering or a related field, or equivalent experience.

Aufgaben

• Design realistic, multi-step coding tasks that test an AI model's ability to reason and code.
• Build tasks around codebases, tech specs, architecture docs, and datasets.
• Evaluate AI-generated code and content for correctness, efficiency, and clarity.
• Grade model outputs against rubrics and provide structured feedback.

Kenntnisse

Software engineering

Data science

Analytics

System design

Data pipelines

ML/LLM applications

Code review

Technical writing

Ausbildung

Bachelor's degree in computer science, engineering or related field

Tools

Python

Java

C++

JavaScript

TypeScript

Go

Jobbeschreibung

Patronus AI in the United States seeks experienced software engineers for task-based contracts to design and judge complex AI evaluation tasks. You will create multi-step coding challenges, review model outputs, and provide structured feedback to advance real-world engineering practice.

Remote, hourly contract with flexible 15-20 hours per week, suitable alongside a full-time role. Must have strong coding skills in Python/Java/C++ and a background in software engineering, data science, or