Remote Contract Software Engineer for AI Model Evaluation

Patronus AI, Inc.

United States

Remote

USD 69,000 - 138,000

Part time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Patronus AI is seeking experienced software engineers for task-based contracts to design realistic, multi-step coding and technical tasks for frontier AI models and to judge how well those models handle them. You’ll apply day-to-day engineering judgment to assess code, reasoning and technical instructions, grade outputs against rubrics, and provide clear, structured feedback.

This is a remote, hourly project-based role based in India, with typical commitment of 15–20 hours per week and the

Qualifications

  • 2+ years of professional experience in software engineering. (internships excluded)
  • Experience with system design, data pipelines, ML or LLM-based applications.
  • Prior work in code review, technical writing, AI training or model evaluation.
  • Strong coding skills in at least one major language, such as Python, Java, C++, JavaScript, TypeScript or Go.
  • Experience building, reviewing and debugging production-grade code.
  • Ability to follow detailed technical guidelines, work independently, respond to feedback, and meet deadlines.
  • Excellent written communication and close attention to detail.
  • A bachelor’s degree in computer science, engineering or a related field, or equivalent experience.

Responsibilities

  • Design realistic, multi-step coding and technical tasks that test an AI model’s ability to reason, write code and follow precise instructions.
  • Build tasks around realistic workspace materials such as codebases, technical specs, architecture docs and datasets.
  • Evaluate AI-generated code and technical content for correctness, efficiency, clarity and alignment with real-world engineering practice.
  • Grade model outputs against rubrics and ground-truth answers, and write clear, structured feedback.
  • Help improve how AI understands programming, system design, data workflows and engineering best practices.
  • Work independently and asynchronously, meet deadlines, and collaborate with the Patronus team to keep quality high.
  • Test and refine content to reflect real-world complexity.
  • Review other contributors’ work for accuracy, clarity, and practical relevance.
  • Incorporate feedback promptly and flag gaps.

Skills

System design
Data pipelines
ML/LLM applications
Code review
Technical writing
AI training/evaluation
Python/Java/C++/JS/TS/Go
Production-grade code

Education

Bachelor's degree in CS/Engineering or related field

Job description

Patronus AI is seeking experienced software engineers for task-based contracts to design realistic, multi-step coding and technical tasks for frontier AI models and to judge how well those models handle them. You’ll apply day-to-day engineering judgment to assess code, reasoning and technical instructions, grade outputs against rubrics, and provide clear, structured feedback.

This is a remote, hourly project-based role based in India, with typical commitment of 15–20 hours per week and the

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Contract Software Engineer for AI Evaluation
Remote Contract Software Engineer for AI Evaluation

Patronusaiinc • United States

Remote
USD 69,000 - 138,000
Software Engineer – AI Evaluation (Task-based Contract, Remote)
Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronus AI • United States

Remote
USD 69,000 - 138,000
AI Evaluation Engineer: Real-World Coding & Systems Tasks
AI Evaluation Engineer: Real-World Coding & Systems Tasks

Patronus AI • United States

Remote
USD 69,000 - 138,000
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronus AI, Inc. • United States

Remote
USD 69,000 - 138,000
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronusaiinc • United States

Remote
USD 69,000 - 138,000
Remote AI Evaluation Engineer — Code Tasks & Review
Remote AI Evaluation Engineer — Code Tasks & Review

Patronus AI • United States

Remote
USD 69,000 - 138,000
Flexible schedule
Remote contract
15–20 hours per week
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronus AI • United States

Remote
USD 69,000 - 138,000
Flexible schedule
Remote contract
15–20 hours per week
Senior Mechanical Design Engineer - Remote Contract
Senior Mechanical Design Engineer - Remote Contract

Patronus AI, Inc. • United States

Remote
USD 69,000 - 207,000
Remote AI Evaluation Task Architect
Remote AI Evaluation Task Architect

Patronus AI, Inc. • United States

Remote
USD 69,000 - 207,000
Senior Legal Researcher for AI Evaluation (Remote)
Senior Legal Researcher for AI Evaluation (Remote)

Patronus AI • United States

Remote
USD 69,000 - 207,000
Fully remote (US‑only)
Paid trial task
Flexible, asynchronous schedule