Remote Senior Software Engineer - AI Evaluation & Coding

SaidGig; New York (NY); Remote; USD 138,000 - 207,000; Part time

Job summary

SaidGig seeks a remote independent-contractor to improve AI coding agents by reviewing work across real-world software repositories. You will assess task understanding, approach selection, and code correctness, providing precise feedback to strengthen model performance.

You will design rubrics, label data to high quality, build pipelines, and document findings for the team, collaborating with researchers and engineers to scale qualitative judgment into robust processes.

Qualifications

• 5+ years of hands-on software engineering experience.
• Proficiency in Python, TypeScript/JavaScript, Go, or another major production language.
• Experience working in substantial, real-world codebases.
• Strong code-review skills and technical judgment.
• Ability to clearly explain why an implementation is correct, incorrect, or could be improved.
• Strong written communication skills.
• Experience using modern LLMs or AI coding tools.
• Experience with LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training is helpful but not required.

Responsibilities

• Design, refine, and iterate on rubrics and evaluation criteria for preference data.
• Review and label data to a high quality standard, identifying subtle issues that may be missed by others.
• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation workflows.
• Turn findings from data work into clear written updates, recommendations, and documentation for the team.
• Work closely with researchers and engineers to convert qualitative technical judgment into scalable processes.

Skills

Python

TypeScript

JavaScript

Go

Code review

Written communication

LLMs / AI coding tools

RLHF / data design

Job description

SaidGig seeks a remote independent-contractor to improve AI coding agents by reviewing work across real-world software repositories. You will assess task understanding, approach selection, and code correctness, providing precise feedback to strengthen model performance.

You will design rubrics, label data to high quality, build pipelines, and document findings for the team, collaborating with researchers and engineers to scale qualitative judgment into robust processes.