Senior Reviewer, Code

Reinforced Learning; Town of Tonawanda (NY); On-site; USD 130,000 - 170,000; Full time

Job summary

Reinforced Learning is seeking a senior reviewer to serve as the final technical reviewer on coding datasets. You will read trajectories, check diffs, run tests, and decide whether work meets specifications and whether a model should learn from it.

Your experience in software engineering and code review will be used to maintain high quality in large-scale data. You will train and calibrate other reviewers, document recurring defects, and perform spot checks of shipped data to ensure reliability

Qualifications

• Six or more years of software engineering experience.
• Experience with code review and ensuring quality.
• Ability to explain high standards concisely.

Responsibilities

• Review code demonstrations, agent trajectories and preference judgments.
• Verify, run and extend tests where present and write where absent.
• Calibrate reviewers and resolve disagreements.
• Document recurring defects and feed to dataset leads.
• Spot-check shipped data and report escape rates.

Skills

Software engineering

Code review

Testing

Debugging

Reproducing failures

Job description

The role

You'll be the final technical reviewer on coding datasets: reading trajectories, checking diffs, running tests, and deciding whether the work meets the specification and whether a model should learn from it.

What you’ll own

• Review of code demonstrations, agent trajectories and preference judgements for correctness and quality.
• Verification, run and extended where it exists and written where it doesn’t.
• Calibration of other reviewers and resolution of disagreements.
• Recurring defects, documented and fed to dataset leads and spec writers.
• Spot checks of shipped data and reported escape rates.

What you bring

• Six or more years of software engineering across multiple languages, with substantial code review experience.
• High standards and the ability to explain them concisely.
• Stamina for careful reading at volume.
• Testing, debugging and reproducing failures.

Helpful, not required

• Open source maintainer experience.
• Prior training data or evaluation work.