Senior Software Engineer, Agentic Coding Evaluation (Python, C++)

Vettedlance; United States; Remote; USD 110,000 - 165,000; Part time

Benefits offered by this job

Remote work

Hourly freelance contract

Part-time (10-40 hrs/week)

Job summary

Vettedlance is seeking senior software engineers to review and grade long-horizon coding work by AI agents in real repositories.

You will review agent trajectories, write reference solutions and tests in Python or C++, and judge code quality, security and maintainability against a rubric.

Explain the root cause when an agent's fix fails, and work remotely on an hourly contract with 10-40 hours per week, paid weekly in USD after a short paid assessment.

Qualifications

• 5+ years of professional software engineering experience.
• Expert in Python or C++.
• Fluent written English.
• Comfortable reading unfamiliar codebases and writing tests.

Responsibilities

• Review agent trajectories on multi-file, long-horizon coding tasks and score each step.
• Write reference solutions and tests in Python or C++.
• Judge code quality, correctness, security and maintainability against a rubric.
• Explain the root cause when an agent's fix fails.

Skills

Python

C++

Code review

Testing

English proficiency

Job description

Senior engineers review and grade long-horizon, multi-step coding work by AI agents in real repositories.

What you will do

• Review agent trajectories on multi-file, long-horizon coding tasks and score each step.
• Write reference solutions and tests in Python or C++.
• Judge code quality, correctness, security and maintainability against a rubric.
• Explain the root cause when an agent's fix fails.

What you need

• 5+ years of professional software engineering.
• Expert in Python or C++.
• Comfortable reading unfamiliar codebases and writing tests.
• Fluent written English.

Contract

• Remote, hourly contract (independent contractor)
• Part-time, flexible: 10-40 hrs/week
• Remote

Pay: $80 - $120 / hour (USD). Paid in USD, weekly. Rate set after a short paid assessment.