C++ Systems Engineer – AI Model Evaluation & Code Review | Remote

Crossing Hurdles; United States; On-site; USD 82,656 - 137,760; Part time

Job summary

A technology evaluation firm is seeking software engineering experts with a strong background in C++ and data science. This role involves evaluating LLM-generated content, conducting accuracy assessments, and ensuring alignment with technical standards. Candidates should hold at least a bachelor's degree in computer science and have real-world experience in software engineering. The position offers flexibility between full-time or part-time commitments and can be performed remotely.

Qualifications

• Real-world experience in software engineering or related technical roles.
• Ability to solve HackerRank or LeetCode Medium and Hard–level problems independently.
• Experience contributing to open-source projects.
• Prior experience with RLHF, model evaluation, or data annotation work preferred.

Responsibilities

• Evaluate LLM-generated responses for accuracy and completeness.
• Conduct fact-checking using trusted public sources.
• Conduct accuracy testing by executing code.
• Annotate model responses for strengths and improvements.
• Assess code quality and alignment with guidelines.

Skills

C++ programming language

Problem-solving (HackerRank, LeetCode)

Attention to detail

Complex technical reasoning evaluation

Experience with LLMs

Education

BS, MS, or PhD in Computer Science

Job description

Position: Software Engineering, Data Science, and Systems Design Experts, C++ (5+ YOE)

Type: Hourly contract

Compensation: $60-$100 per hour

Location: Remote

Commitment: Full-time or Part-time Contract Work

Role Responsibilities

• Evaluate LLM-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness
• Conduct fact-checking using trusted public sources and authoritative references
• Conduct accuracy testing by executing code and validating outputs using appropriate tools
• Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
• Assess code quality, readability, algorithmic soundness, and explanation quality
• Ensure model responses align with expected conversational behavior and system guidelines
• Apply consistent evaluation standards by following taxonomies, benchmarks, and detailed evaluation guidelines

Requirements

• BS, MS, or PhD in Computer Science or a closely related field
• Real-world experience in software engineering or related technical roles
• Expertise in C++ programming language
• Ability to solve HackerRank or LeetCode Medium and Hard–level problems independently
• Experience contributing to open-source projects, including merged pull requests
• Experience using LLMs while coding and understanding their strengths and failure modes
• Strong attention to detail and ability to evaluate complex technical reasoning and identify bugs or logical flaws
• Prior experience with RLHF, model evaluation, or data annotation work preferred
• Track record in competitive programming preferred
• Experience reviewing code in production environments preferred
• Familiarity with multiple programming paradigms or ecosystems preferred
• Experience explaining complex technical concepts to non-expert audiences preferred