Senior AI Data & Evaluation Engineer - Remote, Part-Time

Crossing Hurdles; United States; Remote; USD 170,000 - 243,000; Part time

Job summary

Crossing Hurdles is seeking a Senior Software Engineer specializing in Python and TypeScript for a contractor assignment. The role is remote and part-time (20-40 hours/week) with overlap during PST business hours.

You will build pipelines and contribute to AI data evaluation workflows while refining rubrics and evaluation criteria. You will review and label data, collaborate with researchers and engineers, and translate findings into clear, actionable recommendations.

Qualifications

• Hands-on experience with preference data collection and/or rubric creation.
• Experience navigating or contributing to large codebases; experience with codebases such as VSCode is a plus.
• Experience with AI/LLM evaluation or related engineering workflows is preferred.
• A LinkedIn profile is mandatory for review.
• Candidates should not currently be working with Microsoft.

Responsibilities

• Build and maintain pipelines and infrastructure supporting AI data generation, collection, and evaluation workflows.
• Design, refine, and iterate on rubrics and evaluation criteria for preference data.
• Review and label preference data while identifying subtle quality issues.
• Collaborate with researchers and engineers to translate qualitative judgment into scalable evaluation processes.
• Synthesize technical findings into clear, actionable write-ups, updates, and recommendations.

Skills

Preference data collection

Rubric creation

Large codebases navigation

VSCode

AI/LLM evaluation workflows

Tools

VSCode

Job description

Crossing Hurdles is seeking a Senior Software Engineer specializing in Python and TypeScript for a contractor assignment. The role is remote and part-time (20-40 hours/week) with overlap during PST business hours.

You will build pipelines and contribute to AI data evaluation workflows while refining rubrics and evaluation criteria. You will review and label data, collaborate with researchers and engineers, and translate findings into clear, actionable recommendations.