Remote Senior AI Evaluation & Infrastructure Engineer

Crossing Hurdles; United States; Remote; USD 138,000 - 207,000; Part time

Job summary

Crossing Hurdles is seeking a Senior Software Engineer proficient in Python and TypeScript for a remote, part-time contractor assignment. You will build and maintain AI data pipelines, contribute to evaluation rubrics, and collaborate with researchers to scale qualitative judgments into reproducible processes.

The role emphasizes reviewing and labeling data quality, working within large codebases, and producing clear technical write-ups for stakeholders.

Qualifications

• Hands-on experience with preference data collection and rubric creation.
• Experience navigating large codebases; VSCode experience is a plus.
• Experience with AI/LLM evaluation or related engineering workflows preferred.
• LinkedIn profile is mandatory for review.
• Candidates should not currently be working with Microsoft.

Responsibilities

• Build and maintain pipelines and infrastructure supporting AI data generation, collection, and evaluation workflows.
• Design, refine, and iterate on rubrics and evaluation criteria for preference data.
• Review and label preference data while identifying subtle quality issues.
• Collaborate with researchers and engineers to translate qualitative judgment into scalable evaluation processes.
• Synthesize technical findings into clear, actionable write-ups, updates, and recommendations.

Skills

Preference data collection

Rubric creation

Large codebases navigation

AI/LLM evaluation experience

Tools

VSCode

Job description

Crossing Hurdles is seeking a Senior Software Engineer proficient in Python and TypeScript for a remote, part-time contractor assignment. You will build and maintain AI data pipelines, contribute to evaluation rubrics, and collaborate with researchers to scale qualitative judgments into reproducible processes.

The role emphasizes reviewing and labeling data quality, working within large codebases, and producing clear technical write-ups for stakeholders.