Senior AI Evaluation Engineer — Remote, Part-Time

Crossing Hurdles; United States; Teletrabalho; CAD 138 000 - 207 000; Tempo parcial

Resumo da oferta

Crossing Hurdles is seeking a Senior Software Engineer (Python/TypeScript) on a contractor basis. The role is remote and part-time (20-40 hours/week) with compensation of $100–$150 per hour.

The position focuses on building and maintaining AI data pipelines, refining evaluation rubrics, and collaborating with researchers to scale evaluation processes. Strong experience with large codebases and a LinkedIn profile is required.

Qualificações

• Hands-on experience with preference data collection or rubric creation.
• Experience navigating large codebases; VSCode familiarity is a plus.
• Experience with AI/LLM evaluation workflows preferred.
• LinkedIn profile is mandatory for review.
• Candidates should not currently be working with Microsoft.

Responsabilidades

• Build and maintain pipelines and infrastructure for AI data generation, collection, and evaluation workflows.
• Design rubrics and evaluation criteria for preference data.
• Review and label preference data, identifying quality issues.
• Collaborate with researchers and engineers to translate judgments into scalable processes.
• Synthesize findings into clear, actionable write-ups and recommendations.

Conhecimentos

Preference data collection

Rubric creation

Codebase navigation

AI/LLM evaluation

Ferramentas

VSCode

Descrição da oferta de emprego

Crossing Hurdles is seeking a Senior Software Engineer (Python/TypeScript) on a contractor basis. The role is remote and part-time (20-40 hours/week) with compensation of $100–$150 per hour.

The position focuses on building and maintaining AI data pipelines, refining evaluation rubrics, and collaborating with researchers to scale evaluation processes. Strong experience with large codebases and a LinkedIn profile is required.