ML Evaluation Architect (Remote, Part-Time)

Alignerr Corp.; Toronto; Remote; CAD 96,000 - 165,000; Full time

Job summary

Alignerr Corp. is seeking domain experts with deep ML knowledge to design evaluation challenges that push AI systems to their limits. Your work will influence how models are measured, trained, and improved, with remote collaboration and flexible hours.

Ideal candidates have MS/PhD in a relevant field, strong ML knowledge, and a track record of research. This hourly contract offers autonomy and opportunities for extended research involvement.

Qualifications

• Graduate-level expertise (MS or PhD preferred) in a scientific/technical field intersecting ML.
• Strong working knowledge of ML methods including model selection, feature engineering, evaluation metrics, and pipeline design.
• Deep familiarity with active open research problems in your field.
• Sharp eye for where general ML knowledge breaks down and domain insight is essential.
• Experience publishing or conducting original research is highly valued.
• Excellent written communication for precise, nuanced problems.
• Self-motivated and energized by independent, intellectually demanding work.

Responsibilities

• Design complex, original machine learning problems rooted in your domain expertise.
• Create evaluation tasks that demand advanced knowledge beyond standard ML pipelines.
• Draw from your own research to craft challenges testing capable AI models.
• Write clear problem statements, define evaluation criteria, and establish gold-standard solutions.
• Assess AI-generated solutions for correctness, creativity, and rigor.
• Document problem difficulty, domain knowledge required, and expected failure modes.
• Collaborate asynchronously with a global team of researchers and engineers.

Skills

ML methods

Research familiarity

Domain insight

Research publications

Written communication

Independent work

Education

MS or PhD preferred

Job description

Alignerr Corp. is seeking domain experts with deep ML knowledge to design evaluation challenges that push AI systems to their limits. Your work will influence how models are measured, trained, and improved, with remote collaboration and flexible hours.

Ideal candidates have MS/PhD in a relevant field, strong ML knowledge, and a track record of research. This hourly contract offers autonomy and opportunities for extended research involvement.