AI Benchmark Designer (Remote)

Weekday AI; United States; Remote; USD 83,000 - 103,000; Full time

Job summary

Weekday AI is seeking an experienced contractor for an advanced AI research initiative focused on how professionals’ documents are understood and how instructions are executed. You will design realistic, multi-step benchmark tasks based on technical documentation, code repositories, API references, and architecture diagrams to evaluate AI performance in software engineering and data science.

This fully remote, independent role offers flexible hours (15–20 hours per week) and weekly payments.

Qualifications

• Minimum 3 years of hands-on professional experience in software engineering, data science, or data analytics.
• Experience with codebases, APIs, technical specifications, or system architecture documentation.
• Strong written communication and ability to create clear technical instructions and evaluation criteria.
• Ability to work independently with high accuracy and consistency.

Responsibilities

• Design AI evaluation tasks based on professional technology workflows.
• Develop challenges using technical specifications, architecture documents, API documentation, codebases, web research, and code execution.
• Ensure each task includes a clearly defined output and evaluation criteria.
• Write ground-truth solutions and structured scoring rubrics.
• Collaborate with research teams to improve benchmark quality and consistency.
• Operate as an independent contractor with flexible remote hours (15–20 hours per week).

Skills

Software Engineering

Data Science

Data Analytics

Job description

Weekday AI is seeking an experienced contractor for an advanced AI research initiative focused on how professionals’ documents are understood and how instructions are executed. You will design realistic, multi-step benchmark tasks based on technical documentation, code repositories, API references, and architecture diagrams to evaluate AI performance in software engineering and data science.

This fully remote, independent role offers flexible hours (15–20 hours per week) and weekly payments.