Remote ML Benchmark Engineer & Evaluation Specialist

Weekday 1; United States; Remote; USD 83,000 - 124,000; Full time

Job summary

Weekday 1 is seeking experienced ML Engineers and Researchers for a fully remote, independent contractor role focused on building evaluation benchmarks for frontier AI models. You will design multi-step benchmarks, implement experiments, and analyze model behavior.

The position is about 35 hours per week, paid hourly at $60-$90, with weekly payments. It requires strong Python/Git skills, ML research experience, and familiarity with LLMs and reinforcement learning.

Qualifications

• Master's degree, PhD, or equivalent in a quantitative STEM discipline.
• Minimum 1 year of professional experience in ML research or related field.
• Hands‑on experience designing, training, and evaluating ML models through full experimental workflows.
• Experience with ML experiments: setup, tuning, execution, validation, analysis.
• Strong understanding of modern LLMs and evaluation methodologies.
• Proficiency in Python and Git; experience in script-based and notebook environments.
• Familiarity with reinforcement learning concepts is preferred.
• Experience in AI evaluation, benchmark development, or research experimentation is highly desirable.
• Excellent analytical thinking, creativity, attention to detail, and ability to work independently.

Responsibilities

• Design realistic ML benchmark tasks based on research workflows.
• Translate research concepts into structured, reproducible evaluation tasks.
• Implement ML solutions in Python, run experiments, and provide reference implementations.
• Develop benchmark tasks involving reinforcement learning concepts where applicable.
• Evaluate AI solutions by identifying errors, flaws, and unsupported conclusions.
• Collaborate with researchers to improve benchmark quality and evaluation consistency.

Skills

LLMs

Reinforcement learning

Experimentation

Analytical thinking

Education

Master's degree, PhD, or equivalent in ML/CS/AI/Data Science

Tools

Python

Git

Job description

Weekday 1 is seeking experienced ML Engineers and Researchers for a fully remote, independent contractor role focused on building evaluation benchmarks for frontier AI models. You will design multi-step benchmarks, implement experiments, and analyze model behavior.

The position is about 35 hours per week, paid hourly at $60-$90, with weekly payments. It requires strong Python/Git skills, ML research experience, and familiarity with LLMs and reinforcement learning.