Remote AI Benchmark & Datasets Engineer

Pathway Genomics Corporation; Palo Alto (CA); Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Intellectually stimulating work environment

Work with a pioneering AI startup

Flexible remote work options

Job summary

An innovative AI startup is seeking a Benchmark Specialist to design and execute rigorous benchmarks and evaluate datasets for their post-transformer models. This role requires strong experience in ML/LLM evaluation and the ability to communicate technical specifications to both engineers and customers. The position is full-time and offers remote work flexibility. If you are passionate about high-quality data and groundbreaking research, this opportunity could be for you.

Qualifications

• Experience with ML/LLM evaluation, data science, or technical product roles, ideally around benchmarks.
• Comfortable reading papers, leaderboards, and Github repos.
• Ability to communicate technical details in business terms.

Responsibilities

• Design and execute rigorous benchmarks.
• Evaluate candidate benchmarks for clarity and quality.
• Organize benchmark results and maintain shared documentation.

Skills

ML/LLM evaluation

Data science

Technical product roles

High-quality data

Job description

An innovative AI startup is seeking a Benchmark Specialist to design and execute rigorous benchmarks and evaluate datasets for their post-transformer models. This role requires strong experience in ML/LLM evaluation and the ability to communicate technical specifications to both engineers and customers. The position is full-time and offers remote work flexibility. If you are passionate about high-quality data and groundbreaking research, this opportunity could be for you.