Remote AI Benchmark Engineer & Researcher

Pathwaycom; Palo Alto (CA); Remote; USD 120,000 - 200,000; Full time

Benefits offered by this job

Remote work

Office visits

Job summary

Pathway is hiring to design and execute rigorous benchmarks for its post-transformer models. You will collaborate with our R&D team to build evaluation infrastructure, curate benchmarks, and ensure reproducible, high-quality data and results.

Join a fast-moving startup with global reach, offering remote work and opportunities to influence model evaluation practices, datasets, and customer-facing demonstrations.

Qualifications

• Experience with ML/LLM evaluation and benchmarks.
• Ability to read papers and translate to benchmark specs.
• Excellent written and verbal communication.

Responsibilities

• Design and execute rigorous benchmarks and define dataset standards.
• Collaborate with R&D to build evaluation infrastructure guiding model evolution.
• Identify and curate relevant benchmarks across target use cases and markets.
• Evaluate benchmark candidates for data quality, methodology, and roadmap fit.
• Run benchmarks with baseline models to validate setups and uncover edge cases.
• Deliver benchmark-ready packages to R&D ( specs, data, scripts, metrics, constraints ).
• Maintain vocabulary and documentation around benchmarks and evaluation formats.

Skills

ML/LLM evaluation

Benchmarks

Data science

Technical product roles

Reading papers

Tools

GitHub

Job description

Pathway is hiring to design and execute rigorous benchmarks for its post-transformer models. You will collaborate with our R&D team to build evaluation infrastructure, curate benchmarks, and ensure reproducible, high-quality data and results.

Join a fast-moving startup with global reach, offering remote work and opportunities to influence model evaluation practices, datasets, and customer-facing demonstrations.