Remote AI Benchmark Engineer & Researcher

Pathway; United States; À distance; USD 120 000 - 180 000; Plein temps

Avantages offerts par ce poste

Remote work

Office visits

Résumé du poste

Pathway is seeking an experienced professional to design and execute rigorous benchmarks for post-transformer model evaluation. You will collaborate with our R&D team to build evaluation infrastructure, define dataset standards, and drive benchmark readiness for GTM and customers.

You will identify and curate benchmarks, evaluate their quality, and translate findings into clear specs. You will engage with engineers and customers, ensuring data quality and reproducibility while documenting

Qualifications

• Experience with ML/LLM evaluation, data science, or technical product roles, ideally around benchmarks or experimentation.
• Comfort reading papers, leaderboards, and GitHub repos, and turning them into clear, repeatable benchmark specs.
• Talks with engineers and customers, translating between technical detail and business value.
• Care about high-quality data, reproducible experiments, and crisp documentation.
• Fluent in English.

Responsabilités

• Design and execute rigorous benchmarks and define dataset standards.
• Collaborate with R&D to build evaluation infrastructure guiding Pathway’s models.
• Maintain benchmarks, datasets, and evaluation formats for GTM and R&D use.
• Contribute to demos and public-facing proof points based on benchmark outcomes.

Connaissances

ML/LLM evaluation

Data science

Technical product roles

Benchmarks

Reading papers

GitHub repos

Description du poste

Pathway is seeking an experienced professional to design and execute rigorous benchmarks for post-transformer model evaluation. You will collaborate with our R&D team to build evaluation infrastructure, define dataset standards, and drive benchmark readiness for GTM and customers.

You will identify and curate benchmarks, evaluate their quality, and translate findings into clear specs. You will engage with engineers and customers, ensuring data quality and reproducibility while documenting