AI Benchmark Architect

Pathway; Paris; On-site; EUR 70,000 - 90,000; Full time

Job summary

Pathway is seeking a seasoned colleague to design and execute rigorous benchmarks for post-transformer models and to define dataset standards.

You will work closely with our R&D team to build evaluation infrastructure guiding the evolution of Pathway's models, curate benchmarks, assess data quality and methodology, and hand off benchmark-ready packages to R&D. You will also help maintain a shared benchmarks vocabulary and track model leaderboards for diverse customers.

Job description

Pathway is seeking a seasoned colleague to design and execute rigorous benchmarks for post-transformer models and to define dataset standards.

You will work closely with our R&D team to build evaluation infrastructure guiding the evolution of Pathway's models, curate benchmarks, assess data quality and methodology, and hand off benchmark-ready packages to R&D. You will also help maintain a shared benchmarks vocabulary and track model leaderboards for diverse customers.