Remote AI Benchmark & Datasets Engineer

Ignite Next GmbH; Palo Alto, Northern (CA, KY); Hybrid; USD 120,000 - 190,000; Full time

Benefits offered by this job

Remote work

Office visits Palo Alto, Paris, Wroclw

Job summary

Pathway, headquartered in Palo Alto, is seeking a benchmarking-focused ML researcher. You will design and run rigorous benchmarks for post-transformer models, curate datasets, and translate research into repeatable specs for engineering and GTM teams.

You will collaborate with our R&D and product groups, track model performance, and help shape demonstrations and customer-ready results that showcase our evaluation framework. Fluency in English and strong communication are essential.

Qualifications

• Publish at NeurIPS, ICLR or ICML as lead author or with significant contributions.
• Contributed to an LLM training effort that gained notable attention.
• Spent at least 6 months in a leading ML research center (e.g., Google Brain, DeepMind, Apple, Meta, Anthropic, Nvidia).
• ICPC World Finalist or IOI/IMO/IPhO medalist in high school.
• Experience with ML/LLM evaluation, data science or technical product roles focused on benchmarks.

Responsibilities

• Design and execute benchmarks and define dataset standards.
• Collaborate with R&D to build evaluation infrastructure for post‑transformer models.
• Identify, curate benchmarks and develop repeatable benchmark specs.
• Maintain documentation and track benchmark results and leaderboards.
• Contribute to demos and public proof points based on benchmark outcomes.

Skills

Fluent English

Benchmarking

LLM Evaluation

Reading Papers

Interdisciplinary Communication

Job description

Pathway, headquartered in Palo Alto, is seeking a benchmarking-focused ML researcher. You will design and run rigorous benchmarks for post-transformer models, curate datasets, and translate research into repeatable specs for engineering and GTM teams.

You will collaborate with our R&D and product groups, track model performance, and help shape demonstrations and customer-ready results that showcase our evaluation framework. Fluency in English and strong communication are essential.