AI Benchmark Architect — Remote Data & Evaluation

Pathway Genomics Corporation; Palo Alto (CA); Remote; USD 140,000 - 190,000; Full time

Job summary

Pathway Genomics Corporation in Palo Alto, CA is seeking a benchmark-focused ML/LLM evaluation engineer to design and run rigorous benchmarks and to help shape evaluation standards for our post-transformer models.

You will build evaluation infrastructure, curate benchmarks, analyze results, and collaborate with R&D to guide model development and customer-ready demos, with remote work options.

Qualifications

• Experience with ML/LLM evaluation and benchmarking.
• Ability to read papers, leaderboards, and GitHub repos.

Responsibilities

• Design and execute rigorous ML/LLM benchmarks across use cases.
• Build evaluation infrastructure and benchmark datasets.
• Collaborate with R&D and translate results to business value.
• Maintain benchmarks docs and model leaderboards.

Skills

ML/LLM evaluation

Benchmarking

Documentation

Communication

Tools

GitHub

Job description

Pathway Genomics Corporation in Palo Alto, CA is seeking a benchmark-focused ML/LLM evaluation engineer to design and run rigorous benchmarks and to help shape evaluation standards for our post-transformer models.

You will build evaluation infrastructure, curate benchmarks, analyze results, and collaborate with R&D to guide model development and customer-ready demos, with remote work options.