ML Evaluation Engineer - Reliable, Reproducible Benchmarks

Reinforced Learning; Town of Tonawanda (NY); Presencial; USD 140 000 - 200 000; Tempo integral

Resumo da oferta

Reinforced Learning is seeking an engineer to design and run robust evaluation systems. You will build the evaluation harness, graders, and reporting that enable hundreds of tasks across environments with reproducible results.

You will own the harness, scoring, and dashboards, ensuring deterministic, resumable, and cost-efficient runs and producing clear, publishable reports for researchers and customers.

Qualificações

• Strong software engineering in Python with an eye for reliability.
• Experience with model APIs, batch systems and asynchronous processing.
• You care about reproducibility and know how to achieve it.
• Comfort reading research code and making it production-grade.

Responsabilidades

• The evaluation harness: task loading, environment orchestration, model adapters, scoring.
• Graders and verifiers implemented with the environment and verifier engineers.
• Runs that are deterministic, resumable and cheap to repeat.
• Dashboards and reports that researchers and customers read.
• Discrepancies between runs, chased until they’re explained.

Conhecimentos

Python

Reliability engineering

Model APIs

Batch systems

Asynchronous processing

Reproducibility

Production-grade code reading

Ferramentas

Containers

Cloud infrastructure

Descrição da oferta de emprego

Reinforced Learning is seeking an engineer to design and run robust evaluation systems. You will build the evaluation harness, graders, and reporting that enable hundreds of tasks across environments with reproducible results.

You will own the harness, scoring, and dashboards, ensuring deterministic, resumable, and cost-efficient runs and producing clear, publishable reports for researchers and customers.