ML Platform Engineer - End-to-End Model Evaluation

Runway; United States; Remote; USD 240,000 - 290,000; Full time

Job summary

Runway is seeking an ML infrastructure engineer to own model evaluation end to end. You will design and build the platform that generates samples at scale, scores them with automated metrics, and presents results to researchers within minutes.

You’ll work with research teams across video, image, audio, agents, and robotics to measure what matters and generalize results into the platform. This high‑leverage role sits in the ML Platform team, shaping standards for reproducibility and reliability

Qualifications

• 5+ years of experience building ML infrastructure or data platforms in production.
• Strong Python and PyTorch; hands-on experience with large batch GPU workloads on Kubernetes.
• Experience designing data pipelines and storage for large volumes of media or model outputs with emphasis on versioning and reproducibility.
• Familiarity with experimental statistics: paired comparisons, confidence intervals, multiple‑comparison pitfalls.
• Comfort building internal tools end to end, from CLI to browser.
• Ability to lead a broad technical area and drive a roadmap.
• Familiarity with the full model development lifecycle: data, training, evaluation, serving.
• Self‑starter who can work embedded with research teams and move fast.
• Strong systems thinking and production reliability mindset.
• Humility and openness to learn from others.

Responsibilities

• Own the evaluation platform end to end: tooling and systems to generate, annotate, review and adapt at frontier scale.
• Define CLIs, APIs, GUIs and storage layers to make the process seamless and collaborative.
• Collaborate with research teams on video, image, audio, agents, and robotics to measure needs and generalize into the platform.
• Set standards for evaluation: reproducibility, metrics, reporting formats, and trustworthy results.
• Support broad adoption of the platform across training and production model serving.
• Contribute to tools and systems that help Runway train and serve frontier models.

Skills

ML infra experience

Python & PyTorch

Kubernetes workloads

Data pipelines

Experimental stats

Internal tools

Technical leadership

Model lifecycle

Self-starter

Systems thinking

Humility

Tools

Kubernetes

PyTorch

Python

Job description

Runway is seeking an ML infrastructure engineer to own model evaluation end to end. You will design and build the platform that generates samples at scale, scores them with automated metrics, and presents results to researchers within minutes.

You’ll work with research teams across video, image, audio, agents, and robotics to measure what matters and generalize results into the platform. This high‑leverage role sits in the ML Platform team, shaping standards for reproducibility and reliability