Research Scientist, Vision & Benchmarking for Real-World AI

Socket.dev; San Francisco (CA); On-site; USD 250,000 - 450,000; Full time

Benefits offered by this job

Travel stipend

Health insurance

Unlimited PTO

Parental leave

Job summary

Roboflow is seeking a Research Scientist for Roboflow Labs to connect benchmarks, models, and frontier labs. You will run failure analyses, specify data needs, and validate datasets that move real-world metrics in vision-language and multi-modal tasks.

You will publish and extend benchmarks, contribute to open models, and shape the research agenda based on millions of deployments rather than synthetic tests. This is a high-impact role with global collaboration.

Qualifications

• PhD or equivalent in computer vision, multi-modal learning, machine learning, or closely related field, with a track record of published work or shipped research.
• Hands-on experience with post-training: fine-tuning, RL, evaluation design, or data curation for large models.
• Strong engineering fundamentals — you write the code for your own experiments and can hand it to an engineer to productionize.
• Clear written communication; you can explain a failure mode to a lab researcher and a data spec to an annotator.
• Nice to have: experience at a frontier lab or as a data/evaluation partner to one; experience with vision-language models, robotics learning, or agentic coding evaluations.

Responsibilities

• Design datasets, evaluations, and RL environments that target measurable gaps in frontier vision-language, multi-modal, robotics, and coding models.
• Run structured failure analyses on partner labs' models against Roboflow's real-world task distribution and write the remediation specs that say what data will close each gap.
• Own the research behind new benchmarks that we publish.
• Validate data quality empirically — fine-tune, ablate, and measure so that every dataset we sell comes with evidence it works.
• Contribute to Roboflow's open models (RF-DETR and successors) and represent Labs within the research community.
• Work directly with research scientists at frontier labs as a technical peer, translating their model roadmaps into concrete data programs.

Skills

Fine-tuning

Reinforcement learning

Evaluation design

Data curation

Clear communication

Frontier lab experience

Education

PhD or equivalent in CV / ML / multi-modal learning

Job description

Roboflow is seeking a Research Scientist for Roboflow Labs to connect benchmarks, models, and frontier labs. You will run failure analyses, specify data needs, and validate datasets that move real-world metrics in vision-language and multi-modal tasks.

You will publish and extend benchmarks, contribute to open models, and shape the research agenda based on millions of deployments rather than synthetic tests. This is a high-impact role with global collaboration.