Computer Vision Research Scientist

Frisson Labs; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Frisson Labs in San Francisco is seeking a research scientist to advance companions that can follow a match, understand player intent, and respond. You will read changing scenes, track events over time, and link perception to action.

You will publish and implement ideas in Python and PyTorch, collaborate with teams building speech and memory systems, and push for efficient inference under latency constraints.

Qualifications

• PhD in computer vision, machine learning, computer science, or a related field from a leading research university.
• Strong publication record at major research conferences such as CVPR, ICCV, ECCV, NeurIPS, ICLR, or ICML.

Responsibilities

• Develop video and vision-language models that identify objects, actions, spatial relationships, and meaningful events in gameplay.
• Research temporal representations that preserve relevant context across frames and long sessions, including changes in goals, progress, and player behavior.
• Build training datasets and evaluation tasks from gameplay observations.
• Design controlled experiments and ablations to measure visual grounding, event recognition, robustness across games, and the effect of perception errors on companion behavior.
• Improve inference efficiency so the companion can process visual context alongside live conversation and game interaction.
• Turn research findings into working models and reproducible results, in collaboration with the people building the companion’s speech, memory, and action systems.

Skills

Python

PyTorch

Video models

Vision-language models

Multimodal transformers

Independent research judgment

Education

PhD in computer vision, machine learning, computer science, or a related field

Job description

We are building companions that can follow a match, understand what a player is trying to do, and participate. This role develops the visual representations that make that possible: reading a changing scene, tracking events over time, and connecting what the companion sees to its next action or response.

What you'll work on

• Develop video and vision-language models that identify objects, actions, spatial relationships, and meaningful events in gameplay.
• Research temporal representations that preserve relevant context across frames and long sessions, including changes in goals, progress, and player behavior.
• Build training datasets and evaluation tasks from gameplay observations. Study where supervision from demonstrations, synthetic data, and interaction improves generalization.
• Design controlled experiments and ablations to measure visual grounding, event recognition, robustness across games, and the effect of perception errors on companion behavior.
• Improve inference efficiency so the companion can process visual context alongside live conversation and game interaction.
• Turn research findings into working models and reproducible results, in collaboration with the people building the companion’s speech, memory, and action systems.

What you'll bring

• A PhD in computer vision, machine learning, computer science, or a related field from a leading research university.
• A strong publication record at major research conferences such as CVPR, ICCV, ECCV, NeurIPS, ICLR, or ICML.
• Deep experience with modern visual representation learning, including video models, vision-language models, or multimodal transformers.
• The ability to implement research ideas in Python and PyTorch, train models, diagnose failures, and run careful evaluations.
• Evidence of independent research judgment: choosing a useful question, establishing credible baselines, and explaining what the results support.

Helpful experience

• Research on visual grounding, action recognition, video understanding, embodied learning, or learning from human demonstrations.
• Experience with game environments, interactive systems, or visual policies that act from screen observations.
• Experience making models practical under latency and memory constraints, including distillation, quantization, or GPU profiling.
• Open-source research code, datasets, or evaluation tools that other researchers use.