Vision & Vision-Language ML Research Engineer

Bison; San Francisco (CA); On-site; USD 140,000 - 230,000; Full time

Benefits offered by this job

Medical, dental, vision

401(k) plan

Relocation support

In-office snacks

Unlimited PTO

San Francisco office

Job summary

Zyphra in San Francisco is seeking a Research Engineer - Vision & Vision-Language Models to contribute across the full model development stack, from data sourcing to pretraining, post-training, and evaluation.

You will work on end-to-end training of vision-language models, multi-turn visual tasks, and experiments across architecture and training methods, collaborating with researchers and engineers.

Qualifications

• Experience with vision, vision-language, video, or other multimodal models or datasets is highly valuable.
• Experience training vision-language models is a plus, but not required.
• Experience with large-scale model pretraining, post-training, reinforcement learning, or related methodologies.
• Experience sourcing, generating, or curating vision datasets, including synthetic data generation.
• Familiarity with image processing tools, libraries, or pipelines.
• Experience with computer use agents, visual agents, GUI understanding, or other multi-turn multimodal tasks is a plus.
• Experience with frontend development, web frameworks, UI/UX implementation, or mobile development is valuable, particularly for computer use and interface-understanding tasks.
• Experience training models on large-scale, multi-node GPU clusters
• Strong grasp of experimental methodology for running rigorous ablations and hypothesis testing
• Understanding of large-scale, highly parallel data processing pipelines
• Proficiency with PyTorch and Python
• Strong ability to dive into large pre-existing codebases and rapidly get up to speed
• Published machine learning research in well-respected venues is a plus
• Postgraduate study in a scientific or technical field such as Computer Science, EE/EECS, Mathematics, Physics, or Machine Learning is a plus, but we care more about demonstrated ability and trajectory than credentials

Responsibilities

• End-to-end pretraining and post-training of vision-language models
• Large-scale vision and multimodal training runs
• Vision and multimodal dataset sourcing, generation, curation, processing, and evaluation
• Reinforcement learning and other post-training methodologies for vision-language models
• Computer use, visual agents, and other multi-turn agentic vision tasks
• Architecture, data, and training methodology ablations and improvements
• Performance optimization of our training and data processing stacks

Skills

Research intuition

Prototyping ability

Rapid learning

ML intuition

Communication

Collaboration

Education

Postgraduate study in CS/EE/Math/Physics/ML

Tools

PyTorch

Python

Job description

Zyphra in San Francisco is seeking a Research Engineer - Vision & Vision-Language Models to contribute across the full model development stack, from data sourcing to pretraining, post-training, and evaluation.

You will work on end-to-end training of vision-language models, multi-turn visual tasks, and experiments across architecture and training methods, collaborating with researchers and engineers.