Research Engineer / Research Scientist, Vision-Language Model Pretraining

Alt; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Epsilon Health in San Francisco seeks a Research Scientist to push the frontier of vision-language models for medical imaging. You will own the pretraining of multimodal foundation models, developing architectures and training methods that enable AI to understand complex medical images and generate clinically accurate radiology reports.

You’ll tackle multimodal learning challenges across X-rays, CT, and MRI, drawing on real-world clinical data and collaborating with researchers and engineers to

Qualifications

• Deep expertise in vision-language models or multimodal learning.
• Ability to develop, train, or improve state-of-the-art ML models.
• Strong understanding of modern VLM architectures and vision-language alignment.
• Experience with large-scale distributed training using PyTorch or JAX.
• Independent formulation of research questions, design experiments, and turn ideas into working models.
• Strong engineering skills and a track record of implementing research concepts.
• Rigid, experimental mindset and ambition to solve difficult problems.

Responsibilities

• Advance the frontier of multimodal AI by designing, pretraining, and scaling vision-language models for medical imaging.
• Invent new architectures and explore fusion approaches for high-resolution image understanding.
• Develop multimodal training objectives and scaling strategies across image-text data.
• Improve visual grounding to relate findings to specific regions and structures in images.
• Own research end to end from hypothesis through large-scale training, evaluation, and handoff to post-training.
• Define rigorous evaluations for visual understanding and model performance.
• Publish original research and contribute to the medical AI field.

Skills

Vision-language models

Multimodal learning

Pretraining

PyTorch/JAX

Experiment design

Research leadership

Engineering skills

Job description

About Us

Epsilon Health is transforming medical imaging and diagnostics by bringing frontier AI into clinical practice. We're tackling some of healthcare's most critical challenges, building technology that directly improves patient outcomes.

This year, we've served over 300,000 patients and now serve 3,000+ daily, working with some of the largest medical institutions in the U.S. By year-end, we're on track to interpret 1% of all daily U.S. X-rays.

Our research team brings experience from DeepMind and Meta, powered by one of the industry's most comprehensive proprietary medical imaging datasets, continuously enriched by real-world clinical data. Together, we're building a future where frontier AI helps save millions of lives today, not tomorrow.

Recently out of stealth, we're proudly backed by Jack Altman, Uncork Capital, Renegade Partners, and AlleyCorp, one of NYC's most active healthcare venture firms.

About the Role

We're looking for an exceptional Research Scientist to push the frontier of vision-language models for medical imaging.

You'll own the pretraining of our multimodal foundation models, developing architectures and training methods that enable AI to understand complex medical images and generate clinically accurate radiology reports.

You'll tackle fundamental research challenges in multimodal learning, high-resolution visual understanding, and large-scale pretraining across X-rays, CT, and MRI.

Unlike traditional research labs, your work will draw on continuously growing real-world clinical data and help power models deployed into clinical practice, creating a direct feedback loop between research and patient care.

You'll work alongside a small team of researchers and engineers, with the freedom to pursue ambitious ideas and the resources to bring them to life. You'll collaborate closely with researchers responsible for post-training and reinforcement learning.

What You'll Do

• Advance the frontier of multimodal AI. Design, pretrain, and scale vision-language foundation models for medical imaging.

• Invent new architectures. Explore novel approaches to vision-language fusion, high-resolution image understanding, and efficient visual token representations.

• Push the limits of pretraining. Develop multimodal training objectives, data mixtures, and scaling strategies across large-scale image-text datasets.

• Solve visual grounding. Improve how models connect medical findings to specific regions, structures, and abnormalities within images.

• Own research end to end. Take ideas from hypothesis and experimentation through large-scale distributed training, evaluation, and handoff to post-training.

• Define how progress is measured. Develop rigorous evaluations for visual understanding, grounding, generalization, and downstream model performance.

• Contribute to the research community. Publish original research, share technical insights, and help shape the direction of medical AI.

What We're Looking For

We care more about what you've built and discovered than where you've worked or how many years you've spent doing it.

• Deep expertise in vision-language models, multimodal learning, or large-scale foundation model pretraining.

• Demonstrated ability to develop, train, or improve state-of-the‑art machine learning models.

• Strong understanding of modern VLM architectures, multimodal training objectives, and vision-language alignment.

• Experience with large-scale distributed training using PyTorch or JAX.

• Ability to independently formulate research questions, design experiments, and turn promising ideas into working models.

• Strong engineering skills and a track record of implementing and extending ideas from research.

• A rigorous, experimental mindset and the ambition to solve difficult, unsolved problems.

Nice to Have

• Publications at leading ML or computer vision conferences such as NeurIPS, ICML, ICLR, CVPR, or MICCAI.

• Experience developing or scaling models such as LLaVA, Qwen-VL, InternVL, or similar multimodal architectures.

• Experience training vision encoders or developing novel vision-language connector architectures.

• Familiarity with high-resolution imaging, visual grounding, or 3D/volumetric data.

• Experience with contrastive learning, generative pretraining, or large-scale multimodal data curation.

• Background in medical imaging, radiology, or other scientific imaging applications.

• Experience evaluating model factuality, interpretability, or reliability in high-stakes settings.

We welcome exceptional researchers from both academic and industry backgrounds. A traditional career path or prior healthcare experience is not required.