AI Engineer, Medical Vision Language Models

Lumexa Imaging; San Francisco (CA); Remote; USD 120,000 - 190,000; Full time

Job summary

Lumexa Imaging seeks an early-career AI Engineer to build, adapt, and deploy vision language models and imaging foundation models for clinical radiology. You will work hands-on using agentic coding tools to translate radiologist ideas into validated, deployed models, with a build-focused emphasis on model development and deployment.

Initial priorities include clinical LLM pipelines and tooling on infrastructure shared with VLM work, with access to large imaging and reporting data under privacy

Qualifications

• Fine-tune transformer models in PyTorch (vision/multimodal/language).
• Familiar with embeddings and vector search.
• Experience with open VLMs such as Qwen-VL or LLaVA.
• Daily use of agentic coding tools (Claude Code, OpenAI Codex).
• Familiarity with Docker and GPU-based model serving.
• Comfort with real-world data and reproducible experiments.

Responsibilities

• VLM development and adaptation for report drafting, impression generation, and finding detection.
• Work with vision encoders and contrastive models; build embedding pipelines for retrieval and case similarity search.
• Curate datasets from internal PACS and reporting systems with de-identification.
• Use Claude Code and OpenAI Codex to build, test, and maintain code; manage project context files.

Skills

Fine-tuning PyTorch

Embeddings & vector search

Open VLMs

Agentic coding tools

Docker

GPU serving

Education

Bachelor's degree in CS/Biomedical Engineering

Tools

Hugging Face

MONAI

DICOM tooling

Docker

Job description

Lumexa Imaging is hiring an early-career AI Engineer to help build, adapt, and deploy vision language models (VLMs) and imaging foundation models for clinical radiology. You will work hands-on using agentic coding tools (Claude Code, OpenAI Codex) to transition ideas from radiologists into validated, deployed models. This is a build-focused role centered on model development and deployment.

Initial priorities include building clinical LLM pipelines (prior report summarization, scanned document summarization, and relevant prior selection) on infrastructure shared with our VLM work.

Why this role is a rare opportunity: You will have access to a large, growing repository of clinical imaging and reporting data. A central part of this job is leveraging this real-world, domain-specific data, consistent with applicable privacy, security, and governance requirements, to develop, adapt, and evaluate AI models beyond what is possible using public data alone.

You will work closely with the Chief Clinical AI Officer, the Clinical Applications team, and Infrastructure and Information Security.

Essential Duties and Responsibilities

• VLM Development and Adaptation: Fine-tune and adapt open VLMs (e.g., Qwen-VL, LLaVA-Med, MedGemma, InternVL) and imaging foundation models for report drafting, impression generation, finding detection, and prior comparison. Apply parameter-efficient methods (LoRA, QLoRA) and full fine-tuning under senior guidance.
• Vision Encoders and Representation Learning: Work with vision encoders, contrastive image-text models (CLIP, SigLIP), and self-supervised approaches (JEPA). Build and maintain embedding pipelines for retrieval, cohort discovery, case similarity search, and drift detection.
• Data Curation: Curate and version datasets from our internal PACS and reporting systems, including DICOM header, burned-in pixel, and paired-report de-identification.
• Agentic Engineering: Use Claude Code and OpenAI Codex daily to build, test, and maintain code. Maintain project context (CLAUDE.md, AGENTS.md), and help build and run MCP servers for internal system access.
• Deployment and MLOps: Deploy models via containerized inference and GPU serving (vLLM, Triton), support CI/CD pipelines, and monitor production models for drift and performance.
• Evaluation: Build automated evaluation harnesses for clinical accuracy, negation, laterality, and hallucination scoring against radiologist ground truth.
• Workflow Automation: Build agentic and LLM driven tools for operational workflows, such as converting unstructured requests into RIS scheduling actions and supporting intelligent operations dashboards.

Required Qualifications

• Hands-on experience fine-tuning transformer models in PyTorch (vision, multimodal, or language) using Hugging Face, MONAI, or similar (via internships, research, open-source work, or projects).
• Working knowledge of embeddings and vector search.
• Experience with open VLMs such as Qwen-VL or LLaVA.
• Daily use of agentic coding tools (Claude Code, OpenAI Codex).
• Familiarity with Docker and GPU-based model serving.
• Comfort working with real-world data and a reproducible approach to experiments.

Preferred Qualifications

• Hands-on experience with vision encoders and contrastive models (CLIP, SigLIP, JEPA).
• Exposure to medical VLMs, imaging foundation models, or self-supervised methods (DINOv2, MAE).
• Experience building MCP servers in Python or TypeScript.
• Familiarity with medical imaging formats (DICOM, HL7, FHIR) and HIPAA/PHI handling.
• Awareness of MLOps tooling (MLflow, Weights & Biases, model registries).
• Awareness of FDA considerations for clinical AI (510(k), PCCP).

Education/Licensing/Certification

Degree in Computer Science, Biomedical Engineering, or a related field (or equivalent practical experience).

Experience

Experience 0 to 3 years (internships, research, or projects count).

Travel Required

Minimal, up to 10%

Physical Requirements & Work Environment

Standard remote office work