Research Scientist - World Model

Luma; Redwood City (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Luma in California seeks a researcher to turn its generative video models into world models: interactive, controllable, physically faithful, and useful for embodied reasoning. You will invent architectures (diffusion/transformer/AR hybrids), develop action/view conditioning, define metrics for fidelity and coherence, run scaling studies, and publish toward an open-source release.

This role targets PhD-level expertise in ML, CV, or robotics with strong PyTorch skills and a track record of

Qualifications

• PhD or equivalent research record in ML, CV, robotics, or related field.
• Strong expertise in large-scale generative modeling, self-supervised learning, or model-based RL.
• Proficient PyTorch and multi-node distributed training experience.
• Proven research track record with top venues or widely used releases.

Responsibilities

• Invent world-model architectures focused on controllability and physical consistency.
• Develop controllability mechanisms for action and view conditioning.
• Define metrics for physical fidelity, long-horizon coherence, and policy usefulness.
• Publish at the frontier and contribute to the open-source release.
• Run scaling studies and push promising directions toward open release.

Skills

Large-scale generative modeling

Self-supervised learning

Model-based RL

PyTorch & distributed training

Research publications

Education

PhD in ML/CV/Robotics

Job description

You'll turn Luma's industry-leading generative video models into world models: interactive, controllable, physically faithful, and useful as a substrate for embodied reasoning. This is the role at the center of the thesis.

You'll invent next-generation world-model architectures and the controllability that lets an agent step into a generated world, and own the metrics that define success. It fits a researcher with deep generative-modeling or model-based-RL expertise who has trained models to the limits of a multi-node cluster. If you want a narrow, well-scoped research problem, this is broader and more open-ended than that.

What You'll Own

• Invent next-generation world-model architectures (diffusion, transformer, autoregressive, or hybrid), focused on controllability and physical consistency.
• Develop controllability mechanisms — action conditioning, view conditioning, long-horizon rollouts — that let an agent step into the world.
• Define and own the metrics: physical fidelity, long-horizon coherence, action-following, and downstream usefulness for policy training.
• Run scaling studies that show where compute, data, and architecture pay off.
• Publish at the frontier and contribute to the open-source release that is the long-term deliverable.

First 90 Days

One way the first 90 could unfold.

• Days 1–30 — Immerse & Diagnose: Get deep on the current video models and where they fall short as world models.
• Days 30–60 — Ship & Validate: Prototype a controllability mechanism or architecture change and measure it against physical-fidelity and action-following metrics.
• Days 60–90 — Scale & Systemize: Run scaling studies and push the most promising direction toward the open release.

What You Bring

• PhD or equivalent research record in ML, computer vision, robotics, or a related field.
• Deep expertise in at least one of: large-scale generative modeling (video/3D/world), self-supervised representation learning, or model-based RL.
• Strong PyTorch and large-scale training experience, to the limits of a multi-node cluster.
• A research record the field knows (top-venue publications and/or widely used open releases).

Nice to Have

• Prior work on world models, model-based RL, generative video, neural simulation, or 4D scene representations.
• Experience using generative models for downstream embodied tasks (planning, control, evaluation).
• Enthusiasm for open-sourcing frontier models.

About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.