Research Scientist / Engineer - Controllability, Personalization & Productization

lumalabs-ai; New York (NY); On-site; USD 200,000 - 450,000; Full time

Job summary

Luma seeks a fullstack researcher to refine and deploy foundation-model capabilities for creative workflows in New York. You will bridge research, product, and partnerships to ship expressive, controllable, and personalized model behaviors.

You’ll work across modeling, data, systems, and evaluation, using SFT, RL, distillation, and adapters to enable high-fidelity, partner-grade solutions while grounding data in real-world usage and feedback.

Qualifications

• Strong foundation in machine learning with visual generative models.
• Experience with fine-tuning, personalization, or data curation.
• Hands-on experience with PyTorch and large model training.

Responsibilities

• Develop controllability and features using SFT, RL, distillation, and adapters.
• Implement personalization with context management and PEFT.
• Define end-user quality metrics and evaluations.
• Collaborate with Product and Design to translate feedback into specs and ship models.

Skills

ML foundations

Visual generative models

Fine-tuning / personalization

Context management / PEFT

End-user research

Tools

PyTorch

Ray

Slurm

Kubernetes

Job description

About the Role

This is a foundational opportunity to refine, personalize, and build the final capabilities and control interface of Luma’s foundation models and drive real-world value.

You’ll sit at the intersection of research, product, and partnerships, helping close the gap between generalist capability and production needs. Your mission is to make our video foundation models more expressive, controllable, and personalized – solving the “last mile” challenges demanded by top-tier creative workflows.

What You’ll Do

You will work as a fullstack researcher across modeling, data, systems, and evaluation to translate foundational capabilities into the personalized and high-fidelity tools required by top-tier creative production.

• Controllability and Features: You will use SFT, RL, distillation, and adapter-based methods to give Luma’s models precise control and creative capabilities for high-fidelity, partner-grade workflows.
• Personalization: You will use context management, PEFT, and preference learning to embed domain expertise and long-horizon memory into our models. You’ll ground our data engine in real-world workflows, translating usage into training and data insights that shape future models.
• End-User Quality: You will define and drive end-user quality – setting success metrics, building user-aligned evaluations, and iterating on the model/data/evals loop to meet strict fidelity and reliability targets in specific verticals.
• Cross-functional Collaboration: You will partner with Product and Design to turn creative intent and user feedback into clear specs and shippable model behaviors, delivering intuitive control and dependable capabilities for users and partners.

Who You Are

• Strong foundation in machine learning with deep experience in visual generative models (diffusion/transformers or related architectures). Ideal candidates also have a deep understanding of at least one: fine-tuning, personalization, domain adaptation, data curation, targeted distillation, interpretability, or human-feedback-driven refinement.
• You are a product-obsessed researcher/engineer. You treat end users and partners as collaborators and enjoy solving specific “last mile” problems – not just optimizing public metrics.
• Hands-on experience with PyTorch and large model training.

Bonus Points

• Contributions to state-of-the-art models in image/video generation.
• Experience collaborating with creative partners (VFX, animation, film, design tools).
• Track record building workflows/tools that materially improve iteration speed and evaluation rigor.
• Familiarity with large-scale training infrastructure and distributed systems (Ray, Slurm, Kubernetes).

Compensation

The base pay range for this role is $200,000 – $450,000 per year.