Member of Technical Staff - Image / Video Applications

Black Forest Labs; Freiburg im Breisgau; Vor Ort; EUR 40.000 - 60.000; Vollzeit

Zusammenfassung

A generative AI company is looking for an Applied Researcher to develop control mechanisms for image and video generation models. Responsibilities include training large-scale models and developing features like color palettes and transparency channels. Candidates should have strong knowledge of PyTorch and experience with large-scale model training. This position is located in Freiburg im Breisgau, Germany.

Qualifikationen

• Experience finetuning Diffusion models for image and video applications.
• Strong proficiency in transformer models and other neural network architectures.
• Experience with profiling and optimizing multi-GPU operations.

Aufgaben

• Train large-scale Diffusion models with advanced control mechanisms.
• Develop conditioning mechanisms for production needs.
• Communicate design results and decisions with the team.
• Analyze trade-offs of control architectures.

Kenntnisse

Experience training large scale Diffusion models for image and video data

Strong proficiency in PyTorch

Deep understanding of training techniques such as FSDP, low precision training

Deep understanding of evaluating image and video generative models

Jobbeschreibung

Overview

At Black Forest Labs, we're on a mission to advance the state of the art in generative deep learning for media, building powerful, creative, and open models that push what's possible. Born from foundational research, we continuously create advanced infrastructure to transform ideas into images and videos. Our team pioneered Latent Diffusion, Stable Diffusion, and FLUX.1 – milestones in the evolution of generative AI. Today, these foundations power millions of creations worldwide, from individual artists to enterprise applications.

Role and Responsibilities

We are looking for an Applied Researcher to develop precise control mechanisms for our image and video generation models, enabling users to direct outputs through practical controls like color palettes, transparency channels, and other production-ready features.

• Training large-scale Diffusion (transformer) models with advanced control mechanisms (hex color control, transparency generation, custom aspect ratios, etc.)
• Developing conditioning mechanisms for practical production requirements in image and video generation
• Rigorously ablating design choices for applied controls and communicating results & decisions with the broader team
• Reasoning about the speed and quality trade-offs of control architectures for real-world applications

What we look for

• Experience training large scale Diffusion models for image and video data
• Finetuning Diffusion models for image and video applications, such as, image and video upscalers, in and out painting models, etc.
• Deep understanding of how to effectively evaluating image and video generative models
• Strong proficiency in PyTorch, transformer models and other NN architectures
• Deep understanding of training techniques such as FSDP, low precision training, and model parallelism

Nice to have

• Experience with writing forward and backward Triton kernels and ensuring their correctness while considering floating point errors
• Profiling, debugging, and optimizing single and multi-GPU operations using tools such as Nsight or stack trace viewers