Jay Alammar@JayAlammar
Nov 25, 2022
6 tweets
Big update to "The Illustrated Stable Diffusion" post
jalammar.github.io/illustrated-st
14 new and updated visuals.
The biggest update is that forward diffusion is more precisely explained -- not as a process of steps (that are easy to confuse with de-noising steps).
-1-
Forward Diffusion is the process of making training examples by sampling an image, noise, and an amount of noise, and mixing them to create a training example.
-2-
Do this with lots of images and lots of noise samples & amounts, and there's a training dataset for your model -- the noise prediction Unet.
-3-
Your training steps for the Unet then follow the familiar supervised learning recipe:
1- Make prediction
2- Compare to label, calculate loss
3- Update model so it does better the next time
-4-
The post then goes on discussing how text prompts are added to the picture.
One prediction in the earlier draft has already happened (swapping CLIP for OpenCLIP).
-5-
Major thanks to @@jh@sigmoid.social (Mastodon) and @Hamel Husain for incredible feedback. Be sure to check the fast.ai lectures and course which go a lot more in-depth.
fast.ai/posts/part2-20
Jay Alammar
@JayAlammar
Machine learning R&D. Builder. Writer. Visualizing artificial intelligence & machine learning one concept at a time. @CohereAI. https://t.co/UOtuwBj289
Missing some tweets in this thread? Or failed to load images or videos? You can try to .