sxela
patreon
Stable WarpFusion v0.28 - IP Adapters
🕑 Added 2023-12-01 11:53:55 +0000 UTCLocal install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv
If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D
This notebook needs an updated transformers 4.31.0 package. Make a new env with a fresh install.bat or update the package by running !pip install transformers==4.31.0 in a new code cell.
Changelog:
- add ipadapter support for sd15/sdxl modes from https://github.com/Mikubill/sd-webui-controlnet
- add ipadapter clip vit-h/-g preprocessors
- add ipadapter sd15/sdxl model urls to autodownloader
- add clip vit-h/-g model urls to autodownloader
- add ipadapters to controlnet gui
- add clip vision embedding cache for ipadapters
- add vae encoder cache
- remove unnecessary vae runs for inactive guidance functions
- removed unused gui options for ipadapters
- fix reappearing unused options for ipadapters, inpaint, and other controlnets that do not have preprocessors or resolution setting
- increase source textbox width for ipadapters
v0.28.2, 2.12.2023:
- add ipadapter for animatediff
- add v1 motion module support for animatediff
- fix some loras not working with animatediff thanks to #runshouse
v0.28.3, 3.12.2023:
- fix ldm.modules.encoders.modules error for colab thanks to #runshouse
v0.28.4, 5.12.2023:
- add safetensors support for aniimatediff motion modules
23.12.2023:
- move to L tier
12.1.2024, v0.28.5:
- fix dwpose annotator weights access denied error
18.3.2024:
- move to M tier
IP Adapters
Added numerous versions of IP Adapters for sd 1.5 and sdxl models.
They are available in the GUI -> Controlnet tab.
You should provide source image for them otherwise previously stylized frames will be used.
(GUI->Controlnet-> Source on the right side of a given adapter)
I haven't tested multiple adapters at the same time. For better and more consistent results I suggest picking source image with content similar to your video if you want to keep the content recognizable (i.e. a stylized human source if you want to keep human in the output)
You can provide different source images for each frame, if you specify a video of a folder with images as the source. Mind the file naming convention to have numbers with leading zeros to be sure you get source-frame pairs sorted correctly.
IP Adapter embeddings are cached, so if you only have 1 source image, it will be only preprocessed once per video.
IP Adapters and their CLIP Vision model counterparts are downloaded automatically and placed into your ControlNet folder.
Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md
Guides made by users:
- 05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ
- 11.05.2023, v0.11 How to use Stable Warp Fusion
- 13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration
- 14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking
- 23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install
- 15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)
- 17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation
- 21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion
- 21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial
- 04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)
- 15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!
- 25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI
- 2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.
- 3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide
- 10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)
- 4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue
- 13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog
YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq
For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7
Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.
Comments
Jackson Doran
Sure. On the discord now. Working again but prompt is now always resetting to "a beautifully detailed..." on both prompt settings no matter how many times I re enter and re run my prompts. Will try again tomorrow. Thank you.
Warpfusion
Hi, can you send me the error screenshot on discord?
Jackson Doran
Diffusion stopping after 1st frame is rendered on 28.4 π€ Getting a little chart after the error
Daniel Salvatierra
Sxela, plans on adding https://dreamoving.github.io/dreamoving/ as soon as it is released?
Warpfusion
ip adapters are available in comfyui
Haydn Mann
Are you making these available in ComfyUI?
Warpfusion
check the changelog in this post
Runs
Sorry, I mean the difference between say stable_warpfusion_v0_28_1_ip_adapter.ipynb and stable_warpfusion_v0_28_2_ip_adapter.ipynb
Warpfusion
the latest one that's available for your tier
Runs
Dumb question, what's the difference between the different numbered notebooks? I never know which one to download :)
