sxela
patreon
Stable WarpFusion v0.34 - VidToMe, variable cfg, adiff, rec noise updates
🕑 Added 2024-05-29 05:22:48 +0000 UTCLocal install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv
If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D
Changelog:
fix predicted noise randomness not working in animatediff mode
turn off depth logs
fix controlnets and ipadapters in 'stylized' source mode sometimes using raw init frame during the 1st frame render
fix ip-adapter_sd15_light URL
fix animatediff mode only processing the last scene thanks to usergenic issue #105
fix animatediff not working with scenes smaller than the context size
fix mask extraction error in animatediff mode thanks to #thenightsfall1
add mask in animatediff mode (applied after the render)
enable rec noise in animatediff mode with batch length > context length
remove error trace when exporting video with no available audio
add ToMe 3d to animatediff mode
fix rec noise in animatediff mode with batch length > context length triggered on the long total length
fix rec noise in animatediff mode for multiple batches
disable guidance during noise reconstruction
add VidToMe
fix VidToMe not working with batch_length > context_length
add CFG scheduling during diffusion
add rec noise blending in animatediff mode with context_length < batch_length
add sliding context for rec noise in animatediff mode
add dev_settings for temporary settings
add dev_settings comparison to compare settings cell
add ffmpeg downloader to install.bat
add VidToME to GUI
add delayed_adapter_apply to apply ipadapters after controlnet preprocessing
0.34.1, 3.6.24:
import vidtome only when it's on
fix ipadapter not working on colab
28.7.2024
move to L tier
Animatediff mode updates
in animatediff mode: GUI -> animatediff -> rec_sliding_ctx
Added rec_sliding_ctx - sliding context for noise reconstruction in animatediff mode. You can now use batch size > context length with it. Keep on by default.
in animatediff mode: GUI -> animatediff -> stop_early_adiff
Added stop_early_adiff. You can use it to eject animatediff motion module after some diffusion steps and continue as img2img to add more detail at the cost of flickering. You can also use use_vidtome_with_adiff together with this setting to apply VidToMe after ejecting animatediff motion module
VidToMe
More about it here - VISION-SJTU/VidToMe: [CVPR 2024] VidToMe: Video Token Merging for Zero-Shot Video Editing (github.com)
in animatediff mode: GUI -> animatediff -> VidToMe group
use_vidtome - enable VidToMe
use_vidtome_with_adiff - used together with stop_early_adiff - enable to apply VidToMe after ejecting animatediff motion module. need to disable use_vidtome, or it will be applied during the whole diffusion process
vidtome_rand - randomness
vidtome_local_merge - % of tokens to merge per frame
vidtome_global_merge - % of tokens to merge across 4 frames
vidtome_max_down - max downscaling layers to be affected by VidToMe
1 means only merge tokens at full resolution layers, 2 - full+half, and so on.
vidtome_control - merge controlnet tokens
VidToMe (Video Token Merging) is used to merge attention tokens to smooth video and lower vram usage. It's still inferior to animatediff or traditional warp in terms of flickering, but can provide interesting ebsynth-like results when pushed to its limits. it can also be used with animatediff at the end of diffusion to add more detail at the cost of some flicker :D
Variable CFG Scale
You can now schedule cfg scale during the diffusion process. It will be automatically interpolated to your number of steps.
Formats:
[7] - 7 for the 0th frame and the rest
[7,5] - 7 for the 0th frame, 5 for the rest.
{0:7} - 7 for the 0th frame and the rest
{0:7, 1:5} - 7 for the 0th frame, 5 for the rest.
{0:7, 10:5} - 7 for 0-9 frames, 5 for frame 10 and the rest.
Instead of a single number value like 7, you can now use a list like [3,7,4] which will be interpolated used during the diffusion process like this: [4, 4.316, 4.632, 4.947, 5.263, 5.579, 5.895, 6.211, 6.526, 6.842, 6.789, 6.368, 5.947, 5.526, 5.105, 4.684, 4.263, 3.842, 3.421, 3.0] for 20 steps.
Value is bein taken from the end of the list, so the 1st step gets cfg scale = 3, the second - 3.421 and so on.
This allows to get stronger prompt adherence without oversaturation, especially with reconstructed noise.
Reconstructed Noise Updates
in animatediff mode: GUI -> animatediff -> rec_sliding_ctx
Added rec_sliding_ctx - sliding context for noise reconstruction in animatediff mode. You can now use batch size > context length with it. Keep on by default.
Guidance has also been disabled for rec noise, so you can use latent\image scale with it.
GUI -> contronet tab -> delayed_adapter_apply
Added delayed_adapter_apply, so that ipadapters are applied after rec noise is calculated. Before that most ipadapters were unusable with reconstructed noise. Keep on by default.
Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md
Guides made by users:
05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ
11.05.2023, v0.11 How to use Stable Warp Fusion
13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration
14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking
23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install
15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)
17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation
21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion
21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial
04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)
15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!
25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI
2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.
3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide
10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)
4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue
13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog
2.03.2024 v0.16 Create a INSANE Video Using AI Animation | Stable Warpfusion (youtube.com)
YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq
Setting on GitHub:
WarpFusion/examples at main · Sxela/WarpFusion (github.com)
For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7
Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.
Comments
Kytra@Midnight
Hell yeah! Sick update!
