Stable WarpFusion v0.29 - QoL fixes, wav2lip, FaceFusion installers
🕑 Added 2023-12-23 14:42:51 +0000 UTCMerry Christmas Incoming!
Local install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv
If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D
This notebook needs an updated transformers 4.31.0 package. Make a new env with a fresh install.bat or update the package by running !pip install transformers==4.31.0 in a new code cell.
Changelog:
add scheduling for controlnet\ipadapter sources
add mouth mask to consistency masked diffusion and inpaint controlnet mask
add fill_lips setting to dilate the mask
add fill_lips to gui/settings
rewrite PDF colormatching in pytorch (x20 faster)
add input frames support for video init
add input frames support for background mask source
add input frames support for RIFE, SAMTrack
add fast multiprompts for non-animatediff mode. visually different sometiems from the slow mode, doesn't work well with lcm lora
move fast multiprompt option to diffuse tab
fix uc_mask_shape error when using predicted noise
fix masks not working in None blend_mode video export
add FaceFusion install - https://github.com/facefusion/facefusion
add simple cli interface for FaceFusion (gradio is still available)
disable optical flow blend_mode for animatediff video export
add wav2lip installer & interface - https://github.com/snpranav/wav2lip-HD-dockerized?tab=readme-ov-file
add fill lips to saved settings
add flow maxsize
add flow resizing and rescaling to support flow maxsize
fix errors in animatediff/controlnet preview only modes
24.12.2023, v0.29.1:
add fill lips to saved settings
add flow maxsize
add flow resizing and rescaling to support flow maxsize
fix errors in animatediff/controlnet preview only modes
26.12.2023, v0.29.2:
fix unboundvariable cc error thanks to #yuraelkin
hide pdf colormatch debug output spam
9.01.2024, v0.29.3:
fix face controlnet error when no faces detected
upd link to linux installer author
12.1.2024, v0.29.4:
fix dwpose annotator weights access denied error
17.01.2024:
move to L tier
23.01.2024, v0.29.5:
fix face_cc error in non-controlnet model version thanks to #infinitevibes
v0.29.6, 30.01.2024:
fix pool not running error when exporting video with missing flow files
11.4.2024:
move to M
Flow Maxsize
Sets max size for flow maps (by the longest side, keeping the aspect ratio)
Can be useful when you render 1080p, given that the flow detection model was trained on 720p for example. Also useful if you're running OOM during flow stage.
Controlnet \ Ipadapter scheduling
You can now schedule Controlnet \ Ipadapter source images. For example, if you need to change ipadapter styles on a keyframe basis. Uses the same syntax as all json \ list schedules.
Face Controlnet consistency mask
gui -> controlnet - controlnet annotator settings -> fill_lips
When face controlnet is enabled, the mouth area is added to the consistency mask to help with lipsync. Openpose Controlnet is still working better with lipsync as a controlnet, but an extra mouth mask helps.
Enabled by default when face controlnet is on and fill_lips > 0
Add input frames support to init video cell
You can now use frames\folders with frames\glob patterns as inputs in video init cell, background mask source, RIFE, SAMTrack.
FaceFusion
Faceswaps based on a single face reference. Works best with faces 512x512 and smaller. Added installer. You can run a fast cli with minimum settings, or launch the FaceFusion Gradio UI.
Wav2Lip
Syncs lips to audio, comes with GFPGan enhancer. Works best with faces 256x256 and smaller.
Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md
Guides made by users:
05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ
11.05.2023, v0.11 How to use Stable Warp Fusion
13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration
14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking
23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install
15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)
17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation
21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion
21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial
04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)
15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!
25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI
2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.
3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide
10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)
4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue
13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog
YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq
For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7
Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.
Comments
Warpfusion
restart and run all
Kamil
Sorry not that one, but this one with cuda: RuntimeError Traceback (most recent call last) Cell In[58], line 865 863 thread_pool.join() 864 gc.collect() --> 865 torch.cuda.empty_cache() 866 executed_cells[cell_name] = True File D:\WarpFusion\WarpFusion\env\lib\site-packages\torch\cuda\memory.py:133, in empty_cache() 122 r"""Releases all unoccupied cached memory currently held by the caching 123 allocator so that those can be used in other GPU application and visible in 124 `nvidia-smi`. (...) 130 more details about GPU memory management. 131 """ 132 if is_initialized(): --> 133 torch._C._cuda_emptyCache() RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Kamil
Hi, I can work with the 1.5 model, but after switching to XL, there are errors. Can you try to help me with this?" DO THE RUN: NameError Traceback (most recent call last) Cell In[27], line 78 77 try: ---> 78 batchNum 79 except: NameError: name 'batchNum' is not defined During handling of the above exception, another exception occurred: ValueError Traceback (most recent call last) Cell In[27], line 82 80 renders = glob(f"{batchFolder}/*00.png") 81 renders_s = sorted([int(o.split('(')[-1].split(')')[0]) for o in renders]) ---> 82 batchNum = max(renders_s) 83 # batchNum = len(glob(f"{settings_out}/{batch_name}(*)_settings.txt"))-1 84 85 else: 86 batchNum = int(run_to_resume) ValueError: max() arg is an empty sequence
Jackson Doran
Woohoo!
Sergiy Dovgal
๐๐๐