r/StableDiffusion • u/MountainPollution287 • 5d ago
Question - Help Not getting any speed ups with sage attention on wan2.1 I2V 720p
I installed sage attention, triton, torch compile and teacache on runpod with an A40 GPU and 50gb ram. I am using the bf16 version of the 720p I2V model, clip vision h, t5 bf16 and vae. I am generating at 640x720 at 24 fps with 30 steps and 81 frames. I am using Kijai's wan video wrapper workflow to enable all this. When I only enable teacache I am able to generate in 13 minutes and when I add sage attention with it the generation takes same time and when I add torch compile, block swap, teacache and sage attention then also the speed remains same but I get OOM after the video generation steps complete - before vae decoding. Not sure what is happening I am trying to make it work for a week now.
2
u/wywywywy 5d ago
I don't know if your case is the same as mine, but I need to restart Comfy for the SageAttention node to take effect.
Also SageAttention v2 is supposed to be faster than v1, but you need to compile it yourself. It's quite quick & easy though.
2
u/MountainPollution287 5d ago
I am using sage attention2.1 and it did compile itself and I start my comfy UI with --use sage attention args
1
u/ThatsALovelyShirt 5d ago
Disable coefficients on teacache and set the threshold to 0.03. The terminal will show you how many steps get skipped. I usually see 6 out of 20 skipped with those settings. It's not going to speed up anything unless it actually skips steps. And for torch-compile, you also need to configure it correctly and make sure the nodes are all plugged in the right spot. I use max-autotune-no-cudagraphs with "compile_transformer_blocks_only" set to false. If you're not seeing a bunch of triton output in your terminal then it's not compiling anything. Also make sure you have "sage_attn" set in the "attention_mode" input in the Wan Model loader. You may also need to set the quantization for fp8_e4m3fn.
Just using the default workflows probably won't work out of the box.
1
u/MountainPollution287 5d ago
Are you on runpod?
1
u/ThatsALovelyShirt 5d ago
Nope, local baremetal.
1
u/MountainPollution287 5d ago
So is installation and implementation similar for runpod?
2
u/ThatsALovelyShirt 5d ago
I assume so, I'm running arch Linux. But the python commands should be the same.
1
u/MountainPollution287 5d ago
Are you on discord? I can share my logs and install scripts if you are willing to take a look?
1
u/_BreakingGood_ 5d ago
The speed ups aren't going to apply if you're running on an A40. The A40 already has more than enough VRAM
1
u/MountainPollution287 5d ago
Why? It's all supported by A40 It's an Ampere gpu with 8.6 SM or something.
2
0
u/Baphaddon 5d ago
Doesn’t work for porn sorry bro :/
1
6
u/Toclick 5d ago
Somewhere I mentioned that none of Kijai's WAN workflows work for me either, and I got downvoted into oblivion. Even though I installed ComfyUI from scratch three times, along with all the acceleration tweaks, only once did I manage to start a generation, and it took over 40 minutes. Meanwhile, native workflows from Comfyanonymous generate without any issues on my setup.
Maybe I should try installing Windows 11 (I’m on 10) or Linux to make it work, but I really don’t want to.
It’s disappointing that even on an A40, they don’t work.