-
Notifications
You must be signed in to change notification settings - Fork 25.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Performance 3/6] Disable nan check by default #15805
Conversation
can the nan check only enable for VAE? |
nan check is not great but disabling that has a lot of implications for example the VAE fallback will no longer work |
long term wise it may be desirable to load VAE as bfloat16 instead |
As it is now it will break automatically switching to full precision VAE. |
Can we maybe just get the needed performance improvement by checking a single element instead of the whole tensor? Since there are batch norms, single values becoming NaN dooms the whole tensor to become all NaNs. I pushed 547778b to dev with this change. |
Also what tool is being used here for those performance visualizations? I'd like that too. Edit: it's torch's profiler visualized in chrome https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html |
I think checking only a single element is a better way to handle this. Thanks for doing that! |
But I changed the nan checking to only happen once after all steps are done in 6214aa7, so this is not an issue. |
Description
According to lllyasviel/stable-diffusion-webui-forge#716 (comment) , nan check has ~20ms/it overhead. The overhead is large enough that option should only be used for debugging purpose.
Screenshots/videos:
Checklist: