failed to apply concat_linear on unet

[nihilityer]

Describe the bug

When I use plugin ComfyUI-layerdiffuse for Comfyui on a Windows system, I get these error logs output in the terminal

C:\Users\***\.conda\envs\comfyui\lib\site-packages\intel_extension_for_pytorch\frontend.py:465: UserWarning: Conv BatchNorm folding failed during the optimize process.
  warnings.warn(
C:\Users\***\.conda\envs\comfyui\lib\site-packages\intel_extension_for_pytorch\frontend.py:472: UserWarning: Linear BatchNorm folding failed during the optimize process.
  warnings.warn(
C:\Users\***\.conda\envs\comfyui\lib\site-packages\intel_extension_for_pytorch\fx\concat_linear.py:190: UserWarning: failed to apply concat_linear on unet, please report bugs
  warnings.warn("failed to apply concat_linear on unet, please report bugs")

When I use LayerDiffusionDecode(RGBA) node, this error occurred:
Provided range is out of integer limits

Versions

C:\Users*.conda\envs\comfyui\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users*.conda\envs\comfyui\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
Collecting environment information...
PyTorch version: 2.1.0.post2+cxx11.abi
PyTorch CXX11 ABI: No
IPEX version: 2.1.30+xpu
IPEX commit: 474a6b3
Build type: Release

OS: Microsoft Windows 10 专业版
GCC version: N/A
Clang version: N/A
IGC version: 2024.1.0 (2024.1.0.20240308)
CMake version: version 3.29.0-rc2
Libc version: N/A

Python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:40:08) [MSC v.1938 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is XPU available: True
DPCPP runtime version: N/A
MKL version: N/A
GPU models and configuration:
[0] _DeviceProperties(name='Intel(R) Arc(TM) A770 Graphics', platform_name='Intel(R) Level-Zero', dev_type='gpu', driver_version='1.3.28044', has_fp64=0, total_memory=15930MB, max_compute_units=512, gpu_eu_count=512)
Intel OpenCL ICD version: N/A
Level Zero version: N/A

CPU:
Architecture=9
CurrentClockSpeed=2100
DeviceID=CPU0
Family=198
L2CacheSize=8192
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2100
Name=13th Gen Intel(R) Core(TM) i7-13700
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] intel-extension-for-pytorch==2.1.30+xpu
[pip3] numpy==1.26.4
[pip3] open-clip-torch==2.24.0
[pip3] torch==2.1.0.post2+cxx11.abi
[pip3] torchaudio==2.1.0.post2+cxx11.abi
[pip3] torchsde==0.2.6
[pip3] torchvision==0.16.0.post2+cxx11.abi
[conda] intel-extension-for-pytorch 2.1.30+xpu pypi_0 pypi
[conda] numpy 1.26.4 pypi_0 pypi
[conda] open-clip-torch 2.24.0 pypi_0 pypi
[conda] torch 2.1.0.post2+cxx11.abi pypi_0 pypi
[conda] torchaudio 2.1.0.post2+cxx11.abi pypi_0 pypi
[conda] torchsde 0.2.6 pypi_0 pypi
[conda] torchvision 0.16.0.post2+cxx11.abi pypi_0 pypi

[wangkl2]

Thanks for reporting this. I will try to reproduce the issue and get back to you later.

[wangkl2]

@nihilityer Could you please provide us the minimum code reproducer for both cases of the failed to apply concat_linear warning and the Provided range is out of integer limits error. Thanks.

[nihilityer]

@wangkl2 The ComfyUI-layerdiffuse plug-in uses the deprecated API in diffusers. I modified the code and it ran normally.
before:

from diffusers.models.unet_2d_blocks import UNetMidBlock2D, get_down_block, get_up_block

after;

from diffusers.models.unets.unet_2d_blocks import UNetMidBlock2D, get_down_block, get_up_block

Thank you for coming to help me solve the problem。