Qwen1.5-14B-Chat在Langchain-Chatchat启动，长时间停止在Loading checkpoint shards: 75%

[KevinFanng]

问题描述 / Problem Description
Qwen1.5-14B-Chat在Langchain-Chatchat启动，长时间停止在Loading checkpoint shards: 75%

操作系统：Linux-4.19.24-7.34.cbp.al7.x86_64-x86_64-with-glibc2.35.
python版本：3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0]
项目版本：v0.2.10
langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['Qwen1.5-14B-Chat'] @ cuda
{'device': 'cuda',
'host': '0.0.0.0',
'infer_turbo': False,
'model_path': '/mnt/workspace/Qwen1.5-14B-Chat',
'model_path_exists': True,
'port': 20002}
当前Embbedings模型： bge-large-zh-v1.5 @ cuda
==============================Langchain-Chatchat Configuration==============================

2024-05-19 10:40:47,918 - startup.py[line:655] - INFO: 正在启动服务：
2024-05-19 10:40:47,919 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 /mnt/workspace/Langchain-Chatchat/logs
/opt/conda/envs/lc/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃
warn_deprecated(
2024-05-19 10:40:53 | ERROR | stderr | INFO: Started server process [17244]
2024-05-19 10:40:53 | ERROR | stderr | INFO: Waiting for application startup.
2024-05-19 10:40:53 | ERROR | stderr | INFO: Application startup complete.
2024-05-19 10:40:53 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit)
2024-05-19 10:40:54 | INFO | model_worker | Loading the model ['Qwen1.5-14B-Chat'] on worker 90de9bfd ...
Loading checkpoint shards: 0%| | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards: 12%|████████████▋ | 1/8 [00:21<02:28, 21.18s/it]
Loading checkpoint shards: 25%|█████████████████████████▎ | 2/8 [00:43<02:11, 21.84s/it]
Loading checkpoint shards: 38%|█████████████████████████████████████▉ | 3/8 [01:05<01:49, 21.96s/it]
Loading checkpoint shards: 50%|██████████████████████████████████████████████████▌ | 4/8 [01:27<01:27, 21.99s/it]
Loading checkpoint shards: 62%|███████████████████████████████████████████████████████████████▏ | 5/8 [01:49<01:05, 22.00s/it]
Loading checkpoint shards: 75%|███████████████████████████████████████████████████████████████████████████▊ | 6/8 [02:12<00:44, 22.18s/it]

[HongyuJiang]

会不会是你的机器内存不足了，所以载入不完

[KevinFanng]

资源充足，24G显卡，启动的时候都没怎么用显存。

[HongyuJiang]

内存，哥~

[KevinFanng]

内存，哥~

感谢兄弟，其实也不是内存，是Qwen1.5-14B-Chat需要24G以上的显存。