langchain-chatchat开源项目，运行成功后，使用LLM（chatglm2-6b-int4）模型，不能正常对话

[zxu9704]

错误信息：ERROR: RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)

[dosubot[bot]]

To address the RemoteProtocolError and inability to converse using the LLM (chatglm2-6b-int4) model in the Langchain-Chatchat project, consider the following steps:

1. Install jinja2: Some users resolved similar issues by installing the jinja2 package. Use pip install jinja2 to add it to your environment.

2. Examine Application Logs: Detailed error messages or stack traces in the logs can pinpoint the source of the problem.

3. Adjust Uvicorn and FastAPI Configuration: If payload size limits are causing the issue, modifying Uvicorn and FastAPI settings to accept larger payloads might help.

4. Ensure Sufficient Memory: Verify that your Docker container has enough memory allocated to handle the processing requirements.

5. Update Langchain and Dependencies: Keeping Langchain and its dependencies up to date can prevent or resolve issues due to bugs in older versions.

If these steps do not resolve the issue, providing more detailed error messages, stack traces, and any relevant configurations can help in diagnosing the problem further.

Details

Sources

• 使用agent问答报错
• 报错：API通信遇到错误
• [BUG] 简洁阐述问题 / 我经常遇到API通信错误，这是什么原因导致的呢？
• webui界面输入内容后出现报错：RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)
• LLM对话没问题，知识库对话报错peer closed connection without sending complete message body (incomplete chunked read)
• [BUG] ERROR: RemoteProtocolError: API通信遇到错误：peer closed connection without sending complete message body (incomplete chunked read)
• [BUG] RemoteProtocolError: Caught exception: peer closed connection without sending
• server/api.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[zRzRzRzRzRzRzR]

没有做int4的适配，需要自己改

[zxu9704]

谢谢回复，提示以下信息，是int4没设配好吗？

==============================Langchain-Chatchat Configuration==============================
操作系统：Windows-10-10.0.19045-SP0.
python版本：3.11.0 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:21) [MSC v.1916 64 bit (AMD64)]
项目版本：v0.2.10
langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['chatglm2-6b-int4'] @ cpu
{'device': 'cpu',
'host': '127.0.0.1',
'infer_turbo': False,
'model_path': 'C:\Users\DELL\Downloads\chatglm2-6b-int4',
'model_path_exists': True,
'port': 20002}
当前Embbedings模型： m3e-base @ cpu

服务端运行信息：
OpenAI API Server: http://127.0.0.1:20000/v1
Chatchat API Server: http://127.0.0.1:7861
Chatchat WEBUI Server: http://127.0.0.1:8501
==============================Langchain-Chatchat Configuration==============================

You can now view your Streamlit app in your browser.

URL: http://127.0.0.1:8501

。。。。。
2024-05-16 14:04:18 | INFO | httpx | HTTP Request: POST http://127.0.0.1:20002/worker_generate_stream "HTTP/1.1 200 OK"
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "d:\Users\DELL\anaconda3\envs\langchain\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 407, in run_asgi
result = await app( # type: ignore[func-returns-value]
。。。。
File "C:\Users\DELL.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 228, in extract_weight_to_float
func(
TypeError: 'NoneType' object is not callable
2024-05-16 14:04:18 | ERROR | stderr | ERROR: Exception in ASGI application
2024-05-16 14:04:18 | ERROR | stderr | Traceback (most recent call last):
。。。。
httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
2024-05-16 14:04:18,608 - utils.py[line:40] - ERROR: RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)