项目需要Python>=3.8.1, 默认已安装torch
git clone https://github.com/thomas-yanxin/LangChain-ChatGLM-Webui.git
. 若国内用户访问Github存在网络问题, 也可以执行: https://openi.pcl.ac.cn/Learning-Develop-Union/LangChain-ChatGLM-Webui.git
cd LangChain-ChatGLM-Webui
pip install -r requirements.txt
, 国内用户可设置清华源加速下载.另: 若您想要安装测试ModelScope版本, 需要额外安装ModelScope包: pip install modelscope==1.4.3 -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
OpenI启智社区部署推荐的Docker镜像: dockerhub.pcl.ac.cn:5000/user-images/openi:LangChain-ChatLLM-2.0
在terminal中执行命令: python3 app.py
cd modelscope
python3 app.py
docker run -it --rm --runtime=nvidia --gpus all --network host registry.cn-beijing.aliyuncs.com/public-development-resources/langchain-chatglm-webui:Base bash
git clone https://github.com/thomas-yanxin/LangChain-ChatGLM-Webui.git
cd LangChain-ChatGLM-Webui
pip3 install -r requirements.txt
python3 app.py
运行镜像:docker run -d --name langchain-ChatGLM-webui --runtime=nvidia --gpus all --network host registry.cn-beijing.aliyuncs.com/public-development-resources/langchain-chatglm-webui:latest
访问服务:http://ip:7860
运行环境,镜像大小约14G。
nvidia-runtime 请参考: container-toolkit
本地模型放置目录:
BELLE-LLaMA-Local:/pretrainmodel/belle
Vicuna-Local:/pretrainmodel/vicuna
ChatGLM-Local:/pretrainmodel/chatglm
挂载cache目录,容器重启或更新无需重新下载相关模型。
-v langchain-ChatGLM-webui-cache:/root/.cache/
启动服务:lc-serve deploy local jina_serving
执行curl初始化模型命令
curl -X 'POST' \
'http://localhost:8080/reinit_model' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"large_language_model": "ChatGLM-6B-int8",
"embedding_model": "text2vec-base"
}'
curl -X 'POST' \
'http://localhost:8080/vector_store' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"file_path": "./README.md"
}'
curl -X 'POST' \
'http://localhost:8080/predict' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"input": "ChatGLM-6B的具体局限性?"
"use_web": true,
"top_k": 3,
"history_len": 1,
"temperature": 0.01,
"top_p": 0.1,
"history": []
}'
docker run -d --name LangChain-ChatGLM-Webui --runtime=nvidia --gpus all --network host registry.cn-beijing.aliyuncs.com/public-development-resources/langchain-chatglm-webui:api
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》