Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
huolongshe 67443d463d | 2 months ago | |
---|---|---|
app | 11 months ago | |
docs | 1 year ago | |
webapp | 11 months ago | |
.gitignore | 1 year ago | |
Dockerfile | 1 year ago | |
LICENSE | 1 year ago | |
README.md | 2 months ago | |
application.yml | 11 months ago | |
build-docker.sh | 1 year ago | |
pack_model.py | 1 year ago | |
pip-install-reqs.sh | 9 months ago | |
requirements.txt | 9 months ago | |
run_model_server.py | 1 year ago |
本模型封装自魔搭(ModelScope)社区项目: ChatGLM-中英对话大模型-6B-Int4
https://www.modelscope.cn/models/ZhipuAI/ChatGLM-6B-Int4/summary
ChatGLM-6B 是一个开源的、支持中英双语问答的对话语言模型,基于 General Language Model (GLM) 架构,具有 62 亿参数。结合模型量化技术,用户可以在消费级的显卡上进行本地部署(INT4 量化级别下最低只需 6GB 显存)。ChatGLM-6B 使用了和 ChatGLM 相同的技术,针对中文问答和对话进行了优化。经过约 1T 标识符的中英双语训练,辅以监督微调、反馈自助、人类反馈强化学习等技术的加持,62 亿参数的 ChatGLM-6B 已经能生成相当符合人类偏好的回答。
ChatGLM-6B-INT4 是 ChatGLM-6B 量化后的模型权重。具体的,ChatGLM-6B-INT4 对 ChatGLM-6B 中的 28 个 GLM Block 进行了 INT4 量化,没有对 Embedding 和 LM Head 进行量化。量化后的模型理论上 6G 显存(使用 CPU 即内存)即可推理,具有在嵌入式设备(如树莓派)上运行的可能。
在 CPU 上运行时,会根据硬件自动编译 CPU Kernel ,请确保已安装 GCC 和 OpenMP (Linux一般已安装,对于Windows则需手动安装),以获得最佳并行计算能力。
本模型基于 ServiceBoot微服务引擎 进行服务化封装,参见: 《CubeAI模型开发指南》
$ sh pip-install-reqs.sh
$ serviceboot start
或
$ python3 run_model_server.py
一键式本地容器化部署和运行,参见: 《CubeAI模型独立部署指南》 或 CubeAI Docker Builder
本模型服务可一键发布至 CubeAI智立方平台 进行共享和部署,参见: 《CubeAI模型发布指南》
本模型提供了1个API接口:
API接口1:
API端点: /api/data
HTTP方法: POST
HTTP请求体:
{
"action": "predict"
"args": {
"input": {
"text": <提问>,
"history": [历史对话列表]
}
}
}
HTTP响应体:
{
"status": "ok"|"err",
"value": {
"response": <回答>,
"history": [历史对话列表]
}
}
No Description
TypeScript HTML Python Shell Text other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》