关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
avadesian
/
Vicuna
mirror of https://github.com/lm-sys/FastChat
Not watched
1
0
Fork 0
Code
Releases 23 Wiki Activity
Issues 0 Datasets Model Cloudbrain
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
830 Commits
14 Branches
138 MiB
159 Download
 
 
 
 
 
Branch: operation
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'operation'
${ noResults }
Vicuna/tests
Christopher Chou 98b8f64262
Gradio Web Server for Multimodal Models (#2960) 		3 months ago
..
README.md [Logprobs] Support logprobs=1 (#2612) 6 months ago
killall_python.sh kill only fastchat process (#2641) 6 months ago
launch_openai_api_test_server.py Gradio Web Server for Multimodal Models (#2960) 3 months ago
test_cli.py Update README.md (vicuna-v1.3 -> vicuna-1.5) (#2592) 6 months ago
test_cli_inputs.txt Fix ChatGLM prompt template (#1786) 10 months ago
test_openai_api.py fix openai api server docs 4 months ago
test_openai_langchain.py Update README.md (vicuna-v1.3 -> vicuna-1.5) (#2592) 6 months ago
test_openai_vision_api.py Gradio Web Server for Multimodal Models (#2960) 3 months ago

README.md

Unit tests for FastChat

Test CLI Inference

python3 test_cli.py

Test OpenAI API Server

python3 launch_openai_api_test_server.py

python3 test_openai_api.py

Test GUI Serving

python3 -m fastchat.serve.controller

CUDA_VISIBLE_DEVICES=0,1 python3 -m fastchat.serve.model_worker --model-path ~/model_weights/koala-13b --num-gpus 2 --port 30000 --worker http://localhost:30000
CUDA_VISIBLE_DEVICES=2,3 python3 -m fastchat.serve.model_worker --model-path ~/model_weights/alpaca-13b --num-gpus 2 --port 30002 --worker http://localhost:30002
CUDA_VISIBLE_DEVICES=4,5 python3 -m fastchat.serve.model_worker --model-path ~/model_weights/vicuna-13b --port 30004 --worker http://localhost:30004 --num-gpus 2
CUDA_VISIBLE_DEVICES=6,7 python3 -m fastchat.serve.model_worker --model-path OpenAssistant/oasst-sft-1-pythia-12b --port 30006 --worker http://localhost:30006 --num-gpus 2

CUDA_VISIBLE_DEVICES=0,1 python3 -m fastchat.serve.model_worker --model-path StabilityAI/stablelm-tuned-alpha-7b --num-gpus 2 --port 30000 --worker http://localhost:30000
CUDA_VISIBLE_DEVICES=2,3 python3 -m fastchat.serve.model_worker --model-path databricks/dolly-v2-12b --num-gpus 2 --port 30002 --worker http://localhost:30002
CUDA_VISIBLE_DEVICES=4 python3 -m fastchat.serve.model_worker --model-path THUDM/chatglm-6b --port 30004 --worker http://localhost:30004
CUDA_VISIBLE_DEVICES=5 python3 -m fastchat.serve.model_worker --model-path lmsys/fastchat-t5-3b-v1.0 --port 30005 --worker http://localhost:30005
CUDA_VISIBLE_DEVICES=6 python3 -m fastchat.serve.model_worker --model-path ~/model_weights/baize-7b --port 30006 --worker http://localhost:30006
CUDA_VISIBLE_DEVICES=7 python3 -m fastchat.serve.model_worker --model-path ~/model_weights/RWKV-4-Raven-7B-v11x-Eng99%-Other1%-20230429-ctx8192.pth --port 30007 --worker http://localhost:30007

python3 -m fastchat.serve.gradio_web_server_multi

Test Peft Serving

python3 -m fastchat.serve.controller

PEFT_SHARE_BASE_WEIGHTS=true python3 -m fastchat.serve.multi_model_worker \
  --model-path SurfaceData/dummy_pythia160m_lora16_peft_chat \
  --model-path SurfaceData/dummy_pythia160m_lora8_peft_chat
Delete task

Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.

cancel
confirm

No Description

Python Jupyter Notebook Shell Nginx other

Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit