Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
Yimi81 36f9b8175d | 1 month ago | |
---|---|---|
.. | ||
README.md | 6 months ago | |
text_generation.py | 1 month ago | |
text_generation_tp.py | 6 months ago | |
web_demo.py | 3 months ago |
To run text generation task in the streaming mode:
python text_generation.py \
--model 01-ai/Yi-6B \
--tokenizer 01-ai/Yi-6B \
--max-tokens 512 \
--eos-token $'\n' \
--streaming
You can also provide an extra --prompt
argument to try some other prompts.
When dealing with extremely long input sequences, you may need multiple GPU devices and to enable tensor parallelism acceleration during inference to avoid insufficient memory error.
To run text generation task using tensor parallelism acceleration with 2 GPU devices:
torchrun --nproc_per_node 2 \
text_generation_tp.py \
--model 01-ai/Yi-6B \
--max-tokens 512 \
--eos-token $'\n' \
--streaming
No Description
Python Markdown Text Dockerfile Shell other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》