关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
zeizei
/
OpenI_Learning
Not watched
58
203
Fork 205
Code
Releases 0 Wiki Activity
Issues 291 Pull Requests 0 Datasets Model Cloudbrain
Issues
/
Detail
Labels Milestones
New Issue

#1208 git拉取huggingface权重失败

Open
created 4 months ago by CWY · 5 comments
CWY commented 4 months ago

问题描述

fatal: unable to access 'https://huggingface.co/lmsys/vicuna-7b-v1.1/': Failed to connect to huggingface.co port 443 after 130844 ms: Connection timed out

相关环境（GPU/NPU）

A100 40G

相关集群（启智/智算）

任务类型（调试/训练/推理）

调试

任务名

日志说明或问题截图

期望的解决方案或建议

<!-- 为了更有效地识别与解决您的问题，请尽可能的补充如下信息 --> ### 问题描述 fatal: unable to access 'https://huggingface.co/lmsys/vicuna-7b-v1.1/': Failed to connect to huggingface.co port 443 after 130844 ms: Connection timed out ### 相关环境（GPU/NPU） A100 40G ### 相关集群（启智/智算） ### 任务类型（调试/训练/推理） 调试 ### 任务名 ### 日志说明或问题截图 ### 期望的解决方案或建议
thomas-yanxin commented 3 months ago
用https://hf-mirror.com/ 下载
👍 3
eanson commented 2 months ago

用https://hf-mirror.com/ 下载

一样不行，比如我下载roberta，会在调试的时候报错：

# git lfs install
# git clone https://hf-mirror.com/FacebookAI/roberta-base
Cloning into 'roberta-base'...
remote: Enumerating objects: 81, done.
remote: Total 81 (delta 0), reused 0 (delta 0), pack-reused 81
Unpacking objects: 100% (81/81), done.
Downloading flax_model.msgpack (499 MB)
Error downloading object: flax_model.msgpack (7647e7d): Smudge error: Error downloading flax_model.msgpack (7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3): LFS: Get https://cdn-lfs.huggingface.co/roberta-base/7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3?Expires=1707730156&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcwNzczMDE1Nn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yb2JlcnRhLWJhc2UvNzY0N2U3ZGYxNWRmZGRiN2Q1YzA3ZmNiZWQ3YTIxZTM4ODFjY2ZlYTA4NTc5MWNmMDk5MTdhYTA1OTQ3YmJhMyJ9XX0_&Signature=LsQIQvoxcN0gwwQ21fDEYQX6tP7FtGNsegX9vh8F1tktbFQsJ-qrkmlfE7eNV9yso5Bo4Z7EbvgL5AY8yAcfWdITGvKDVHZ-Jt16YBSu15VgTP1QgaTEo0r5MUnL1uu2IkcWSDoSUIClRH6OlPjAJE4RQnrW6gqNFHDElbfwExgyVE8Nf-yZR9jWhmMnPK38Hu694pE2SUxymxO9CVlOdk4v%7EtXQzk8OchJVw1nU-Jq%7E13sV8grFF9pliuaqrxrj25bHqzrYpbtnXKoQLdEgIiAYS1iBbM8KogF-Ns104rwHqeeLUTyJA0cHNU%7EOBZk2mVpK4pecEbDSUWwoKCVyUA__&Key-Pair-Id=KVTP0A1DKRTAX: read tcp 10.244.75.18:58110->13.224.167.110:443: read: connection reset by peer
> 用https://hf-mirror.com/ 下载 一样不行，比如我下载roberta，会在调试的时候报错： ``` # git lfs install # git clone https://hf-mirror.com/FacebookAI/roberta-base Cloning into 'roberta-base'... remote: Enumerating objects: 81, done. remote: Total 81 (delta 0), reused 0 (delta 0), pack-reused 81 Unpacking objects: 100% (81/81), done. Downloading flax_model.msgpack (499 MB) Error downloading object: flax_model.msgpack (7647e7d): Smudge error: Error downloading flax_model.msgpack (7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3): LFS: Get https://cdn-lfs.huggingface.co/roberta-base/7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3?Expires=1707730156&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcwNzczMDE1Nn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yb2JlcnRhLWJhc2UvNzY0N2U3ZGYxNWRmZGRiN2Q1YzA3ZmNiZWQ3YTIxZTM4ODFjY2ZlYTA4NTc5MWNmMDk5MTdhYTA1OTQ3YmJhMyJ9XX0_&Signature=LsQIQvoxcN0gwwQ21fDEYQX6tP7FtGNsegX9vh8F1tktbFQsJ-qrkmlfE7eNV9yso5Bo4Z7EbvgL5AY8yAcfWdITGvKDVHZ-Jt16YBSu15VgTP1QgaTEo0r5MUnL1uu2IkcWSDoSUIClRH6OlPjAJE4RQnrW6gqNFHDElbfwExgyVE8Nf-yZR9jWhmMnPK38Hu694pE2SUxymxO9CVlOdk4v%7EtXQzk8OchJVw1nU-Jq%7E13sV8grFF9pliuaqrxrj25bHqzrYpbtnXKoQLdEgIiAYS1iBbM8KogF-Ns104rwHqeeLUTyJA0cHNU%7EOBZk2mVpK4pecEbDSUWwoKCVyUA__&Key-Pair-Id=KVTP0A1DKRTAX: read tcp 10.244.75.18:58110->13.224.167.110:443: read: connection reset by peer ```
thomas-yanxin commented 2 months ago

用https://hf-mirror.com/ 下载

一样不行，比如我下载roberta，会在调试的时候报错：

# git lfs install
# git clone https://hf-mirror.com/FacebookAI/roberta-base
Cloning into 'roberta-base'...
remote: Enumerating objects: 81, done.
remote: Total 81 (delta 0), reused 0 (delta 0), pack-reused 81
Unpacking objects: 100% (81/81), done.
Downloading flax_model.msgpack (499 MB)
Error downloading object: flax_model.msgpack (7647e7d): Smudge error: Error downloading flax_model.msgpack (7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3): LFS: Get https://cdn-lfs.huggingface.co/roberta-base/7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3?Expires=1707730156&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcwNzczMDE1Nn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yb2JlcnRhLWJhc2UvNzY0N2U3ZGYxNWRmZGRiN2Q1YzA3ZmNiZWQ3YTIxZTM4ODFjY2ZlYTA4NTc5MWNmMDk5MTdhYTA1OTQ3YmJhMyJ9XX0_&Signature=LsQIQvoxcN0gwwQ21fDEYQX6tP7FtGNsegX9vh8F1tktbFQsJ-qrkmlfE7eNV9yso5Bo4Z7EbvgL5AY8yAcfWdITGvKDVHZ-Jt16YBSu15VgTP1QgaTEo0r5MUnL1uu2IkcWSDoSUIClRH6OlPjAJE4RQnrW6gqNFHDElbfwExgyVE8Nf-yZR9jWhmMnPK38Hu694pE2SUxymxO9CVlOdk4v%7EtXQzk8OchJVw1nU-Jq%7E13sV8grFF9pliuaqrxrj25bHqzrYpbtnXKoQLdEgIiAYS1iBbM8KogF-Ns104rwHqeeLUTyJA0cHNU%7EOBZk2mVpK4pecEbDSUWwoKCVyUA__&Key-Pair-Id=KVTP0A1DKRTAX: read tcp 10.244.75.18:58110->13.224.167.110:443: read: connection reset by peer

两种方案：

  1. 自己从HF上拉到本地，然后本地传到OpenI上；
  2. 如果modelscope没有相关模型，可以先用colab中转，利用colab将模型从HF传到ModelScope，然后你再从ModelScope拉。
    期待：
  3. 希望OpenI官方能出HF同步工具~
> > 用https://hf-mirror.com/ 下载 > > 一样不行，比如我下载roberta，会在调试的时候报错： > > ``` > # git lfs install > # git clone https://hf-mirror.com/FacebookAI/roberta-base > Cloning into 'roberta-base'... > remote: Enumerating objects: 81, done. > remote: Total 81 (delta 0), reused 0 (delta 0), pack-reused 81 > Unpacking objects: 100% (81/81), done. > Downloading flax_model.msgpack (499 MB) > Error downloading object: flax_model.msgpack (7647e7d): Smudge error: Error downloading flax_model.msgpack (7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3): LFS: Get https://cdn-lfs.huggingface.co/roberta-base/7647e7df15dfddb7d5c07fcbed7a21e3881ccfea085791cf09917aa05947bba3?Expires=1707730156&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcwNzczMDE1Nn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yb2JlcnRhLWJhc2UvNzY0N2U3ZGYxNWRmZGRiN2Q1YzA3ZmNiZWQ3YTIxZTM4ODFjY2ZlYTA4NTc5MWNmMDk5MTdhYTA1OTQ3YmJhMyJ9XX0_&Signature=LsQIQvoxcN0gwwQ21fDEYQX6tP7FtGNsegX9vh8F1tktbFQsJ-qrkmlfE7eNV9yso5Bo4Z7EbvgL5AY8yAcfWdITGvKDVHZ-Jt16YBSu15VgTP1QgaTEo0r5MUnL1uu2IkcWSDoSUIClRH6OlPjAJE4RQnrW6gqNFHDElbfwExgyVE8Nf-yZR9jWhmMnPK38Hu694pE2SUxymxO9CVlOdk4v%7EtXQzk8OchJVw1nU-Jq%7E13sV8grFF9pliuaqrxrj25bHqzrYpbtnXKoQLdEgIiAYS1iBbM8KogF-Ns104rwHqeeLUTyJA0cHNU%7EOBZk2mVpK4pecEbDSUWwoKCVyUA__&Key-Pair-Id=KVTP0A1DKRTAX: read tcp 10.244.75.18:58110->13.224.167.110:443: read: connection reset by peer > ``` 两种方案： 1. 自己从HF上拉到本地，然后本地传到OpenI上； 2. 如果modelscope没有相关模型，可以先用colab中转，利用colab将模型从HF传到ModelScope，然后你再从ModelScope拉。 期待： 3. 希望OpenI官方能出HF同步工具~
eanson commented 2 months ago

第一种方案不是很明白，本地传到openl怎么传呀？如果传好了，我新建训练任务怎么指定呢？

我把外部模型作为数据集丢进项目里试试

第一种方案不是很明白，本地传到openl怎么传呀？如果传好了，我新建训练任务怎么指定呢？ 我把外部模型作为数据集丢进项目里试试
thomas-yanxin commented 2 months ago

第一种方案不是很明白，本地传到openl怎么传呀？如果传好了，我新建训练任务怎么指定呢？

我把外部模型作为数据集丢进项目里试试

模型拉到本地，打个包，传到数据集，（新建任务时可以不指定），然后通过数据集的下载link在环境里wget，解压。

> 第一种方案不是很明白，本地传到openl怎么传呀？如果传好了，我新建训练任务怎么指定呢？ > > 我把外部模型作为数据集丢进项目里试试 模型拉到本地，打个包，传到数据集，（新建任务时可以不指定），然后通过数据集的下载link在环境里wget，解压。
Sign in to join this conversation.
No Branch/Tag Specified
Clear branch or tag
Branches Tags
Labels
Apply labels
Clear labels
bug
Something is not working dataset
数据集已通过 duplicate
This issue or pull request already exists enhancement
New feature help wanted
Need some help invalid
Something is wrong need review question
More information is needed test wontfix
This won't be fixed 大神经验分享
被平台采纳的用户分享经验贴 开发中
正在开发进行中 待完善
项目待完善后可加分 待确认 评审中 趣开源
精选项目标签 面向社区开发者
No Label
bug
dataset
duplicate
enhancement
help wanted
invalid
need review
question
test
wontfix
大神经验分享
开发中
待完善
待确认
评审中
趣开源
面向社区开发者
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
3 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes
Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit