Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
huolongshe 5d5ad87308 | 2 months ago | |
---|---|---|
app | 3 months ago | |
demo_data | 3 months ago | |
docs | 3 months ago | |
.gitignore | 3 months ago | |
Dockerfile | 3 months ago | |
LICENSE | 3 months ago | |
README.md | 2 months ago | |
application.yml | 3 months ago | |
build-docker.sh | 3 months ago | |
pack_model.py | 3 months ago | |
pip-install-reqs.sh | 3 months ago | |
requirements.txt | 3 months ago | |
run_model_server.py | 3 months ago |
This model is the pretrained version of Pix2Struct, use this model for fine-tuning purposes only.
Pix2Struct is an image encoder - text decoder model that is trained on image-text pairs for various tasks, including image captionning and visual question answering.
Visually-situated language is ubiquitous—sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms. Perhaps due to this diversity, previous work has typically relied on domainspecific recipes with limited sharing of the underlying data, model architectures, and objectives. We present Pix2Struct, a pretrained image-to-text model for purely visual language understanding, which can be finetuned on tasks containing visually-situated language. Pix2Struct is pretrained by learning to parse masked screenshots of web pages into simplified HTML. The web, with its richness of visual elements cleanly reflected in the HTML structure, provides a large source of pretraining data well suited to the diversity of downstream tasks. Intuitively, this objective subsumes common pretraining signals such as OCR, language modeling, image captioning. In addition to the novel pretraining strategy, we introduce a variable-resolution input representation and a more flexible integration of language and vision inputs, where language prompts such as questions are rendered directly on top of the input image. For the first time, we show that a single pretrained model can achieve state-of-the-art results in six out of nine tasks across four domains: documents, illustrations, user interfaces, and natural images.
模型来源: https://hf-mirror.com/google/pix2struct-large
本模型基于 ServiceBoot微服务引擎 进行服务化封装,参见: 《CubeAI模型开发指南》
$ sh pip-install-reqs.sh
$ serviceboot start
或
$ python3 run_model_server.py
一键式本地容器化部署和运行,参见: 《CubeAI模型独立部署指南》 或 CubeAI Docker Builder
本模型服务可一键发布至 CubeAI智立方平台 进行共享和部署,参见: 《CubeAI模型发布指南》
No Description
Python Shell Dockerfile Text
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》