mindone/examples/pangu_draw_v3

• PanGu Draw 3.0
• • Features
  • What is New
  • Getting Started with PanGu Draw 3.0
  • • Installation
    • Pretrained Weights
    • Inference
  • Examples

中文|English

PanGu Draw 3.0

This folder contains PanGu Draw 3.0 models implemented with MindSpore.

Features

In contrast to version 2.0, Pangu Draw 3.0 has been subject to experimentation and updates across various aspects, including multi-language support, diverse resolutions, improved image quality, and model scaling. This includes:

• The current industry's largest 5-billion-parameter Chinese text-to-image model.
• Supports bilingual input in both Chinese and English.
• Supports output of native 1K resolution images.
• Outputs images in multiple size ratios.
• Quantifiable stylized adjustments: cartoon, aesthetic, photography controller.
• Based on Ascend+MindSpore for large-scale training and inference, using a self-developed MindSpore platform and Ascend 910 hardware.
• Utilizes self-developed RLAIF to enhance image quality and artistic expression.

What is New

Dec 12, 2023

Support inference of PanGu Draw 3.0 model for text-to-image generation.

Getting Started with PanGu Draw 3.0

Installation

Please make sure the following frameworks are installed.

• python >= 3.7
• mindspore >= 2.2.10 [install]

Install the dependent packages by running:

pip install -r requirements.txt

Pretrained Weights

The text-to-image task of the Pangu model requires pre-training parameters for both the low timestamp model and the high timestamp model (The pre-training model parameters is coming soon).

Version	MindSpore Checkpoint
Pangu3-low-timestamp-model	pangu_low_timestamp-127da122.ckpt
Pangu3-high-timestamp-model	pangu_high_timestamp-c6344411.ckpt

Inference

After obtaining the weights, start the demo using:

# run txt2img on Ascend
export MS_PYNATIVE_GE=1
python demo/pangu/pangu_sampling.py \
--device_target "Ascend" \
--ms_amp_level "O2" \
--config "configs/inference/pangu_sd_xl_base.yaml" \
--high_solution \
--weight "path/to/low_timestamp_model.ckpt" \
--high_timestamp_weight "path/to/high_timestamp_model.ckpt" \
--prompt "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"

Examples

Note: sampled 40 steps by PanGu Draw 3.0 on Ascend 910*.

Fig1: "一幅中国水墨画：一叶轻舟漂泊在波光粼粼的湖面上，舟上的人正在饮酒放歌"
Fig2: "坐在海边看海浪的少年，黄昏"
Fig3: "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"