Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
nc438 f54e7eb469 | 1 year ago | |
---|---|---|
.. | ||
images | 1 year ago | |
src | 1 year ago | |
animeganv2.ipynb | 1 year ago | |
readme.md | 1 year ago |
Anime is a common artistic form in our daily life. This artistic form is widely used in several fields including advertising, film and children’s education.
Currently, the production of animation mainly relies on manual implementation. However, manually creating anime is very laborious and involves substantial artistic skills.
For animation artists, creating high-quality anime works requires careful consideration of lines, textures, colors and shadows, which means that it is difficult and time-consuming to create the works.
Therefore, the automatic techniques that can automatically transform real-world photos to high-quality animation style images are very valuable and necessary.
AnimeGANv2 algorithm can rapidly transform real-world photos into high-quality anime images. AnimeGANv2 is a lightweight generative adversarial model with fewer network parameters.
model | style | ckpt |
---|---|---|
animeganv2_generator_Hayao | Hayao | ckpt |
animeganv2_generator_Paprika | Paprika | ckpt |
animeganv2_generator_Shinkai | Shinkai | ckpt |
The pre-trained vgg19 model is used for feature extraction and loss function calculation. Please place this file in the same directory as this file.
Parameter | Default | Description |
---|---|---|
device_target | GPU | Device type |
device_id | 0 | Device ID |
dataset | Hayao | Dataset name |
data_dir | ../dataset | Path of training dataset |
checkpoint_dir | ../checkpoints | Path to save checkpoint |
vgg19_path | ../vgg.ckpt | Path of vgg19 |
save_image_dir | ../images | Path to save images |
resume | False | Whether to load pretrained model |
phase | train | Setup phase |
epochs | 40 | The number of epochs to run |
init_epochs | 5 | The number of epochs for weight initialization |
batch_size | 4 | The size of batch size |
num_parallel_workers | 1 | Number of parallel workers |
save_interval | 1 | Save the model during training |
debug_samples | 0 | Dataset name |
lr_g | 2.0e-4 | Generator learning rate |
lr_d | 4.0e-4 | Discriminator learning rate |
init_lr | 1.0e-3 | Initial learning rate |
gan_loss | lsgan | Network loss type |
wadvg | 1.2 | Adversarial loss weight for Generator |
wadvd | 300 | Adversarial loss weight for Discriminator |
wcon | 1.8 | Content loss weight |
wgra | 2.0 | Gram loss weight |
wcol | 10.0 | Color loss weight |
img_ch | 3 | The size of image channel |
ch | 64 | Base channel number per layer |
n_dis | 3 | The number of discriminator layer |
└──animeganv2
├──readme.md
├──animeganv2.ipynb
├──vgg.ckpt
├──src
├──animeganv2_utils
├──adjust_brightness.py # Adjust the brightness of output image.
├──edge_smooth.py # Smooth the animation image and save it in a new directory.
└──pre_process.py # Common image processing functions and tool functions.
├──losses
├──color_loss.py # Loss cell.
├──gram_loss.py # Loss cell.
├──loss.py # Generator loss and discriminator loss.
├──utils.py # Image processing functions.
└──vgg19.py # Build vgg19 network.
├──models
├──animegan.py # The connector of animegan network, loss and optimizer.
├──con2d_block.py # Convolution block.
├──discriminator.py # Discriminator.
├──generator.py # Generator network.
├──instance_norm_2d.py # Define up-sampling operation.
├──inverted_residual_block.py # Inverted residual block.
└──upsample.py # Define up-sampling operation.
├──process_datasets
├──animeganv2_dataset.py # Create the AnimeGAN dataset.
└──utils.py # Image processing functions.
├──infer.py # Test the performance in the specified directory.
├──train.py # Build and train model.
└──video2anime.py # Convert real world video into anime style video.
├──dataset
├──train_photo
├──test
├──val
├──Hayao
├──Shinkai
└──Paprika
├──images # Used to save intermediate training result images.
├──Hayao
├──Shinkai
└──Paprika
├──checkpoints # Used to save training model files.
├──Hayao
├──Shinkai
└──Paprika
Here, how to use AnimeGANv2 model will be introduced as following.
First of all, we recommend using the official dataset for training, which provides various anime styles such as Hayao, Shinkai, Paprika, and also provides real world images in train_photo
directory. The directory structure of the dataset is described as follows:
.dataset/
├── test # Contains multiple sizes of images for testing.
├── val # Validation set.
├── Hayao # Hayao style.
├──smooth # The smoothed animation image.
└──style # Original animation image.
├── Paprika # Paprika style.
├──smooth
└──style
├── Shinkai # Shinkai style.
├──smooth
└──style
├── SummerWar # SummerWar style.
├──smooth
└──style
├── train_photo # Real world images, when you change the style dataset, this dataset does not need to be changed.
├──0.jpg
├──1.jpg
......
└──2017-01-03 09_45_13.jpg
Before you start to train the model, if you use your own animation dataset, you should first smooth these images.
This step is not necessary if you use the official dataset downloaded above.
The directory structure of the input image:
your own style
├──smooth
└──style
Then run the edge_smooth.py
script,the two parameters of these commands represent the style directory path and the output directory path.
For example:
python edge_smooth.py --style_path dataset/Sakura/style/ --output_path dataset/Sakura/smooth/
Put the processed directory into the dataset directory:
.dataset/
├── your own style
├──smooth
└──style
......
After you have all the datasets ready, run the train.py to start to train the model.
python train.py --dataset Hayao --batch_size 4 --epochs 40
output:
1551it [05:28, 4.74it/s][40/40][1550/1664] Loss_D: 8.6518 Loss_G: 2.0143
1601it [05:38, 4.76it/s][40/40][1600/1664] Loss_D: 16.2060 Loss_G: 1.7243
1651it [05:49, 4.74it/s][40/40][1650/1664] Loss_D: 6.3708 Loss_G: 1.8706
When the training starts, the program automatically created images
and checkpoints
directories, using the former to save test results and the latter to save model files.
Subjective judgment can be made based on the test results in images
to select an appropriate model for the following model inferring and video conversion steps.
After training, you can use your own image to test your model. Select the model file you think is best from the checkpoints
directory and select the image folder that you want to test, then run infer.py
to do inference.
If you use a pre-trained model for inference, you can use the following command.
python infer.py --infer_dir ../dataset/test/real --infer_output ../dataset/output --ckpt_file_name ../checkpoints/Hayao/netG_28.ckpt
You can also convert landscape videos in MP4 format to anime style, but the sound of the video will not be retained.
python video2anime.py --video_input ../video/test.mp4 --video_output ../video/output.mp4 --video_ckpt_file_name ../checkpoints/Hayao/netG_28.ckpt
Demo Video Link: https://www.bilibili.com/video/BV1Nd4y1u7e8/
The following is the cartoonization result of a single landscape image of each style.
The following is the output of each style of video.
Original
MindSpore实验,仅用于教学或培训目的。配合MindSpore官网使用。 MindSpore experiments, for teaching or training purposes only. Use it together with the MindSpore official website.
CSV Jupyter Notebook Text Python Markdown other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》