mindocr-copy/configs/rec/crnn

• CRNN
• • Introduction
  • Results
  • • • Notes
  • Quick Start
  • • Preparation
    • • Installation
      • Dataset Preparation
    • Training
    • Evaluation
  • References

English | 中文

CRNN

An End-to-End Trainable Neural Network for Image-based Sequence
Recognition and Its Application to Scene Text Recognition

Introduction

Convolutional Recurrent Neural Network (CRNN) integrates CNN feature extraction and RNN sequence modeling as well as transcription into a unified framework.

As shown in the architecture graph (Figure 1), CRNN firstly extracts a feature sequence from the input image via Convolutional Layers. After that, the image is represented by a squence extracted features, where each vector is associated with a receptive field on the input image. For futher process the feature, CRNN adopts Recurrent Layers to predict a label distribution for each frame. To map the distribution to text field, CRNN adds a Transcription Layer to translate the per-frame predictions into the final label sequence. [1]

Figure 1. Architecture of CRNN [1]

Results

According to our experiments, the evaluation results on public benchmark datasets (IC03, IC13, IC15, IIIT, SVT, SVTP, CUTE) is as follow:

Model	Backbone	Avg Accuracy	Recipe	Download
CRNN	VGG7	82.03	yaml	weights
CRNN	ResNet34	84.45	yaml	weights

Notes

• Both VGG and ResNet models are trained from scratch without any pre-training.
• The above models are trained with MJSynth (MJ) and SynthText (ST) datasets. For more data details, please refer to Data Preparation
• Evaluations are tested individually on each benchmark dataset, and Avg Accuracy is the average of accuracies across all sub-datasets.

Quick Start

Preparation

Installation

Please refer to the installation instruction in MindOCR.

Dataset Preparation

Please download lmdb dataset for traininig and evaluation from here (ref: deep-text-recognition-benchmark). There're several zip files:

• data_lmdb_release.zip contains the entire datasets including train, valid and evaluation.
• validation.zip is the union dataset for Validation
• evaluation.zip contains several benchmarking datasets.

Training

• Distributed Training

It is easy to reproduce the reported results with the pre-defined training recipe. For distributed training on multiple Ascend 910 devices, please modify the configuration parameter distribute as True and run

# distributed training on multiple GPU/Ascend devices
mpirun -n 8 python tools/train.py --config configs/rec/crnn/crnn_resnet34.yaml

If the script is executed by the root user, the --allow-run-as-root parameter must be added to mpirun.

Similarly, you can train the model on multiple GPU devices with the above mpirun command.

Note: As the global batch size (batch_size x num_devices) is an important hyper-parameter, it is recommended to keep the global batch size unchanged for reproduction or adjust the learning rate linearly to a new global batch size.

• Standalone Training

If you want to train or finetune the model on a smaller dataset without distributed training, please modify the configuration parameter distribute as False and run:

# standalone training on a CPU/GPU/Ascend device
python tools/train.py --config configs/rec/crnn/crnn_resnet34.yaml

Evaluation

To evaluate the accuracy of the trained model, you can use eval.py. Please add an additional configuration parameter ckpt_load_path in eval section and set it to the path of the model checkpoint and then run:

python tools/eval.py --config configs/rec/crnn/crnn_vgg7.yaml

References

[1] Baoguang Shi, Xiang Bai, Cong Yao. An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition. arXiv preprint arXiv:1507.05717, 2015.