|
- [INFO]: current net device: eth0, ip: 172.28.2.136
- [INFO]: paddle job envs:
- POD_IP=job-924f4ea464deba39a957d67bd989e797-trainer-0.job-924f4ea464deba39a957d67bd989e797
- PADDLE_PORT=12345
- PADDLE_TRAINER_ID=0
- PADDLE_TRAINERS_NUM=1
- PADDLE_USE_CUDA=1
- NCCL_SOCKET_IFNAME=eth0
- PADDLE_IS_LOCAL=1
- OUTPUT_PATH=/root/paddlejob/workspace/output
- LOCAL_LOG_PATH=/root/paddlejob/workspace/log
- LOCAL_MOUNT_PATH=/mnt/code_20220413112157,/mnt/datasets_20220413112157
- JOB_ID=job-924f4ea464deba39a957d67bd989e797
- TRAINING_ROLE=TRAINER
- [INFO]: user command: python run.py
- [INFO]: start trainer
- ~/paddlejob/workspace/code /mnt
- nvcc: NVIDIA (R) Cuda compiler driver
- Copyright (c) 2005-2019 NVIDIA Corporation
- Built on Sun_Jul_28_19:07:16_PDT_2019
- Cuda compilation tools, release 10.1, V10.1.243
- Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple, https://pypi.tuna.tsinghua.edu.cn/simple
- Collecting filelock
- Downloading https://pypi.tuna.tsinghua.edu.cn/packages/cd/f1/ba7dee3de0e9d3b8634d6fbaa5d0d407a7da64620305d147298b683e5c36/filelock-3.6.0-py3-none-any.whl (10.0 kB)
- Installing collected packages: filelock
- Successfully installed filelock-3.6.0
- WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
- WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
- You should consider upgrading via the '/opt/_internal/cpython-3.7.0/bin/python -m pip install --upgrade pip' command.
- data
- DDRNet.zip
- run.py
- run.sh
- Archive: DDRNet.zip
- creating: configs/
- creating: configs/_base_/
- extracting: configs/_base_/ade20k.yml
- extracting: configs/_base_/autonue.yml
- extracting: configs/_base_/chase_db1.yml
- extracting: configs/_base_/cityscapes.yml
- extracting: configs/_base_/cityscapes_1024x1024.yml
- extracting: configs/_base_/cityscapes_769x769.yml
- extracting: configs/_base_/cityscapes_769x769_setr.yml
- extracting: configs/_base_/coco_stuff.yml
- extracting: configs/_base_/drive.yml
- extracting: configs/_base_/hrf.yml
- extracting: configs/_base_/pascal_context.yml
- extracting: configs/_base_/pascal_voc12.yml
- extracting: configs/_base_/pascal_voc12aug.yml
- extracting: configs/_base_/stare.yml
- creating: configs/ann/
- extracting: configs/ann/ann_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/ann/ann_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/ann/ann_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/ann/ann_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/ann/README.md
- creating: configs/attention_unet/
- extracting: configs/attention_unet/attention_unet_cityscapes_1024x512_80k.yml
- extracting: configs/attention_unet/README.md
- creating: configs/bisenet/
- extracting: configs/bisenet/bisenet_cityscapes_1024x1024_160k.yml
- extracting: configs/bisenet/README.md
- creating: configs/bisenetv1/
- extracting: configs/bisenetv1/bisenetv1_resnet18_os8_cityscapes_1024x512_160k.yml
- extracting: configs/bisenetv1/README.md
- creating: configs/danet/
- extracting: configs/danet/danet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/danet/danet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/danet/danet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/danet/README.md
- creating: configs/ddrnet/
- extracting: configs/ddrnet/ddrnet23_cityscapes_1024x1024_120k.yml
- extracting: configs/ddrnet/README.md
- creating: configs/decoupled_segnet/
- extracting: configs/decoupled_segnet/decoupledsegnet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/decoupled_segnet/decoupledsegnet_resnet50_os8_cityscapes_832x832_80k.yml
- extracting: configs/decoupled_segnet/README.md
- creating: configs/deeplabv3/
- extracting: configs/deeplabv3/deeplabv3_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/deeplabv3/deeplabv3_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/deeplabv3/deeplabv3_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/deeplabv3/deeplabv3_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/deeplabv3/README.md
- creating: configs/deeplabv3p/
- extracting: configs/deeplabv3p/deeplabv3p_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/deeplabv3p/deeplabv3p_resnet101_os8_cityscapes_769x769_80k.yml
- extracting: configs/deeplabv3p/deeplabv3p_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/deeplabv3p/deeplabv3p_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/deeplabv3p/deeplabv3p_resnet50_os8_cityscapes_1024x512_80k_rmiloss.yml
- extracting: configs/deeplabv3p/deeplabv3p_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/deeplabv3p/README.md
- creating: configs/dmnet/
- extracting: configs/dmnet/dmnet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/dmnet/README.md
- creating: configs/dnlnet/
- extracting: configs/dnlnet/dnlnet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/dnlnet/dnlnet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/dnlnet/dnlnet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/dnlnet/dnlnet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/dnlnet/README.md
- creating: configs/emanet/
- extracting: configs/emanet/emanet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/emanet/emanet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/emanet/emanet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/emanet/emanet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/emanet/README.md
- creating: configs/encnet/
- extracting: configs/encnet/encnet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/encnet/README.md
- creating: configs/enet/
- extracting: configs/enet/enet_cityscapes_1024x512_80k.yml
- extracting: configs/enet/README.md
- creating: configs/espnet/
- extracting: configs/espnet/espnet_cityscapes_1024x512_120k.yml
- extracting: configs/espnet/README.md
- creating: configs/espnetv1/
- extracting: configs/espnetv1/espnetv1_cityscapes_1024x512_120k.yml
- extracting: configs/espnetv1/README.md
- creating: configs/fastfcn/
- extracting: configs/fastfcn/fastfcn_resnet50_os8_ade20k_480x480_120k.yml
- extracting: configs/fastfcn/README.md
- creating: configs/fastscnn/
- extracting: configs/fastscnn/fastscnn_cityscapes_1024x1024_160k.yml
- extracting: configs/fastscnn/fastscnn_cityscapes_1024x1024_40k.yml
- extracting: configs/fastscnn/fastscnn_cityscapes_1024x1024_40k_SCL.yml
- extracting: configs/fastscnn/README.md
- creating: configs/fcn/
- extracting: configs/fcn/fcn_hrnetw18_cityscapes_1024x512_80k.yml
- extracting: configs/fcn/fcn_hrnetw18_cityscapes_1024x512_80k_bs4.yml
- extracting: configs/fcn/fcn_hrnetw18_cityscapes_1024x512_80k_bs4_SCL.yml
- extracting: configs/fcn/fcn_hrnetw18_pphumanseg14k.yml
- extracting: configs/fcn/fcn_hrnetw18_voc12aug_512x512_40k.yml
- extracting: configs/fcn/fcn_hrnetw48_cityscapes_1024x512_80k.yml
- extracting: configs/fcn/fcn_hrnetw48_voc12aug_512x512_40k.yml
- extracting: configs/fcn/README.md
- creating: configs/gcnet/
- extracting: configs/gcnet/gcnet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/gcnet/gcnet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/gcnet/gcnet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/gcnet/gcnet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/gcnet/README.md
- creating: configs/ginet/
- extracting: configs/ginet/ginet_resnet101_os8_ade20k_520x520_150k.yml
- extracting: configs/ginet/ginet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/ginet/ginet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/ginet/ginet_resnet50_os8_ade20k_520x520_150k.yml
- extracting: configs/ginet/ginet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/ginet/ginet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/ginet/README.md
- creating: configs/gscnn/
- extracting: configs/gscnn/gscnn_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/gscnn/README.md
- creating: configs/hardnet/
- extracting: configs/hardnet/hardnet_cityscapes_1024x1024_160k.yml
- extracting: configs/hardnet/README.md
- creating: configs/hrnet_w48_contrast/
- extracting: configs/hrnet_w48_contrast/HRNet_W48_contrast_cityscapes_1024x512_60k.yml
- extracting: configs/hrnet_w48_contrast/README.md
- creating: configs/isanet/
- extracting: configs/isanet/isanet_resnet101_os8_cityscapes_769x769_80k.yml
- extracting: configs/isanet/isanet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/isanet/isanet_resnet50_os8_cityscapes_769x769_80k.yml
- extracting: configs/isanet/isanet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/isanet/README.md
- creating: configs/ocrnet/
- extracting: configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml
- extracting: configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k_lovasz_softmax.yml
- extracting: configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k.yml
- extracting: configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k_lovasz_hinge.yml
- extracting: configs/ocrnet/ocrnet_hrnetw18_voc12aug_512x512_40k.yml
- extracting: configs/ocrnet/ocrnet_hrnetw48_cityscapes_1024x512_160k.yml
- extracting: configs/ocrnet/ocrnet_hrnetw48_cityscapes_1024x512_40k.yml
- extracting: configs/ocrnet/ocrnet_hrnetw48_cityscapes_1024x512_40k_SCL.yml
- extracting: configs/ocrnet/ocrnet_hrnetw48_voc12aug_512x512_40k.yml
- extracting: configs/ocrnet/README.md
- creating: configs/pfpn/
- extracting: configs/pfpn/pfpn_resnet101_os8_cityscapes_512x1024_40k.yml
- extracting: configs/pfpn/README.md
- creating: configs/pointrend/
- extracting: configs/pointrend/pointrend_resnet101_os8_cityscapes_1024×512_80k.yml
- extracting: configs/pointrend/pointrend_resnet101_os8_voc12aug_512×512_40k.yml
- extracting: configs/pointrend/pointrend_resnet50_os8_cityscapes_1024×512_80k.yml
- extracting: configs/pointrend/pointrend_resnet50_os8_voc12aug_512×512_40k.yml
- extracting: configs/pointrend/README.md
- creating: configs/portraitnet/
- extracting: configs/portraitnet/portraitnet_eg1800_224x224_46k.yml
- extracting: configs/portraitnet/portraitnet_supervisely_224x224_60k.yml
- extracting: configs/portraitnet/README.md
- creating: configs/pp_humanseg_lite/
- extracting: configs/pp_humanseg_lite/pp_humanseg_lite_export_398x224.yml
- extracting: configs/pp_humanseg_lite/pp_humanseg_lite_mini_supervisely.yml
- extracting: configs/pp_humanseg_lite/pphumanseg_lite.png
- extracting: configs/pp_humanseg_lite/README.md
- creating: configs/pp_liteseg/
- extracting: configs/pp_liteseg/pp_liteseg_stdc1_camvid_960x720_10k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc1_camvid_960x720_10k_for_test.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc1_cityscapes_1024x512_scale0.5_160k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc1_cityscapes_1024x512_scale0.75_160k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc1_cityscapes_1024x512_scale1.0_160k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc2_camvid_960x720_10k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc2_camvid_960x720_10k_for_test.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc2_cityscapes_1024x512_scale0.5_160k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc2_cityscapes_1024x512_scale0.75_160k.yml
- extracting: configs/pp_liteseg/pp_liteseg_stdc2_cityscapes_1024x512_scale1.0_160k.yml
- extracting: configs/pp_liteseg/README.md
- creating: configs/pspnet/
- extracting: configs/pspnet/pspnet_resnet101_os8_cityscapes_1024x512_80k.yml
- extracting: configs/pspnet/pspnet_resnet101_os8_voc12aug_512x512_40k.yml
- extracting: configs/pspnet/pspnet_resnet50_os8_cityscapes_1024x512_80k.yml
- extracting: configs/pspnet/pspnet_resnet50_os8_voc12aug_512x512_40k.yml
- extracting: configs/pspnet/README.md
- creating: configs/quick_start/
- extracting: configs/quick_start/bisenet_optic_disc_512x512_1k.yml
- extracting: configs/quick_start/deeplabv3p_resnet18_os8_optic_disc_512x512_1k_student.yml
- extracting: configs/quick_start/deeplabv3p_resnet50_os8_optic_disc_512x512_1k_teacher.yml
- extracting: configs/quick_start/pp_liteseg_optic_disc_512x512_1k.yml
- extracting: configs/README.md
- extracting: configs/README_cn.md
- creating: configs/segformer/
- extracting: configs/segformer/README.md
- extracting: configs/segformer/segformer_b0_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b0_cityscapes_1024x512_160k.yml
- extracting: configs/segformer/segformer_b1_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b1_cityscapes_1024x512_160k.yml
- extracting: configs/segformer/segformer_b2_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b2_cityscapes_1024x512_160k.yml
- extracting: configs/segformer/segformer_b3_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b3_cityscapes_1024x512_160k.yml
- extracting: configs/segformer/segformer_b4_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b4_cityscapes_1024x512_160k.yml
- extracting: configs/segformer/segformer_b5_cityscapes_1024x1024_160k.yml
- extracting: configs/segformer/segformer_b5_cityscapes_1024x512_160k.yml
- creating: configs/segmenter/
- extracting: configs/segmenter/README.md
- extracting: configs/segmenter/segmenter_vit_base_linear_ade20k_512x512_160k.yml
- extracting: configs/segmenter/segmenter_vit_base_mask_ade20k_512x512_160k.yml
- extracting: configs/segmenter/segmenter_vit_small_linear_ade20k_512x512_160k.yml
- extracting: configs/segmenter/segmenter_vit_small_mask_ade20k_512x512_160k.yml
- creating: configs/segnet/
- extracting: configs/segnet/README.md
- extracting: configs/segnet/segnet_cityscapes_1024x512_80k.yml
- creating: configs/setr/
- extracting: configs/setr/README.md
- extracting: configs/setr/setr_mla_large_cityscapes_769x769_40k.yml
- extracting: configs/setr/setr_naive_large_cityscapes_769x769_40k.yml
- extracting: configs/setr/setr_pup_large_cityscapes_769x769_40k.yml
- creating: configs/sfnet/
- extracting: configs/sfnet/README.md
- extracting: configs/sfnet/sfnet_resnet18_os8_cityscapes_1024x1024_80k.yml
- extracting: configs/sfnet/sfnet_resnet50_os8_cityscapes_1024x1024_80k.yml
- creating: configs/stdcseg/
- extracting: configs/stdcseg/README.md
- extracting: configs/stdcseg/stdc1_seg_cityscapes_1024x512_80k.yml
- extracting: configs/stdcseg/stdc1_seg_voc12aug_512x512_40k.yml
- extracting: configs/stdcseg/stdc2_seg_cityscapes_1024x512_80k.yml
- extracting: configs/stdcseg/stdc2_seg_voc12aug_512x512_40k.yml
- creating: configs/u2net/
- extracting: configs/u2net/README.md
- extracting: configs/u2net/u2net_cityscapes_1024x512_160k.yml
- extracting: configs/u2net/u2netp_cityscapes_1024x512_160k.yml
- creating: configs/unet/
- extracting: configs/unet/README.md
- extracting: configs/unet/unet_chasedb1_128x128_40k.yml
- extracting: configs/unet/unet_cityscapes_1024x512_160k.yml
- extracting: configs/unet/unet_drive_128x128_40k.yml
- extracting: configs/unet/unet_hrf_256x256_40k.yml
- extracting: configs/unet/unet_stare_128x128_40k.yml
- creating: configs/unet_3plus/
- extracting: configs/unet_3plus/README.md
- extracting: configs/unet_3plus/unet_3plus_cityscapes_1024x512_160k.yml
- creating: configs/unet_plusplus/
- extracting: configs/unet_plusplus/README.md
- extracting: configs/unet_plusplus/unet_plusplus_cityscapes_1024x512_160k.yml
- extracting: ddrnet23_imagenet.pdparams
- creating: deploy/
- creating: deploy/cpp/
- extracting: deploy/cpp/CMakeLists.txt
- extracting: deploy/cpp/README.md
- extracting: deploy/cpp/README_cn.md
- extracting: deploy/cpp/run_seg_cpu.sh
- extracting: deploy/cpp/run_seg_gpu.sh
- extracting: deploy/cpp/run_seg_gpu_trt.sh
- extracting: deploy/cpp/run_seg_gpu_trt_dynamic_shape.sh
- creating: deploy/cpp/src/
- extracting: deploy/cpp/src/test_seg.cc
- creating: deploy/lite/
- creating: deploy/lite/example/
- extracting: deploy/lite/example/human_1.png
- extracting: deploy/lite/example/human_2.png
- extracting: deploy/lite/example/human_3.png
- creating: deploy/lite/human_segmentation_demo/
- extracting: deploy/lite/human_segmentation_demo/.gitignore
- creating: deploy/lite/human_segmentation_demo/app/
- extracting: deploy/lite/human_segmentation_demo/app/.gitignore
- extracting: deploy/lite/human_segmentation_demo/app/build.gradle
- extracting: deploy/lite/human_segmentation_demo/app/gradlew
- extracting: deploy/lite/human_segmentation_demo/app/gradlew.bat
- extracting: deploy/lite/human_segmentation_demo/app/local.properties
- extracting: deploy/lite/human_segmentation_demo/app/proguard-rules.pro
- creating: deploy/lite/human_segmentation_demo/app/src/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/baidu/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/baidu/paddle/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/baidu/paddle/lite/
- creating: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/baidu/paddle/lite/demo/
- extracting: deploy/lite/human_segmentation_demo/app/src/androidTest/java/com/baidu/paddle/lite/demo/ExampleInstrumentedTest.java
- creating: deploy/lite/human_segmentation_demo/app/src/main/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/AndroidManifest.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/assets/
- creating: deploy/lite/human_segmentation_demo/app/src/main/assets/image_segmentation/
- creating: deploy/lite/human_segmentation_demo/app/src/main/assets/image_segmentation/images/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/assets/image_segmentation/images/human.jpg
- creating: deploy/lite/human_segmentation_demo/app/src/main/assets/image_segmentation/labels/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/assets/image_segmentation/labels/label_list
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/AppCompatPreferenceActivity.java
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/config/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/config/Config.java
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/MainActivity.java
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/Predictor.java
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/preprocess/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/preprocess/Preprocess.java
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/SettingsActivity.java
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/Utils.java
- creating: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/visual/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/java/com/baidu/paddle/lite/demo/segmentation/visual/Visualize.java
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/drawable-v24/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/drawable-v24/ic_launcher_foreground.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/drawable/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/drawable/ic_launcher_background.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/layout/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/layout/activity_main.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/menu/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/menu/menu_action_options.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-anydpi-v26/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-hdpi/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-hdpi/ic_launcher.png
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-hdpi/ic_launcher_round.png
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-mdpi/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-mdpi/ic_launcher.png
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-mdpi/ic_launcher_round.png
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xhdpi/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xhdpi/ic_launcher.png
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxhdpi/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxhdpi/ic_launcher.png
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxxhdpi/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/values/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/values/arrays.xml
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/values/colors.xml
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/values/strings.xml
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/values/styles.xml
- creating: deploy/lite/human_segmentation_demo/app/src/main/res/xml/
- extracting: deploy/lite/human_segmentation_demo/app/src/main/res/xml/settings.xml
- creating: deploy/lite/human_segmentation_demo/app/src/test/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/com/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/com/baidu/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/com/baidu/paddle/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/com/baidu/paddle/lite/
- creating: deploy/lite/human_segmentation_demo/app/src/test/java/com/baidu/paddle/lite/demo/
- extracting: deploy/lite/human_segmentation_demo/app/src/test/java/com/baidu/paddle/lite/demo/ExampleUnitTest.java
- extracting: deploy/lite/human_segmentation_demo/build.gradle
- extracting: deploy/lite/human_segmentation_demo/gradle.properties
- creating: deploy/lite/human_segmentation_demo/gradle/
- creating: deploy/lite/human_segmentation_demo/gradle/wrapper/
- extracting: deploy/lite/human_segmentation_demo/gradle/wrapper/gradle-wrapper.jar
- extracting: deploy/lite/human_segmentation_demo/gradle/wrapper/gradle-wrapper.properties
- extracting: deploy/lite/human_segmentation_demo/gradlew
- extracting: deploy/lite/human_segmentation_demo/gradlew.bat
- extracting: deploy/lite/human_segmentation_demo/settings.gradle
- extracting: deploy/lite/README.md
- creating: deploy/onnxruntime_cpp/
- extracting: deploy/onnxruntime_cpp/CMakeLists.txt
- extracting: deploy/onnxruntime_cpp/README.md
- creating: deploy/onnxruntime_cpp/src/
- extracting: deploy/onnxruntime_cpp/src/ort_session_handler.cpp
- extracting: deploy/onnxruntime_cpp/src/ort_session_handler.hpp
- extracting: deploy/onnxruntime_cpp/src/test_seg.cpp
- creating: deploy/python/
- extracting: deploy/python/collect_dynamic_shape.py
- extracting: deploy/python/infer.py
- extracting: deploy/python/infer_benchmark.py
- extracting: deploy/python/infer_dataset.py
- extracting: deploy/python/infer_onnx_trt.py
- extracting: deploy/python/README.md
- creating: deploy/serving/
- extracting: deploy/serving/README.md
- extracting: deploy/serving/test_serving.py
- creating: deploy/web/
- creating: deploy/web/example/
- creating: deploy/web/example/bg/
- extracting: deploy/web/example/bg/bg.jpg
- extracting: deploy/web/example/index.html
- extracting: deploy/web/example/index.ts
- extracting: deploy/web/example/package.json
- extracting: deploy/web/example/tsconfig.json
- extracting: deploy/web/example/webpack.config.js
- extracting: deploy/web/README.md
- creating: docs/
- extracting: docs/add_new_model.md
- extracting: docs/api_example.md
- extracting: docs/api_example_cn.md
- creating: docs/apis/
- extracting: docs/apis/backbones.md
- creating: docs/apis/backbones/
- extracting: docs/apis/backbones/backbones.md
- extracting: docs/apis/backbones/backbones_cn.md
- extracting: docs/apis/backbones/index.rst
- extracting: docs/apis/core.md
- creating: docs/apis/core/
- extracting: docs/apis/core/core.md
- extracting: docs/apis/core/core_cn.md
- extracting: docs/apis/core/index.rst
- extracting: docs/apis/cvlibs.md
- creating: docs/apis/cvlibs/
- extracting: docs/apis/cvlibs/cvlibs.md
- extracting: docs/apis/cvlibs/cvlibs_cn.md
- extracting: docs/apis/cvlibs/index.rst
- extracting: docs/apis/datasets.md
- creating: docs/apis/datasets/
- extracting: docs/apis/datasets/datasets.md
- extracting: docs/apis/datasets/datasets_cn.md
- extracting: docs/apis/index.rst
- extracting: docs/apis/losses.md
- creating: docs/apis/losses/
- extracting: docs/apis/losses/index.rst
- extracting: docs/apis/losses/losses.md
- extracting: docs/apis/losses/losses_cn.md
- extracting: docs/apis/models.md
- creating: docs/apis/models/
- extracting: docs/apis/models/index.rst
- extracting: docs/apis/models/models.md
- extracting: docs/apis/models/models_cn.md
- extracting: docs/apis/README.md
- extracting: docs/apis/README_CN.md
- extracting: docs/apis/transforms.md
- creating: docs/apis/transforms/
- extracting: docs/apis/transforms/index.rst
- extracting: docs/apis/transforms/transforms.md
- extracting: docs/apis/transforms/transforms_cn.md
- extracting: docs/conf.py
- creating: docs/data/
- creating: docs/data/custom/
- extracting: docs/data/custom/data_prepare.md
- extracting: docs/data/custom/data_prepare_cn.md
- extracting: docs/data/custom/index.rst
- creating: docs/data/image/
- extracting: docs/data/image/file_list.png
- extracting: docs/data/image/file_list2.png
- extracting: docs/data/image/image-1.png
- extracting: docs/data/image/image-10.jpg
- extracting: docs/data/image/image-11.png
- extracting: docs/data/image/image-2.png
- extracting: docs/data/image/image-3.png
- extracting: docs/data/image/image-4-1.png
- extracting: docs/data/image/image-4-2.png
- extracting: docs/data/image/image-5.png
- extracting: docs/data/image/image-6-2.png
- extracting: docs/data/image/image-6.png
- extracting: docs/data/image/image-7.png
- extracting: docs/data/image/ITK-SNAP.png
- extracting: docs/data/image/jingling-1.png
- extracting: docs/data/image/jingling-2.png
- extracting: docs/data/image/jingling-3.png
- extracting: docs/data/image/jingling-4.png
- extracting: docs/data/image/jingling-5.png
- extracting: docs/data/image/labelme_polygons.jpg
- extracting: docs/data/image/LabelMeing.png
- creating: docs/data/marker/
- extracting: docs/data/marker/index.rst
- extracting: docs/data/marker/LabelMe.md
- extracting: docs/data/marker/LabelMe_cn.md
- extracting: docs/data/marker/marker.md
- extracting: docs/data/marker/marker_cn.md
- extracting: docs/data/README.md
- creating: docs/data/transform/
- extracting: docs/data/transform/index.rst
- extracting: docs/data/transform/transform.md
- extracting: docs/data/transform/transform_cn.md
- extracting: docs/data_prepare.md
- extracting: docs/data_prepare_cn.md
- creating: docs/deployment/
- creating: docs/deployment/inference/
- extracting: docs/deployment/inference/cpp_inference.md
- extracting: docs/deployment/inference/cpp_inference_cn.md
- extracting: docs/deployment/inference/index.rst
- extracting: docs/deployment/inference/infer_benchmark.md
- extracting: docs/deployment/inference/infer_benchmark_cn.md
- extracting: docs/deployment/inference/inference.md
- extracting: docs/deployment/inference/inference_cn.md
- extracting: docs/deployment/inference/python_inference.md
- extracting: docs/deployment/inference/python_inference_cn.md
- creating: docs/deployment/lite/
- creating: docs/deployment/lite/example/
- extracting: docs/deployment/lite/example/human.png
- extracting: docs/deployment/lite/example/human_1.png
- extracting: docs/deployment/lite/example/human_2.png
- extracting: docs/deployment/lite/example/human_3.png
- extracting: docs/deployment/lite/index.rst
- extracting: docs/deployment/lite/lite.md
- extracting: docs/deployment/lite/lite_cn.md
- creating: docs/deployment/serving/
- extracting: docs/deployment/serving/index.rst
- extracting: docs/deployment/serving/serving.md
- extracting: docs/deployment/serving/serving_cn.md
- creating: docs/deployment/web/
- creating: docs/deployment/web/image/
- extracting: docs/deployment/web/image/figure1.png
- extracting: docs/deployment/web/index.rst
- extracting: docs/deployment/web/web.md
- extracting: docs/deployment/web/web_cn.md
- creating: docs/design/
- creating: docs/design/create/
- extracting: docs/design/create/add_new_model.md
- extracting: docs/design/create/add_new_model_cn.md
- extracting: docs/design/create/index.rst
- creating: docs/design/use/
- extracting: docs/design/use/index.rst
- extracting: docs/design/use/use.md
- extracting: docs/design/use/use_cn.md
- creating: docs/evaluation/
- creating: docs/evaluation/evaluate/
- extracting: docs/evaluation/evaluate/evaluate.md
- extracting: docs/evaluation/evaluate/evaluate_cn.md
- extracting: docs/evaluation/evaluate/index.rst
- creating: docs/faq/
- creating: docs/faq/faq/
- extracting: docs/faq/faq/faq.md
- extracting: docs/faq/faq/faq_cn.md
- creating: docs/faq/faq/faq_imgs/
- extracting: docs/faq/faq/faq_imgs/ann_config.png
- extracting: docs/faq/faq/index.rst
- creating: docs/images/
- extracting: docs/images/activate.png
- extracting: docs/images/anli.png
- extracting: docs/images/api_fig1.png
- extracting: docs/images/api_fig2.png
- extracting: docs/images/chat.png
- extracting: docs/images/cityscapes_predict_demo.png
- extracting: docs/images/deepglobe.png
- extracting: docs/images/eiseg_demo.gif
- extracting: docs/images/f1.png
- extracting: docs/images/f2.png
- extracting: docs/images/f3.png
- extracting: docs/images/feature.png
- extracting: docs/images/fig1.png
- extracting: docs/images/fig2.png
- extracting: docs/images/fig3.png
- extracting: docs/images/fig4.png
- extracting: docs/images/fig5.png
- extracting: docs/images/human.png
- extracting: docs/images/interactive.gif
- extracting: docs/images/Lovasz_Hinge_Evaluate_mIoU.png
- extracting: docs/images/Lovasz_Softmax_Evaluate_mIoU.png
- extracting: docs/images/love.png
- extracting: docs/images/model.png
- extracting: docs/images/optic_test_image.jpg
- extracting: docs/images/paddleseg_logo.png
- extracting: docs/images/QQ_chat.png
- extracting: docs/images/quick_start_predict.jpg
- extracting: docs/images/quick_start_vdl.jpg
- creating: docs/images/readme/
- extracting: docs/images/readme/二次元.gif
- extracting: docs/images/readme/人体解析.gif
- extracting: docs/images/readme/人像分割-0.gif
- extracting: docs/images/readme/人像分割.gif
- extracting: docs/images/seg_news_icon.png
- extracting: docs/images/teach.png
- extracting: docs/images/yinyong.png
- extracting: docs/index.rst
- extracting: docs/install.md
- extracting: docs/install_cn.md
- extracting: docs/loss_usage.md
- extracting: docs/make.bat
- extracting: docs/Makefile
- extracting: docs/model_export.md
- extracting: docs/model_export_cn.md
- extracting: docs/model_export_onnx.md
- extracting: docs/model_export_onnx_cn.md
- extracting: docs/model_zoo_overview.md
- extracting: docs/model_zoo_overview_cn.md
- creating: docs/models/
- extracting: docs/models/deeplabv3.md
- extracting: docs/models/deeplabv3_cn.md
- extracting: docs/models/fascnn.md
- extracting: docs/models/fascnn_cn.md
- creating: docs/models/images/
- extracting: docs/models/images/convolution.png
- extracting: docs/models/images/deeplabv3+.png
- extracting: docs/models/images/Fast-SCNN.png
- extracting: docs/models/images/OCRNet.png
- extracting: docs/models/images/UNet.png
- extracting: docs/models/index.rst
- extracting: docs/models/ocrnet.md
- extracting: docs/models/ocrnet_cn.md
- extracting: docs/models/unet.md
- extracting: docs/models/unet_cn.md
- creating: docs/module/
- creating: docs/module/data/
- extracting: docs/module/data/data.md
- extracting: docs/module/data/data_cn.md
- extracting: docs/module/data/index.rst
- creating: docs/module/images/
- creating: docs/module/images/annotation/
- extracting: docs/module/images/annotation/image-1.png
- extracting: docs/module/images/annotation/image-10.jpg
- extracting: docs/module/images/annotation/image-11.png
- extracting: docs/module/images/annotation/image-2.png
- extracting: docs/module/images/annotation/image-3.png
- extracting: docs/module/images/annotation/image-4-1.png
- extracting: docs/module/images/annotation/image-4-2.png
- extracting: docs/module/images/annotation/image-5.png
- extracting: docs/module/images/annotation/image-6-2.png
- extracting: docs/module/images/annotation/image-6.png
- extracting: docs/module/images/annotation/image-7.png
- extracting: docs/module/images/annotation/jingling-1.png
- extracting: docs/module/images/annotation/jingling-2.png
- extracting: docs/module/images/annotation/jingling-3.png
- extracting: docs/module/images/annotation/jingling-4.png
- extracting: docs/module/images/annotation/jingling-5.png
- extracting: docs/module/images/aug_method.png
- extracting: docs/module/images/cityscapes.png
- extracting: docs/module/images/cityscapes_predict_demo.png
- extracting: docs/module/images/cosine_decay_example.png
- extracting: docs/module/images/data_aug_example.png
- extracting: docs/module/images/data_aug_flip_mirror.png
- extracting: docs/module/images/data_aug_flow.png
- extracting: docs/module/images/deepglobe.png
- extracting: docs/module/images/deeplabv3p.png
- extracting: docs/module/images/dice.png
- extracting: docs/module/images/dice2.png
- extracting: docs/module/images/dice3.png
- extracting: docs/module/images/fast-scnn.png
- extracting: docs/module/images/file_list.png
- extracting: docs/module/images/file_list2.png
- extracting: docs/module/images/gn.png
- extracting: docs/module/images/hrnet.png
- extracting: docs/module/images/icnet.png
- extracting: docs/module/images/image-10.jpg
- extracting: docs/module/images/loss_comparison.png
- extracting: docs/module/images/lovasz-hinge-vis.png
- extracting: docs/module/images/lovasz-hinge.png
- extracting: docs/module/images/lovasz-softmax.png
- extracting: docs/module/images/Lovasz_Hinge_Evaluate_mIoU.png
- extracting: docs/module/images/Lovasz_Softmax_Evaluate_mIoU.png
- extracting: docs/module/images/optic_test_image.jpg
- extracting: docs/module/images/piecewise_decay_example.png
- extracting: docs/module/images/poly_decay_example.png
- extracting: docs/module/images/pspnet.png
- extracting: docs/module/images/pspnet2.png
- extracting: docs/module/images/qq_group2.png
- extracting: docs/module/images/quick_start_predict.jpg
- extracting: docs/module/images/quick_start_vdl.jpg
- extracting: docs/module/images/rangescale.png
- extracting: docs/module/images/seg_news_icon.png
- extracting: docs/module/images/softmax_loss.png
- extracting: docs/module/images/unet.png
- extracting: docs/module/images/usage_vis_demo.jpg
- extracting: docs/module/images/visualdl_image.png
- extracting: docs/module/images/visualdl_scalar.png
- extracting: docs/module/images/VOC2012.png
- extracting: docs/module/images/warmup_with_poly_decay_example.png
- extracting: docs/module/index.rst
- creating: docs/module/loss/
- extracting: docs/module/loss/BCELoss_cn.md
- extracting: docs/module/loss/BCELoss_en.md
- extracting: docs/module/loss/BootstrappedCrossEntropyLoss_cn.md
- extracting: docs/module/loss/BootstrappedCrossEntropyLoss_en.md
- extracting: docs/module/loss/CrossEntropyLoss_cn.md
- extracting: docs/module/loss/CrossEntropyLoss_en.md
- extracting: docs/module/loss/DiceLoss_cn.md
- extracting: docs/module/loss/DiceLoss_en.md
- extracting: docs/module/loss/DualTaskLoss_cn.md
- extracting: docs/module/loss/DualTaskLoss_en.md
- extracting: docs/module/loss/EdgeAttentionLoss_cn.md
- extracting: docs/module/loss/EdgeAttentionLoss_en.md
- extracting: docs/module/loss/index.rst
- extracting: docs/module/loss/L1Loss_cn.md
- extracting: docs/module/loss/L1Loss_en.md
- extracting: docs/module/loss/losses_cn.md
- extracting: docs/module/loss/losses_en.md
- extracting: docs/module/loss/lovasz_loss_cn.md
- extracting: docs/module/loss/lovasz_loss_en.md
- extracting: docs/module/loss/LovaszHingeLoss_cn.md
- extracting: docs/module/loss/LovaszHingeLoss_en.md
- extracting: docs/module/loss/LovaszSoftmaxLoss_cn.md
- extracting: docs/module/loss/LovaszSoftmaxLoss_en.md
- extracting: docs/module/loss/MixedLoss_cn.md
- extracting: docs/module/loss/MixedLoss_en.md
- extracting: docs/module/loss/MSELoss_cn.md
- extracting: docs/module/loss/MSELoss_en.md
- extracting: docs/module/loss/OhemCrossEntropyLoss_cn.md
- extracting: docs/module/loss/OhemCrossEntropyLoss_en.md
- extracting: docs/module/loss/OhemEdgeAttentionLoss_cn.md
- extracting: docs/module/loss/OhemEdgeAttentionLoss_en.md
- extracting: docs/module/loss/RelaxBoundaryLoss_cn.md
- extracting: docs/module/loss/RelaxBoundaryLoss_en.md
- extracting: docs/module/loss/SemanticConnectivityLoss_cn.md
- extracting: docs/module/loss/SemanticConnectivityLoss_en.md
- creating: docs/module/tricks/
- extracting: docs/module/tricks/index.rst
- extracting: docs/module/tricks/tricks.md
- extracting: docs/paddleseg.png
- creating: docs/pr/
- creating: docs/pr/images/
- extracting: docs/pr/images/001_fork.png
- extracting: docs/pr/images/002_clone.png
- extracting: docs/pr/images/003_precommit_pass.png
- extracting: docs/pr/images/004_create_pr.png
- creating: docs/pr/pr/
- extracting: docs/pr/pr/index.rst
- extracting: docs/pr/pr/pr.md
- extracting: docs/pr/pr/pr_cn.md
- extracting: docs/pr/pr/style.md
- extracting: docs/pr/pr/style_cn.md
- creating: docs/predict/
- creating: docs/predict/color_map/
- extracting: docs/predict/color_map/after_mapped.jpeg
- extracting: docs/predict/color_map/before_mapped.jpeg
- extracting: docs/predict/predict.md
- extracting: docs/predict/predict_cn.md
- extracting: docs/README.md
- extracting: docs/release_notes.md
- extracting: docs/release_notes_cn.md
- extracting: docs/requirements.txt
- creating: docs/slim/
- creating: docs/slim/distill/
- extracting: docs/slim/distill/distill.md
- extracting: docs/slim/distill/distill_cn.md
- extracting: docs/slim/distill/index.rst
- creating: docs/slim/prune/
- extracting: docs/slim/prune/index.rst
- extracting: docs/slim/prune/prune.md
- extracting: docs/slim/prune/prune_cn.md
- creating: docs/slim/quant/
- extracting: docs/slim/quant/index.rst
- extracting: docs/slim/quant/quant.md
- extracting: docs/slim/quant/quant_cn.md
- creating: docs/static/
- extracting: docs/static/static.md
- extracting: docs/static/static_cn.md
- creating: docs/train/
- extracting: docs/train/index.rst
- extracting: docs/train/train.md
- extracting: docs/train/train_cn.md
- extracting: docs/whole_process.md
- extracting: docs/whole_process_cn.md
- extracting: export.py
- creating: paddleseg/
- extracting: paddleseg/__init__.py
- creating: paddleseg/core/
- extracting: paddleseg/core/__init__.py
- extracting: paddleseg/core/infer.py
- extracting: paddleseg/core/predict.py
- extracting: paddleseg/core/train.py
- extracting: paddleseg/core/val.py
- creating: paddleseg/cvlibs/
- extracting: paddleseg/cvlibs/__init__.py
- extracting: paddleseg/cvlibs/callbacks.py
- extracting: paddleseg/cvlibs/config.py
- extracting: paddleseg/cvlibs/manager.py
- extracting: paddleseg/cvlibs/param_init.py
- creating: paddleseg/datasets/
- extracting: paddleseg/datasets/__init__.py
- extracting: paddleseg/datasets/ade.py
- extracting: paddleseg/datasets/chase_db1.py
- extracting: paddleseg/datasets/cityscapes.py
- extracting: paddleseg/datasets/cocostuff.py
- extracting: paddleseg/datasets/dataset.py
- extracting: paddleseg/datasets/drive.py
- extracting: paddleseg/datasets/eg1800.py
- extracting: paddleseg/datasets/hrf.py
- extracting: paddleseg/datasets/mini_deep_globe_road_extraction.py
- extracting: paddleseg/datasets/optic_disc_seg.py
- extracting: paddleseg/datasets/pascal_context.py
- extracting: paddleseg/datasets/pp_humanseg14k.py
- extracting: paddleseg/datasets/stare.py
- extracting: paddleseg/datasets/supervisely.py
- extracting: paddleseg/datasets/voc.py
- creating: paddleseg/models/
- extracting: paddleseg/models/__init__.py
- extracting: paddleseg/models/ann.py
- extracting: paddleseg/models/attention_unet.py
- creating: paddleseg/models/backbones/
- extracting: paddleseg/models/backbones/__init__.py
- extracting: paddleseg/models/backbones/hrnet.py
- extracting: paddleseg/models/backbones/mix_transformer.py
- extracting: paddleseg/models/backbones/mobilenetv2.py
- extracting: paddleseg/models/backbones/mobilenetv3.py
- extracting: paddleseg/models/backbones/resnet_vd.py
- extracting: paddleseg/models/backbones/stdcnet.py
- extracting: paddleseg/models/backbones/swin_transformer.py
- extracting: paddleseg/models/backbones/transformer_utils.py
- extracting: paddleseg/models/backbones/vision_transformer.py
- extracting: paddleseg/models/backbones/xception_deeplab.py
- extracting: paddleseg/models/bisenet.py
- extracting: paddleseg/models/bisenetv1.py
- extracting: paddleseg/models/danet.py
- extracting: paddleseg/models/ddrnet.py
- extracting: paddleseg/models/decoupled_segnet.py
- extracting: paddleseg/models/deeplab.py
- extracting: paddleseg/models/dmnet.py
- extracting: paddleseg/models/dnlnet.py
- extracting: paddleseg/models/emanet.py
- extracting: paddleseg/models/encnet.py
- extracting: paddleseg/models/enet.py
- extracting: paddleseg/models/espnet.py
- extracting: paddleseg/models/espnetv1.py
- extracting: paddleseg/models/fast_scnn.py
- extracting: paddleseg/models/fastfcn.py
- extracting: paddleseg/models/fcn.py
- extracting: paddleseg/models/gcnet.py
- extracting: paddleseg/models/ginet.py
- extracting: paddleseg/models/gscnn.py
- extracting: paddleseg/models/hardnet.py
- extracting: paddleseg/models/hrnet_contrast.py
- extracting: paddleseg/models/isanet.py
- creating: paddleseg/models/layers/
- extracting: paddleseg/models/layers/__init__.py
- extracting: paddleseg/models/layers/activation.py
- extracting: paddleseg/models/layers/attention.py
- extracting: paddleseg/models/layers/layer_libs.py
- extracting: paddleseg/models/layers/nonlocal2d.py
- extracting: paddleseg/models/layers/pyramid_pool.py
- extracting: paddleseg/models/layers/tensor_fusion.py
- extracting: paddleseg/models/layers/tensor_fusion_helper.py
- extracting: paddleseg/models/layers/wrap_functions.py
- creating: paddleseg/models/losses/
- extracting: paddleseg/models/losses/__init__.py
- extracting: paddleseg/models/losses/binary_cross_entropy_loss.py
- extracting: paddleseg/models/losses/bootstrapped_cross_entropy.py
- extracting: paddleseg/models/losses/cross_entropy_loss.py
- extracting: paddleseg/models/losses/decoupledsegnet_relax_boundary_loss.py
- extracting: paddleseg/models/losses/detail_aggregate_loss.py
- extracting: paddleseg/models/losses/dice_loss.py
- extracting: paddleseg/models/losses/edge_attention_loss.py
- extracting: paddleseg/models/losses/focal_loss.py
- extracting: paddleseg/models/losses/gscnn_dual_task_loss.py
- extracting: paddleseg/models/losses/kl_loss.py
- extracting: paddleseg/models/losses/l1_loss.py
- extracting: paddleseg/models/losses/lovasz_loss.py
- extracting: paddleseg/models/losses/mean_square_error_loss.py
- extracting: paddleseg/models/losses/mixed_loss.py
- extracting: paddleseg/models/losses/ohem_cross_entropy_loss.py
- extracting: paddleseg/models/losses/ohem_edge_attention_loss.py
- extracting: paddleseg/models/losses/pixel_contrast_cross_entropy_loss.py
- extracting: paddleseg/models/losses/point_cross_entropy_loss.py
- extracting: paddleseg/models/losses/rmi_loss.py
- extracting: paddleseg/models/losses/semantic_connectivity_loss.py
- extracting: paddleseg/models/losses/semantic_encode_cross_entropy_loss.py
- extracting: paddleseg/models/mla_transformer.py
- extracting: paddleseg/models/ocrnet.py
- extracting: paddleseg/models/pfpnnet.py
- extracting: paddleseg/models/pointrend.py
- extracting: paddleseg/models/portraitnet.py
- extracting: paddleseg/models/pp_liteseg.py
- extracting: paddleseg/models/pphumanseg_lite.py
- extracting: paddleseg/models/pspnet.py
- extracting: paddleseg/models/segformer.py
- extracting: paddleseg/models/segmenter.py
- extracting: paddleseg/models/segnet.py
- extracting: paddleseg/models/setr.py
- extracting: paddleseg/models/sfnet.py
- extracting: paddleseg/models/stdcseg.py
- extracting: paddleseg/models/u2net.py
- extracting: paddleseg/models/unet.py
- extracting: paddleseg/models/unet_3plus.py
- extracting: paddleseg/models/unet_plusplus.py
- creating: paddleseg/transforms/
- extracting: paddleseg/transforms/__init__.py
- extracting: paddleseg/transforms/functional.py
- extracting: paddleseg/transforms/transforms.py
- creating: paddleseg/utils/
- extracting: paddleseg/utils/__init__.py
- extracting: paddleseg/utils/config_check.py
- extracting: paddleseg/utils/download.py
- extracting: paddleseg/utils/ema.py
- creating: paddleseg/utils/env/
- extracting: paddleseg/utils/env/__init__.py
- extracting: paddleseg/utils/env/seg_env.py
- extracting: paddleseg/utils/env/sys_env.py
- extracting: paddleseg/utils/logger.py
- extracting: paddleseg/utils/metrics.py
- extracting: paddleseg/utils/op_flops_funs.py
- extracting: paddleseg/utils/progbar.py
- extracting: paddleseg/utils/timer.py
- extracting: paddleseg/utils/train_profiler.py
- extracting: paddleseg/utils/utils.py
- extracting: paddleseg/utils/visualize.py
- extracting: predict.py
- extracting: setup.py
- creating: test_tipc/
- extracting: test_tipc/benchmark_train.sh
- extracting: test_tipc/common_func.sh
- extracting: test_tipc/compare_results.py
- creating: test_tipc/configs/
- creating: test_tipc/configs/_base_/
- extracting: test_tipc/configs/_base_/ade20k.yml
- extracting: test_tipc/configs/_base_/autonue.yml
- extracting: test_tipc/configs/_base_/cityscapes.yml
- extracting: test_tipc/configs/_base_/cityscapes_1024x1024.yml
- extracting: test_tipc/configs/_base_/cityscapes_769x769.yml
- extracting: test_tipc/configs/_base_/cityscapes_769x769_setr.yml
- extracting: test_tipc/configs/_base_/coco_stuff.yml
- extracting: test_tipc/configs/_base_/pascal_context.yml
- extracting: test_tipc/configs/_base_/pascal_voc12.yml
- extracting: test_tipc/configs/_base_/pascal_voc12aug.yml
- creating: test_tipc/configs/bisenetv2/
- extracting: test_tipc/configs/bisenetv2/bisenet_cityscapes_1024x1024_160k.yml
- extracting: test_tipc/configs/bisenetv2/train_infer_python.txt
- creating: test_tipc/configs/ddrnet/
- extracting: test_tipc/configs/ddrnet/ddrnet23_cityscapes_1024x1024_120k.yml
- extracting: test_tipc/configs/ddrnet/train_infer_python.txt
- creating: test_tipc/configs/deeplabv3p_resnet50/
- extracting: test_tipc/configs/deeplabv3p_resnet50/deeplabv3p_resnet50_humanseg_512x512_mini_supervisely.yml
- extracting: test_tipc/configs/deeplabv3p_resnet50/train_infer_python.txt
- creating: test_tipc/configs/deeplabv3p_resnet50_cityscapes/
- extracting: test_tipc/configs/deeplabv3p_resnet50_cityscapes/deeplabv3p_resnet50_1024x512_cityscapes.yml
- extracting: test_tipc/configs/deeplabv3p_resnet50_cityscapes/train_infer_python.txt
- creating: test_tipc/configs/enet/
- extracting: test_tipc/configs/enet/enet_cityscapes_1024x512_adam_0.002_80k.yml
- extracting: test_tipc/configs/enet/train_infer_python.txt
- creating: test_tipc/configs/fastscnn/
- extracting: test_tipc/configs/fastscnn/fastscnn_cityscapes.yml
- extracting: test_tipc/configs/fastscnn/train_infer_python.txt
- creating: test_tipc/configs/fcn_hrnetw18/
- extracting: test_tipc/configs/fcn_hrnetw18/fcn_hrnetw18_1024x512_cityscapes.yml
- extracting: test_tipc/configs/fcn_hrnetw18/train_infer_python.txt
- creating: test_tipc/configs/fcn_hrnetw18_small/
- extracting: test_tipc/configs/fcn_hrnetw18_small/fcn_hrnetw18_small_v1_humanseg_192x192_mini_supervisely.yml
- extracting: test_tipc/configs/fcn_hrnetw18_small/train_infer_python.txt
- creating: test_tipc/configs/ocrnet_hrnetw18/
- extracting: test_tipc/configs/ocrnet_hrnetw18/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml
- extracting: test_tipc/configs/ocrnet_hrnetw18/train_infer_python.txt
- creating: test_tipc/configs/ocrnet_hrnetw48/
- extracting: test_tipc/configs/ocrnet_hrnetw48/ocrnet_hrnetw48_cityscapes_1024x512.yml
- extracting: test_tipc/configs/ocrnet_hrnetw48/train_infer_python.txt
- creating: test_tipc/configs/pfpnnet/
- extracting: test_tipc/configs/pfpnnet/pfpn_resnet101_os8_cityscapes_512x1024_40k.yml
- extracting: test_tipc/configs/pfpnnet/train_infer_python.txt
- creating: test_tipc/configs/pphumanseg_lite/
- extracting: test_tipc/configs/pphumanseg_lite/pphumanseg_lite_mini_supervisely.yml
- extracting: test_tipc/configs/pphumanseg_lite/train_infer_python.txt
- creating: test_tipc/configs/ppmatting/
- extracting: test_tipc/configs/ppmatting/modnet_mobilenetv2.yml
- extracting: test_tipc/configs/ppmatting/train_infer_python.txt
- creating: test_tipc/configs/segformer_b0/
- extracting: test_tipc/configs/segformer_b0/segformer_b0_cityscapes_1024x1024_160k.yml
- extracting: test_tipc/configs/segformer_b0/train_infer_python.txt
- creating: test_tipc/configs/stdc_stdc1/
- extracting: test_tipc/configs/stdc_stdc1/stdc1_seg_cityscapes_1024x512_80k.yml
- extracting: test_tipc/configs/stdc_stdc1/train_infer_python.txt
- creating: test_tipc/data/
- extracting: test_tipc/data/cityscapes_val_5.list
- creating: test_tipc/docs/
- extracting: test_tipc/docs/benchmark_train.md
- extracting: test_tipc/docs/compare_right.png
- extracting: test_tipc/docs/compare_wrong.png
- extracting: test_tipc/docs/guide.png
- extracting: test_tipc/docs/install.md
- extracting: test_tipc/docs/test.png
- extracting: test_tipc/docs/test_infer_js.md
- extracting: test_tipc/docs/test_train_inference_python.md
- extracting: test_tipc/prepare.sh
- extracting: test_tipc/prepare_js.sh
- extracting: test_tipc/README.md
- extracting: test_tipc/requirements.txt
- creating: test_tipc/results/
- extracting: test_tipc/results/python_fcn_hrnetw18_small_results_fp16.txt
- extracting: test_tipc/results/python_fcn_hrnetw18_small_results_fp32.txt
- extracting: test_tipc/test_infer_js.sh
- extracting: test_tipc/test_train_inference_python.sh
- extracting: test_tipc/val.py
- creating: test_tipc/web/
- creating: test_tipc/web/imgs/
- extracting: test_tipc/web/imgs/human.jpg
- extracting: test_tipc/web/imgs/seg.png
- extracting: test_tipc/web/index.html
- extracting: test_tipc/web/index.test.js
- extracting: test_tipc/web/jest-puppeteer.config.js
- extracting: test_tipc/web/jest.config.js
- creating: tests/
- extracting: tests/analyze_infer_models_log.py
- extracting: tests/test_infer_models.sh
- creating: tools/
- extracting: tools/analyze_model.py
- extracting: tools/convert_cityscapes.py
- extracting: tools/convert_cocostuff.py
- extracting: tools/convert_voc2010.py
- extracting: tools/create_dataset_list.py
- extracting: tools/gray2pseudo_color.py
- extracting: tools/labelme2seg.py
- extracting: tools/split_dataset_list.py
- extracting: tools/visualize_annotation.py
- extracting: tools/voc_augment.py
- extracting: train.py
- extracting: val.py
- configs
- data
- ddrnet23_imagenet.pdparams
- DDRNet.zip
- deploy
- docs
- export.py
- paddleseg
- predict.py
- run.py
- run.sh
- setup.py
- tests
- test_tipc
- tools
- train.py
- val.py
- WARNING 2022-04-13 11:22:58,506 launch.py:423] Not found distinct arguments and compiled with cuda or xpu. Default use collective mode
- INFO 2022-04-13 11:22:58,509 launch_utils.py:528] Local start 4 processes. First process distributed environment info (Only For Debug):
- +=======================================================================================+
- | Distributed Envs Value |
- +---------------------------------------------------------------------------------------+
- | PADDLE_TRAINER_ID 0 |
- | PADDLE_CURRENT_ENDPOINT 127.0.0.1:33906 |
- | PADDLE_TRAINERS_NUM 4 |
- | PADDLE_TRAINER_ENDPOINTS ... 0.1:37051,127.0.0.1:45092,127.0.0.1:33430|
- | PADDLE_RANK_IN_NODE 0 |
- | PADDLE_LOCAL_DEVICE_IDS 0 |
- | PADDLE_WORLD_DEVICE_IDS 0,1,2,3 |
- | FLAGS_selected_gpus 0 |
- | FLAGS_selected_accelerators 0 |
- +=======================================================================================+
-
- INFO 2022-04-13 11:22:58,509 launch_utils.py:532] details abouts PADDLE_TRAINER_ENDPOINTS can be found in log/endpoints.log, and detail running logs maybe found in log/workerlog.0
- ----------- Configuration Arguments -----------
- backend: auto
- elastic_server: None
- force: False
- gpus: None
- heter_devices:
- heter_worker_num: None
- heter_workers:
- host: None
- http_port: None
- ips: 127.0.0.1
- job_id: None
- log_dir: log
- np: None
- nproc_per_node: None
- run_mode: None
- scale: 0
- server_num: None
- servers:
- training_script: train.py
- training_script_args: ['--config', 'configs/ddrnet/ddrnet23_cityscapes_1024x1024_120k.yml', '--num_workers', '8', '--do_eval', '--use_vdl', '--log_iter', '50', '--save_interval', '4000', '--save_dir', '/root/paddlejob/workspace/output']
- worker_num: None
- workers:
- ------------------------------------------------
- launch train in GPU mode!
- launch proc_id:540 idx:0
- launch proc_id:549 idx:1
- launch proc_id:559 idx:2
- launch proc_id:562 idx:3
- /root/paddlejob/workspace/code/paddleseg/models/losses/rmi_loss.py:78: DeprecationWarning: invalid escape sequence \i
- """
- 2022-04-13 11:23:10 [INFO]
- ------------Environment Information-------------
- platform: Linux-4.4.0-150-generic-x86_64-with-centos-6.10-Final
- Python: 3.7.0 (default, Nov 24 2018, 08:51:28) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
- Paddle compiled with cuda: True
- NVCC: Cuda compilation tools, release 10.1, V10.1.243
- cudnn: 7.6
- GPUs used: 4
- CUDA_VISIBLE_DEVICES: None
- GPU: ['GPU 0: Tesla V100-SXM2-32GB', 'GPU 1: Tesla V100-SXM2-32GB', 'GPU 2: Tesla V100-SXM2-32GB', 'GPU 3: Tesla V100-SXM2-32GB']
- GCC: gcc (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15)
- PaddleSeg: develop
- PaddlePaddle: 2.2.2
- OpenCV: 4.1.1
- ------------------------------------------------
- 2022-04-13 11:23:10 [INFO]
- ---------------Config Information---------------
- batch_size: 3
- iters: 120000
- loss:
- coef:
- - 1
- types:
- - ignore_index: 255
- type: OhemCrossEntropyLoss
- lr_scheduler:
- end_lr: 0.0
- learning_rate: 0.01
- power: 0.9
- type: PolynomialDecay
- model:
- enable_auxiliary_loss: false
- pretrained: ddrnet23_imagenet.pdparams
- type: DDRNet_23
- optimizer:
- momentum: 0.9
- type: sgd
- weight_decay: 0.0005
- train_dataset:
- dataset_root: data/cityscapes
- mode: train
- transforms:
- - max_scale_factor: 2.0
- min_scale_factor: 0.5
- scale_step_size: 0.25
- type: ResizeStepScaling
- - crop_size:
- - 1024
- - 1024
- type: RandomPaddingCrop
- - type: RandomHorizontalFlip
- - brightness_range: 0.4
- contrast_range: 0.4
- saturation_range: 0.4
- type: RandomDistort
- - type: Normalize
- type: Cityscapes
- val_dataset:
- dataset_root: data/cityscapes
- mode: val
- transforms:
- - type: Normalize
- type: Cityscapes
- ------------------------------------------------
- W0413 11:23:10.908237 540 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
- W0413 11:23:10.908286 540 device_context.cc:465] device: 0, cuDNN Version: 7.6.
- 2022-04-13 11:23:20 [INFO] Loading pretrained model from ddrnet23_imagenet.pdparams
- 2022-04-13 11:23:20 [WARNING] spp.scale1.1.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale1.1.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale1.1._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale1.1._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale1.3.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale2.1.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale2.1.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale2.1._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale2.1._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale2.3.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale3.1.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale3.1.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale3.1._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale3.1._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale3.3.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale4.1.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale4.1.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale4.1._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale4.1._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale4.3.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale0.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale0.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale0.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale0.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.scale0.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process1.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process1.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process1.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process1.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process1.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process2.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process2.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process2.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process2.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process2.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process3.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process3.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process3.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process3.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process3.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process4.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process4.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process4.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process4.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.process4.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.compression.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.compression.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.compression.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.compression.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.compression.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.shortcut.0.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.shortcut.0.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.shortcut.0._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.shortcut.0._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] spp.shortcut.2.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.bn1.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.bn1.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.bn1._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.bn1._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv_bn_relu._conv.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv_bn_relu._batch_norm.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv_bn_relu._batch_norm.bias is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv_bn_relu._batch_norm._mean is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv_bn_relu._batch_norm._variance is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv.weight is not in pretrained model
- 2022-04-13 11:23:20 [WARNING] head.conv.bias is not in pretrained model
- 2022-04-13 11:23:20 [INFO] There are 212/278 variables loaded into DualResNet.
- server not ready, wait 3 sec to retry...
- not ready endpoints:['127.0.0.1:37051', '127.0.0.1:45092', '127.0.0.1:33430']
- I0413 11:23:23.432117 540 nccl_context.cc:74] init nccl context nranks: 4 local rank: 0 gpu id: 0 ring id: 0
- I0413 11:23:24.023981 540 nccl_context.cc:107] init nccl context nranks: 4 local rank: 0 gpu id: 0 ring id: 10
- 2022-04-13 11:23:24,323-INFO: [topology.py:169:__init__] HybridParallelInfo: rank_id: 0, mp_degree: 1, sharding_degree: 1, pp_degree: 1, dp_degree: 4, mp_group: [0], sharding_group: [0], pp_group: [0], dp_group: [0, 1, 2, 3], check/clip group: [0]
- /opt/_internal/cpython-3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:253: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.float32, but right dtype is paddle.int64, the right dtype will convert to paddle.float32
- format(lhs_dtype, rhs_dtype, lhs_dtype))
- 2022-04-13 11:23:38 [INFO] [TRAIN] epoch: 1, iter: 50/120000, loss: 1.7947, lr: 0.009996, batch_cost: 0.2846, reader_cost: 0.06491, ips: 10.5410 samples/sec | ETA 09:28:58
- 2022-04-13 11:23:49 [INFO] [TRAIN] epoch: 1, iter: 100/120000, loss: 1.4244, lr: 0.009993, batch_cost: 0.2184, reader_cost: 0.00140, ips: 13.7333 samples/sec | ETA 07:16:31
- 2022-04-13 11:24:00 [INFO] [TRAIN] epoch: 1, iter: 150/120000, loss: 1.2841, lr: 0.009989, batch_cost: 0.2208, reader_cost: 0.00091, ips: 13.5851 samples/sec | ETA 07:21:06
- 2022-04-13 11:24:10 [INFO] [TRAIN] epoch: 1, iter: 200/120000, loss: 1.2529, lr: 0.009985, batch_cost: 0.2031, reader_cost: 0.00089, ips: 14.7726 samples/sec | ETA 06:45:28
- 2022-04-13 11:24:24 [INFO] [TRAIN] epoch: 2, iter: 250/120000, loss: 1.1831, lr: 0.009981, batch_cost: 0.2662, reader_cost: 0.05398, ips: 11.2717 samples/sec | ETA 08:51:11
- 2022-04-13 11:24:34 [INFO] [TRAIN] epoch: 2, iter: 300/120000, loss: 1.2973, lr: 0.009978, batch_cost: 0.2179, reader_cost: 0.00130, ips: 13.7650 samples/sec | ETA 07:14:47
- 2022-04-13 11:24:45 [INFO] [TRAIN] epoch: 2, iter: 350/120000, loss: 1.2000, lr: 0.009974, batch_cost: 0.2035, reader_cost: 0.00155, ips: 14.7413 samples/sec | ETA 06:45:49
- 2022-04-13 11:24:55 [INFO] [TRAIN] epoch: 2, iter: 400/120000, loss: 1.2292, lr: 0.009970, batch_cost: 0.2131, reader_cost: 0.00042, ips: 14.0749 samples/sec | ETA 07:04:52
- 2022-04-13 11:25:06 [INFO] [TRAIN] epoch: 2, iter: 450/120000, loss: 1.1210, lr: 0.009966, batch_cost: 0.2200, reader_cost: 0.00061, ips: 13.6390 samples/sec | ETA 07:18:15
- 2022-04-13 11:25:20 [INFO] [TRAIN] epoch: 3, iter: 500/120000, loss: 1.1606, lr: 0.009963, batch_cost: 0.2746, reader_cost: 0.05576, ips: 10.9233 samples/sec | ETA 09:06:59
- 2022-04-13 11:25:31 [INFO] [TRAIN] epoch: 3, iter: 550/120000, loss: 1.0975, lr: 0.009959, batch_cost: 0.2174, reader_cost: 0.00086, ips: 13.8001 samples/sec | ETA 07:12:47
- 2022-04-13 11:25:41 [INFO] [TRAIN] epoch: 3, iter: 600/120000, loss: 1.1242, lr: 0.009955, batch_cost: 0.2114, reader_cost: 0.00137, ips: 14.1881 samples/sec | ETA 07:00:46
- 2022-04-13 11:25:52 [INFO] [TRAIN] epoch: 3, iter: 650/120000, loss: 1.1743, lr: 0.009951, batch_cost: 0.2125, reader_cost: 0.00051, ips: 14.1147 samples/sec | ETA 07:02:47
- 2022-04-13 11:26:03 [INFO] [TRAIN] epoch: 3, iter: 700/120000, loss: 1.0871, lr: 0.009948, batch_cost: 0.2229, reader_cost: 0.00082, ips: 13.4577 samples/sec | ETA 07:23:14
- 2022-04-13 11:26:16 [INFO] [TRAIN] epoch: 4, iter: 750/120000, loss: 1.0838, lr: 0.009944, batch_cost: 0.2646, reader_cost: 0.06165, ips: 11.3372 samples/sec | ETA 08:45:55
- 2022-04-13 11:26:27 [INFO] [TRAIN] epoch: 4, iter: 800/120000, loss: 1.1024, lr: 0.009940, batch_cost: 0.2095, reader_cost: 0.00117, ips: 14.3176 samples/sec | ETA 06:56:16
- 2022-04-13 11:26:38 [INFO] [TRAIN] epoch: 4, iter: 850/120000, loss: 1.1196, lr: 0.009936, batch_cost: 0.2268, reader_cost: 0.00079, ips: 13.2303 samples/sec | ETA 07:30:17
- 2022-04-13 11:26:49 [INFO] [TRAIN] epoch: 4, iter: 900/120000, loss: 1.0939, lr: 0.009933, batch_cost: 0.2166, reader_cost: 0.00089, ips: 13.8513 samples/sec | ETA 07:09:55
- 2022-04-13 11:27:00 [INFO] [TRAIN] epoch: 4, iter: 950/120000, loss: 1.1206, lr: 0.009929, batch_cost: 0.2172, reader_cost: 0.00100, ips: 13.8120 samples/sec | ETA 07:10:58
- 2022-04-13 11:27:13 [INFO] [TRAIN] epoch: 5, iter: 1000/120000, loss: 1.0371, lr: 0.009925, batch_cost: 0.2637, reader_cost: 0.05205, ips: 11.3752 samples/sec | ETA 08:43:03
- 2022-04-13 11:27:23 [INFO] [TRAIN] epoch: 5, iter: 1050/120000, loss: 1.0572, lr: 0.009921, batch_cost: 0.2010, reader_cost: 0.00117, ips: 14.9260 samples/sec | ETA 06:38:27
- 2022-04-13 11:27:33 [INFO] [TRAIN] epoch: 5, iter: 1100/120000, loss: 1.0838, lr: 0.009918, batch_cost: 0.1983, reader_cost: 0.00091, ips: 15.1274 samples/sec | ETA 06:32:59
- 2022-04-13 11:27:44 [INFO] [TRAIN] epoch: 5, iter: 1150/120000, loss: 1.1086, lr: 0.009914, batch_cost: 0.2107, reader_cost: 0.00093, ips: 14.2350 samples/sec | ETA 06:57:27
- 2022-04-13 11:27:55 [INFO] [TRAIN] epoch: 5, iter: 1200/120000, loss: 1.1029, lr: 0.009910, batch_cost: 0.2214, reader_cost: 0.00125, ips: 13.5500 samples/sec | ETA 07:18:22
- 2022-04-13 11:28:08 [INFO] [TRAIN] epoch: 6, iter: 1250/120000, loss: 1.0866, lr: 0.009906, batch_cost: 0.2686, reader_cost: 0.05436, ips: 11.1674 samples/sec | ETA 08:51:40
- 2022-04-13 11:28:18 [INFO] [TRAIN] epoch: 6, iter: 1300/120000, loss: 1.0896, lr: 0.009903, batch_cost: 0.1939, reader_cost: 0.00111, ips: 15.4685 samples/sec | ETA 06:23:41
- 2022-04-13 11:28:28 [INFO] [TRAIN] epoch: 6, iter: 1350/120000, loss: 1.0614, lr: 0.009899, batch_cost: 0.1980, reader_cost: 0.00099, ips: 15.1543 samples/sec | ETA 06:31:28
- 2022-04-13 11:28:38 [INFO] [TRAIN] epoch: 6, iter: 1400/120000, loss: 1.0590, lr: 0.009895, batch_cost: 0.2071, reader_cost: 0.00169, ips: 14.4824 samples/sec | ETA 06:49:27
- 2022-04-13 11:28:48 [INFO] [TRAIN] epoch: 6, iter: 1450/120000, loss: 1.0522, lr: 0.009891, batch_cost: 0.2035, reader_cost: 0.00087, ips: 14.7435 samples/sec | ETA 06:42:02
- 2022-04-13 11:29:02 [INFO] [TRAIN] epoch: 7, iter: 1500/120000, loss: 1.0627, lr: 0.009888, batch_cost: 0.2719, reader_cost: 0.05382, ips: 11.0326 samples/sec | ETA 08:57:02
- 2022-04-13 11:29:13 [INFO] [TRAIN] epoch: 7, iter: 1550/120000, loss: 1.0825, lr: 0.009884, batch_cost: 0.2202, reader_cost: 0.00096, ips: 13.6234 samples/sec | ETA 07:14:43
- 2022-04-13 11:29:24 [INFO] [TRAIN] epoch: 7, iter: 1600/120000, loss: 1.0281, lr: 0.009880, batch_cost: 0.2173, reader_cost: 0.00057, ips: 13.8041 samples/sec | ETA 07:08:51
- 2022-04-13 11:29:34 [INFO] [TRAIN] epoch: 7, iter: 1650/120000, loss: 1.0688, lr: 0.009876, batch_cost: 0.1968, reader_cost: 0.00086, ips: 15.2411 samples/sec | ETA 06:28:15
- 2022-04-13 11:29:44 [INFO] [TRAIN] epoch: 7, iter: 1700/120000, loss: 1.0530, lr: 0.009872, batch_cost: 0.2073, reader_cost: 0.00085, ips: 14.4692 samples/sec | ETA 06:48:48
- 2022-04-13 11:29:57 [INFO] [TRAIN] epoch: 8, iter: 1750/120000, loss: 1.0301, lr: 0.009869, batch_cost: 0.2635, reader_cost: 0.04573, ips: 11.3866 samples/sec | ETA 08:39:14
- 2022-04-13 11:30:09 [INFO] [TRAIN] epoch: 8, iter: 1800/120000, loss: 1.0836, lr: 0.009865, batch_cost: 0.2317, reader_cost: 0.00055, ips: 12.9452 samples/sec | ETA 07:36:32
- 2022-04-13 11:30:20 [INFO] [TRAIN] epoch: 8, iter: 1850/120000, loss: 1.0461, lr: 0.009861, batch_cost: 0.2183, reader_cost: 0.00140, ips: 13.7446 samples/sec | ETA 07:09:48
- 2022-04-13 11:30:30 [INFO] [TRAIN] epoch: 8, iter: 1900/120000, loss: 1.0361, lr: 0.009857, batch_cost: 0.2092, reader_cost: 0.00104, ips: 14.3431 samples/sec | ETA 06:51:41
- 2022-04-13 11:30:40 [INFO] [TRAIN] epoch: 8, iter: 1950/120000, loss: 1.0835, lr: 0.009854, batch_cost: 0.2033, reader_cost: 0.00088, ips: 14.7537 samples/sec | ETA 06:40:04
- 2022-04-13 11:30:53 [INFO] [TRAIN] epoch: 9, iter: 2000/120000, loss: 1.0432, lr: 0.009850, batch_cost: 0.2583, reader_cost: 0.05483, ips: 11.6137 samples/sec | ETA 08:28:01
- 2022-04-13 11:31:03 [INFO] [TRAIN] epoch: 9, iter: 2050/120000, loss: 0.9986, lr: 0.009846, batch_cost: 0.1937, reader_cost: 0.00066, ips: 15.4918 samples/sec | ETA 06:20:41
- 2022-04-13 11:31:13 [INFO] [TRAIN] epoch: 9, iter: 2100/120000, loss: 1.0746, lr: 0.009842, batch_cost: 0.2070, reader_cost: 0.00100, ips: 14.4929 samples/sec | ETA 06:46:45
- 2022-04-13 11:31:23 [INFO] [TRAIN] epoch: 9, iter: 2150/120000, loss: 1.0051, lr: 0.009839, batch_cost: 0.2042, reader_cost: 0.00068, ips: 14.6906 samples/sec | ETA 06:41:06
- 2022-04-13 11:31:34 [INFO] [TRAIN] epoch: 9, iter: 2200/120000, loss: 1.0629, lr: 0.009835, batch_cost: 0.2011, reader_cost: 0.00145, ips: 14.9176 samples/sec | ETA 06:34:50
- 2022-04-13 11:31:47 [INFO] [TRAIN] epoch: 10, iter: 2250/120000, loss: 1.0325, lr: 0.009831, batch_cost: 0.2668, reader_cost: 0.05412, ips: 11.2430 samples/sec | ETA 08:43:39
- 2022-04-13 11:31:58 [INFO] [TRAIN] epoch: 10, iter: 2300/120000, loss: 1.0668, lr: 0.009827, batch_cost: 0.2243, reader_cost: 0.00139, ips: 13.3739 samples/sec | ETA 07:20:02
- 2022-04-13 11:32:08 [INFO] [TRAIN] epoch: 10, iter: 2350/120000, loss: 1.0730, lr: 0.009824, batch_cost: 0.1976, reader_cost: 0.00123, ips: 15.1835 samples/sec | ETA 06:27:25
- 2022-04-13 11:32:19 [INFO] [TRAIN] epoch: 10, iter: 2400/120000, loss: 1.0351, lr: 0.009820, batch_cost: 0.2180, reader_cost: 0.00102, ips: 13.7632 samples/sec | ETA 07:07:13
- 2022-04-13 11:32:29 [INFO] [TRAIN] epoch: 10, iter: 2450/120000, loss: 1.0094, lr: 0.009816, batch_cost: 0.2086, reader_cost: 0.00043, ips: 14.3808 samples/sec | ETA 06:48:42
- 2022-04-13 11:32:42 [INFO] [TRAIN] epoch: 11, iter: 2500/120000, loss: 1.0194, lr: 0.009812, batch_cost: 0.2606, reader_cost: 0.05242, ips: 11.5107 samples/sec | ETA 08:30:23
- 2022-04-13 11:32:52 [INFO] [TRAIN] epoch: 11, iter: 2550/120000, loss: 1.0231, lr: 0.009809, batch_cost: 0.1984, reader_cost: 0.00071, ips: 15.1211 samples/sec | ETA 06:28:21
- 2022-04-13 11:33:02 [INFO] [TRAIN] epoch: 11, iter: 2600/120000, loss: 1.0260, lr: 0.009805, batch_cost: 0.1980, reader_cost: 0.00147, ips: 15.1536 samples/sec | ETA 06:27:22
- 2022-04-13 11:33:14 [INFO] [TRAIN] epoch: 11, iter: 2650/120000, loss: 1.0013, lr: 0.009801, batch_cost: 0.2282, reader_cost: 0.00095, ips: 13.1457 samples/sec | ETA 07:26:20
- 2022-04-13 11:33:25 [INFO] [TRAIN] epoch: 11, iter: 2700/120000, loss: 1.0889, lr: 0.009797, batch_cost: 0.2220, reader_cost: 0.00062, ips: 13.5145 samples/sec | ETA 07:13:58
- 2022-04-13 11:33:38 [INFO] [TRAIN] epoch: 12, iter: 2750/120000, loss: 1.0342, lr: 0.009794, batch_cost: 0.2710, reader_cost: 0.05217, ips: 11.0682 samples/sec | ETA 08:49:40
- 2022-04-13 11:33:48 [INFO] [TRAIN] epoch: 12, iter: 2800/120000, loss: 1.0322, lr: 0.009790, batch_cost: 0.1935, reader_cost: 0.00089, ips: 15.5002 samples/sec | ETA 06:18:03
- 2022-04-13 11:33:58 [INFO] [TRAIN] epoch: 12, iter: 2850/120000, loss: 1.0375, lr: 0.009786, batch_cost: 0.2082, reader_cost: 0.00081, ips: 14.4075 samples/sec | ETA 06:46:33
- 2022-04-13 11:34:08 [INFO] [TRAIN] epoch: 12, iter: 2900/120000, loss: 1.0353, lr: 0.009782, batch_cost: 0.1978, reader_cost: 0.00072, ips: 15.1670 samples/sec | ETA 06:26:02
- 2022-04-13 11:34:19 [INFO] [TRAIN] epoch: 12, iter: 2950/120000, loss: 1.0064, lr: 0.009779, batch_cost: 0.2226, reader_cost: 0.00122, ips: 13.4765 samples/sec | ETA 07:14:16
- 2022-04-13 11:34:32 [INFO] [TRAIN] epoch: 13, iter: 3000/120000, loss: 0.9936, lr: 0.009775, batch_cost: 0.2595, reader_cost: 0.05337, ips: 11.5617 samples/sec | ETA 08:25:58
- 2022-04-13 11:34:43 [INFO] [TRAIN] epoch: 13, iter: 3050/120000, loss: 1.0667, lr: 0.009771, batch_cost: 0.2082, reader_cost: 0.00055, ips: 14.4110 samples/sec | ETA 06:45:46
- 2022-04-13 11:34:53 [INFO] [TRAIN] epoch: 13, iter: 3100/120000, loss: 1.0655, lr: 0.009767, batch_cost: 0.1962, reader_cost: 0.00054, ips: 15.2901 samples/sec | ETA 06:22:16
- 2022-04-13 11:35:03 [INFO] [TRAIN] epoch: 13, iter: 3150/120000, loss: 1.0246, lr: 0.009764, batch_cost: 0.2084, reader_cost: 0.00101, ips: 14.3970 samples/sec | ETA 06:45:48
- 2022-04-13 11:35:15 [INFO] [TRAIN] epoch: 13, iter: 3200/120000, loss: 0.9934, lr: 0.009760, batch_cost: 0.2323, reader_cost: 0.00132, ips: 12.9167 samples/sec | ETA 07:32:07
- 2022-04-13 11:35:28 [INFO] [TRAIN] epoch: 14, iter: 3250/120000, loss: 0.9939, lr: 0.009756, batch_cost: 0.2693, reader_cost: 0.04944, ips: 11.1384 samples/sec | ETA 08:44:05
- 2022-04-13 11:35:38 [INFO] [TRAIN] epoch: 14, iter: 3300/120000, loss: 1.0570, lr: 0.009752, batch_cost: 0.1952, reader_cost: 0.00116, ips: 15.3721 samples/sec | ETA 06:19:35
- 2022-04-13 11:35:48 [INFO] [TRAIN] epoch: 14, iter: 3350/120000, loss: 1.0474, lr: 0.009748, batch_cost: 0.2073, reader_cost: 0.00104, ips: 14.4704 samples/sec | ETA 06:43:03
- 2022-04-13 11:35:59 [INFO] [TRAIN] epoch: 14, iter: 3400/120000, loss: 1.0354, lr: 0.009745, batch_cost: 0.2123, reader_cost: 0.00070, ips: 14.1320 samples/sec | ETA 06:52:32
- 2022-04-13 11:36:10 [INFO] [TRAIN] epoch: 14, iter: 3450/120000, loss: 0.9870, lr: 0.009741, batch_cost: 0.2306, reader_cost: 0.00080, ips: 13.0096 samples/sec | ETA 07:27:56
- 2022-04-13 11:36:24 [INFO] [TRAIN] epoch: 15, iter: 3500/120000, loss: 1.0212, lr: 0.009737, batch_cost: 0.2664, reader_cost: 0.05277, ips: 11.2630 samples/sec | ETA 08:37:10
- 2022-04-13 11:36:34 [INFO] [TRAIN] epoch: 15, iter: 3550/120000, loss: 1.0498, lr: 0.009733, batch_cost: 0.1986, reader_cost: 0.00112, ips: 15.1049 samples/sec | ETA 06:25:28
- 2022-04-13 11:36:44 [INFO] [TRAIN] epoch: 15, iter: 3600/120000, loss: 1.0093, lr: 0.009730, batch_cost: 0.1993, reader_cost: 0.00067, ips: 15.0510 samples/sec | ETA 06:26:41
- 2022-04-13 11:36:55 [INFO] [TRAIN] epoch: 15, iter: 3650/120000, loss: 0.9628, lr: 0.009726, batch_cost: 0.2246, reader_cost: 0.00077, ips: 13.3568 samples/sec | ETA 07:15:32
- 2022-04-13 11:37:06 [INFO] [TRAIN] epoch: 15, iter: 3700/120000, loss: 1.0674, lr: 0.009722, batch_cost: 0.2340, reader_cost: 0.00090, ips: 12.8209 samples/sec | ETA 07:33:33
- 2022-04-13 11:37:20 [INFO] [TRAIN] epoch: 16, iter: 3750/120000, loss: 0.9859, lr: 0.009718, batch_cost: 0.2632, reader_cost: 0.04969, ips: 11.3988 samples/sec | ETA 08:29:55
- 2022-04-13 11:37:30 [INFO] [TRAIN] epoch: 16, iter: 3800/120000, loss: 1.0042, lr: 0.009715, batch_cost: 0.2090, reader_cost: 0.00056, ips: 14.3516 samples/sec | ETA 06:44:50
- 2022-04-13 11:37:42 [INFO] [TRAIN] epoch: 16, iter: 3850/120000, loss: 1.0448, lr: 0.009711, batch_cost: 0.2436, reader_cost: 0.00089, ips: 12.3133 samples/sec | ETA 07:51:38
- 2022-04-13 11:37:53 [INFO] [TRAIN] epoch: 16, iter: 3900/120000, loss: 1.0165, lr: 0.009707, batch_cost: 0.2051, reader_cost: 0.00091, ips: 14.6258 samples/sec | ETA 06:36:54
- 2022-04-13 11:38:03 [INFO] [TRAIN] epoch: 16, iter: 3950/120000, loss: 1.0456, lr: 0.009703, batch_cost: 0.2084, reader_cost: 0.00075, ips: 14.3977 samples/sec | ETA 06:43:00
- 2022-04-13 11:38:16 [INFO] [TRAIN] epoch: 17, iter: 4000/120000, loss: 1.0175, lr: 0.009700, batch_cost: 0.2657, reader_cost: 0.05626, ips: 11.2926 samples/sec | ETA 08:33:36
- 2022-04-13 11:38:16 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1958 - reader cost: 0.1529
- 2022-04-13 11:38:41 [INFO] [EVAL] #Images: 500 mIoU: 0.6525 Acc: 0.9407 Kappa: 0.9228 Dice: 0.7726
- 2022-04-13 11:38:41 [INFO] [EVAL] Class IoU:
- [0.9656 0.7509 0.8964 0.3943 0.4289 0.5439 0.5894 0.7015 0.9019 0.5157
- 0.928 0.7491 0.5035 0.9037 0.5104 0.6699 0.414 0.3417 0.6891]
- 2022-04-13 11:38:41 [INFO] [EVAL] Class Precision:
- [0.9754 0.9078 0.9367 0.8078 0.7127 0.7652 0.7904 0.9126 0.9358 0.789
- 0.9564 0.8791 0.6755 0.926 0.919 0.7571 0.48 0.5475 0.8377]
- 2022-04-13 11:38:41 [INFO] [EVAL] Class Recall:
- [0.9897 0.8128 0.9541 0.4351 0.5185 0.6528 0.6986 0.752 0.9613 0.5982
- 0.9691 0.8352 0.6641 0.974 0.5345 0.8532 0.7508 0.4761 0.7953]
- 2022-04-13 11:38:42 [INFO] [EVAL] The model with the best validation mIoU (0.6525) was saved at iter 4000.
- 2022-04-13 11:38:53 [INFO] [TRAIN] epoch: 17, iter: 4050/120000, loss: 0.9800, lr: 0.009696, batch_cost: 0.2156, reader_cost: 0.00488, ips: 13.9125 samples/sec | ETA 06:56:42
- 2022-04-13 11:39:02 [INFO] [TRAIN] epoch: 17, iter: 4100/120000, loss: 1.0060, lr: 0.009692, batch_cost: 0.1980, reader_cost: 0.00107, ips: 15.1547 samples/sec | ETA 06:22:23
- 2022-04-13 11:39:14 [INFO] [TRAIN] epoch: 17, iter: 4150/120000, loss: 1.0250, lr: 0.009688, batch_cost: 0.2215, reader_cost: 0.00136, ips: 13.5422 samples/sec | ETA 07:07:44
- 2022-04-13 11:39:24 [INFO] [TRAIN] epoch: 17, iter: 4200/120000, loss: 0.9728, lr: 0.009685, batch_cost: 0.2104, reader_cost: 0.00055, ips: 14.2596 samples/sec | ETA 06:46:02
- 2022-04-13 11:39:37 [INFO] [TRAIN] epoch: 18, iter: 4250/120000, loss: 0.9988, lr: 0.009681, batch_cost: 0.2647, reader_cost: 0.05792, ips: 11.3335 samples/sec | ETA 08:30:39
- 2022-04-13 11:39:48 [INFO] [TRAIN] epoch: 18, iter: 4300/120000, loss: 1.0212, lr: 0.009677, batch_cost: 0.2139, reader_cost: 0.00130, ips: 14.0275 samples/sec | ETA 06:52:24
- 2022-04-13 11:40:00 [INFO] [TRAIN] epoch: 18, iter: 4350/120000, loss: 0.9904, lr: 0.009673, batch_cost: 0.2320, reader_cost: 0.00092, ips: 12.9331 samples/sec | ETA 07:27:06
- 2022-04-13 11:40:11 [INFO] [TRAIN] epoch: 18, iter: 4400/120000, loss: 0.9695, lr: 0.009669, batch_cost: 0.2213, reader_cost: 0.00056, ips: 13.5556 samples/sec | ETA 07:06:23
- 2022-04-13 11:40:21 [INFO] [TRAIN] epoch: 18, iter: 4450/120000, loss: 1.0919, lr: 0.009666, batch_cost: 0.2079, reader_cost: 0.00141, ips: 14.4326 samples/sec | ETA 06:40:18
- 2022-04-13 11:40:34 [INFO] [TRAIN] epoch: 19, iter: 4500/120000, loss: 0.9978, lr: 0.009662, batch_cost: 0.2663, reader_cost: 0.05679, ips: 11.2660 samples/sec | ETA 08:32:36
- 2022-04-13 11:40:45 [INFO] [TRAIN] epoch: 19, iter: 4550/120000, loss: 0.9823, lr: 0.009658, batch_cost: 0.2092, reader_cost: 0.00112, ips: 14.3395 samples/sec | ETA 06:42:33
- 2022-04-13 11:40:56 [INFO] [TRAIN] epoch: 19, iter: 4600/120000, loss: 0.9900, lr: 0.009654, batch_cost: 0.2144, reader_cost: 0.00099, ips: 13.9957 samples/sec | ETA 06:52:16
- 2022-04-13 11:41:05 [INFO] [TRAIN] epoch: 19, iter: 4650/120000, loss: 0.9819, lr: 0.009651, batch_cost: 0.1979, reader_cost: 0.00109, ips: 15.1611 samples/sec | ETA 06:20:24
- 2022-04-13 11:41:16 [INFO] [TRAIN] epoch: 19, iter: 4700/120000, loss: 0.9621, lr: 0.009647, batch_cost: 0.2030, reader_cost: 0.00078, ips: 14.7795 samples/sec | ETA 06:30:04
- 2022-04-13 11:41:30 [INFO] [TRAIN] epoch: 20, iter: 4750/120000, loss: 0.9878, lr: 0.009643, batch_cost: 0.2899, reader_cost: 0.06004, ips: 10.3495 samples/sec | ETA 09:16:47
- 2022-04-13 11:41:41 [INFO] [TRAIN] epoch: 20, iter: 4800/120000, loss: 0.9932, lr: 0.009639, batch_cost: 0.2119, reader_cost: 0.00067, ips: 14.1543 samples/sec | ETA 06:46:56
- 2022-04-13 11:41:51 [INFO] [TRAIN] epoch: 20, iter: 4850/120000, loss: 1.0001, lr: 0.009636, batch_cost: 0.2073, reader_cost: 0.00082, ips: 14.4721 samples/sec | ETA 06:37:50
- 2022-04-13 11:42:01 [INFO] [TRAIN] epoch: 20, iter: 4900/120000, loss: 1.0238, lr: 0.009632, batch_cost: 0.2051, reader_cost: 0.00165, ips: 14.6279 samples/sec | ETA 06:33:25
- 2022-04-13 11:42:12 [INFO] [TRAIN] epoch: 20, iter: 4950/120000, loss: 1.0005, lr: 0.009628, batch_cost: 0.2233, reader_cost: 0.00056, ips: 13.4343 samples/sec | ETA 07:08:11
- 2022-04-13 11:42:26 [INFO] [TRAIN] epoch: 21, iter: 5000/120000, loss: 0.9995, lr: 0.009624, batch_cost: 0.2695, reader_cost: 0.05289, ips: 11.1311 samples/sec | ETA 08:36:34
- 2022-04-13 11:42:36 [INFO] [TRAIN] epoch: 21, iter: 5050/120000, loss: 1.0113, lr: 0.009621, batch_cost: 0.2100, reader_cost: 0.00136, ips: 14.2857 samples/sec | ETA 06:42:19
- 2022-04-13 11:42:47 [INFO] [TRAIN] epoch: 21, iter: 5100/120000, loss: 0.9626, lr: 0.009617, batch_cost: 0.2040, reader_cost: 0.00061, ips: 14.7056 samples/sec | ETA 06:30:40
- 2022-04-13 11:42:57 [INFO] [TRAIN] epoch: 21, iter: 5150/120000, loss: 0.9631, lr: 0.009613, batch_cost: 0.2057, reader_cost: 0.00081, ips: 14.5870 samples/sec | ETA 06:33:40
- 2022-04-13 11:43:07 [INFO] [TRAIN] epoch: 21, iter: 5200/120000, loss: 0.9986, lr: 0.009609, batch_cost: 0.1997, reader_cost: 0.00056, ips: 15.0255 samples/sec | ETA 06:22:01
- 2022-04-13 11:43:21 [INFO] [TRAIN] epoch: 22, iter: 5250/120000, loss: 0.9716, lr: 0.009605, batch_cost: 0.2839, reader_cost: 0.05708, ips: 10.5680 samples/sec | ETA 09:02:54
- 2022-04-13 11:43:31 [INFO] [TRAIN] epoch: 22, iter: 5300/120000, loss: 0.9845, lr: 0.009602, batch_cost: 0.2074, reader_cost: 0.00059, ips: 14.4681 samples/sec | ETA 06:36:23
- 2022-04-13 11:43:42 [INFO] [TRAIN] epoch: 22, iter: 5350/120000, loss: 0.9727, lr: 0.009598, batch_cost: 0.2016, reader_cost: 0.00171, ips: 14.8840 samples/sec | ETA 06:25:08
- 2022-04-13 11:43:52 [INFO] [TRAIN] epoch: 22, iter: 5400/120000, loss: 0.9736, lr: 0.009594, batch_cost: 0.2003, reader_cost: 0.00164, ips: 14.9749 samples/sec | ETA 06:22:38
- 2022-04-13 11:44:02 [INFO] [TRAIN] epoch: 22, iter: 5450/120000, loss: 1.0105, lr: 0.009590, batch_cost: 0.2032, reader_cost: 0.00123, ips: 14.7664 samples/sec | ETA 06:27:52
- 2022-04-13 11:44:16 [INFO] [TRAIN] epoch: 23, iter: 5500/120000, loss: 0.9628, lr: 0.009587, batch_cost: 0.2775, reader_cost: 0.04743, ips: 10.8096 samples/sec | ETA 08:49:37
- 2022-04-13 11:44:27 [INFO] [TRAIN] epoch: 23, iter: 5550/120000, loss: 1.0061, lr: 0.009583, batch_cost: 0.2183, reader_cost: 0.00081, ips: 13.7397 samples/sec | ETA 06:56:29
- 2022-04-13 11:44:37 [INFO] [TRAIN] epoch: 23, iter: 5600/120000, loss: 0.9708, lr: 0.009579, batch_cost: 0.2013, reader_cost: 0.00084, ips: 14.9026 samples/sec | ETA 06:23:49
- 2022-04-13 11:44:49 [INFO] [TRAIN] epoch: 23, iter: 5650/120000, loss: 0.9316, lr: 0.009575, batch_cost: 0.2387, reader_cost: 0.00122, ips: 12.5658 samples/sec | ETA 07:35:00
- 2022-04-13 11:44:58 [INFO] [TRAIN] epoch: 23, iter: 5700/120000, loss: 0.9540, lr: 0.009572, batch_cost: 0.1885, reader_cost: 0.00083, ips: 15.9163 samples/sec | ETA 05:59:03
- 2022-04-13 11:45:12 [INFO] [TRAIN] epoch: 24, iter: 5750/120000, loss: 0.9762, lr: 0.009568, batch_cost: 0.2753, reader_cost: 0.06117, ips: 10.8958 samples/sec | ETA 08:44:17
- 2022-04-13 11:45:22 [INFO] [TRAIN] epoch: 24, iter: 5800/120000, loss: 0.9823, lr: 0.009564, batch_cost: 0.2093, reader_cost: 0.00081, ips: 14.3343 samples/sec | ETA 06:38:20
- 2022-04-13 11:45:32 [INFO] [TRAIN] epoch: 24, iter: 5850/120000, loss: 0.9988, lr: 0.009560, batch_cost: 0.2055, reader_cost: 0.00087, ips: 14.5963 samples/sec | ETA 06:31:01
- 2022-04-13 11:45:43 [INFO] [TRAIN] epoch: 24, iter: 5900/120000, loss: 0.9924, lr: 0.009556, batch_cost: 0.2050, reader_cost: 0.00113, ips: 14.6343 samples/sec | ETA 06:29:50
- 2022-04-13 11:45:53 [INFO] [TRAIN] epoch: 24, iter: 5950/120000, loss: 0.9522, lr: 0.009553, batch_cost: 0.2113, reader_cost: 0.00119, ips: 14.2002 samples/sec | ETA 06:41:34
- 2022-04-13 11:46:08 [INFO] [TRAIN] epoch: 25, iter: 6000/120000, loss: 0.9073, lr: 0.009549, batch_cost: 0.2956, reader_cost: 0.05323, ips: 10.1484 samples/sec | ETA 09:21:39
- 2022-04-13 11:46:19 [INFO] [TRAIN] epoch: 25, iter: 6050/120000, loss: 1.0016, lr: 0.009545, batch_cost: 0.2243, reader_cost: 0.00069, ips: 13.3726 samples/sec | ETA 07:06:03
- 2022-04-13 11:46:30 [INFO] [TRAIN] epoch: 25, iter: 6100/120000, loss: 0.9978, lr: 0.009541, batch_cost: 0.2181, reader_cost: 0.00063, ips: 13.7538 samples/sec | ETA 06:54:04
- 2022-04-13 11:46:41 [INFO] [TRAIN] epoch: 25, iter: 6150/120000, loss: 0.9725, lr: 0.009538, batch_cost: 0.2187, reader_cost: 0.00143, ips: 13.7182 samples/sec | ETA 06:54:57
- 2022-04-13 11:46:52 [INFO] [TRAIN] epoch: 25, iter: 6200/120000, loss: 0.9915, lr: 0.009534, batch_cost: 0.2084, reader_cost: 0.00055, ips: 14.3967 samples/sec | ETA 06:35:13
- 2022-04-13 11:47:06 [INFO] [TRAIN] epoch: 26, iter: 6250/120000, loss: 0.9764, lr: 0.009530, batch_cost: 0.2802, reader_cost: 0.06309, ips: 10.7059 samples/sec | ETA 08:51:14
- 2022-04-13 11:47:16 [INFO] [TRAIN] epoch: 26, iter: 6300/120000, loss: 1.0242, lr: 0.009526, batch_cost: 0.2061, reader_cost: 0.00123, ips: 14.5528 samples/sec | ETA 06:30:38
- 2022-04-13 11:47:26 [INFO] [TRAIN] epoch: 26, iter: 6350/120000, loss: 0.9909, lr: 0.009523, batch_cost: 0.2014, reader_cost: 0.00141, ips: 14.8953 samples/sec | ETA 06:21:29
- 2022-04-13 11:47:36 [INFO] [TRAIN] epoch: 26, iter: 6400/120000, loss: 0.9635, lr: 0.009519, batch_cost: 0.2044, reader_cost: 0.00108, ips: 14.6773 samples/sec | ETA 06:26:59
- 2022-04-13 11:47:49 [INFO] [TRAIN] epoch: 27, iter: 6450/120000, loss: 1.0030, lr: 0.009515, batch_cost: 0.2549, reader_cost: 0.05691, ips: 11.7682 samples/sec | ETA 08:02:26
- 2022-04-13 11:48:00 [INFO] [TRAIN] epoch: 27, iter: 6500/120000, loss: 0.9702, lr: 0.009511, batch_cost: 0.2119, reader_cost: 0.00104, ips: 14.1550 samples/sec | ETA 06:40:55
- 2022-04-13 11:48:10 [INFO] [TRAIN] epoch: 27, iter: 6550/120000, loss: 0.9676, lr: 0.009507, batch_cost: 0.2061, reader_cost: 0.00103, ips: 14.5563 samples/sec | ETA 06:29:41
- 2022-04-13 11:48:20 [INFO] [TRAIN] epoch: 27, iter: 6600/120000, loss: 1.0316, lr: 0.009504, batch_cost: 0.2010, reader_cost: 0.00103, ips: 14.9271 samples/sec | ETA 06:19:50
- 2022-04-13 11:48:30 [INFO] [TRAIN] epoch: 27, iter: 6650/120000, loss: 1.0187, lr: 0.009500, batch_cost: 0.2108, reader_cost: 0.00054, ips: 14.2335 samples/sec | ETA 06:38:10
- 2022-04-13 11:48:43 [INFO] [TRAIN] epoch: 28, iter: 6700/120000, loss: 1.0024, lr: 0.009496, batch_cost: 0.2500, reader_cost: 0.05051, ips: 11.9980 samples/sec | ETA 07:52:09
- 2022-04-13 11:48:53 [INFO] [TRAIN] epoch: 28, iter: 6750/120000, loss: 0.9723, lr: 0.009492, batch_cost: 0.2076, reader_cost: 0.00082, ips: 14.4514 samples/sec | ETA 06:31:49
- 2022-04-13 11:49:04 [INFO] [TRAIN] epoch: 28, iter: 6800/120000, loss: 0.9940, lr: 0.009489, batch_cost: 0.2230, reader_cost: 0.00104, ips: 13.4548 samples/sec | ETA 07:00:40
- 2022-04-13 11:49:15 [INFO] [TRAIN] epoch: 28, iter: 6850/120000, loss: 0.9880, lr: 0.009485, batch_cost: 0.2081, reader_cost: 0.00141, ips: 14.4155 samples/sec | ETA 06:32:27
- 2022-04-13 11:49:26 [INFO] [TRAIN] epoch: 28, iter: 6900/120000, loss: 0.9856, lr: 0.009481, batch_cost: 0.2269, reader_cost: 0.00089, ips: 13.2193 samples/sec | ETA 07:07:47
- 2022-04-13 11:49:39 [INFO] [TRAIN] epoch: 29, iter: 6950/120000, loss: 0.9841, lr: 0.009477, batch_cost: 0.2636, reader_cost: 0.05488, ips: 11.3805 samples/sec | ETA 08:16:41
- 2022-04-13 11:49:50 [INFO] [TRAIN] epoch: 29, iter: 7000/120000, loss: 0.9970, lr: 0.009474, batch_cost: 0.2048, reader_cost: 0.00073, ips: 14.6507 samples/sec | ETA 06:25:38
- 2022-04-13 11:50:00 [INFO] [TRAIN] epoch: 29, iter: 7050/120000, loss: 0.9970, lr: 0.009470, batch_cost: 0.2033, reader_cost: 0.00076, ips: 14.7563 samples/sec | ETA 06:22:43
- 2022-04-13 11:50:11 [INFO] [TRAIN] epoch: 29, iter: 7100/120000, loss: 0.9898, lr: 0.009466, batch_cost: 0.2167, reader_cost: 0.00090, ips: 13.8432 samples/sec | ETA 06:47:46
- 2022-04-13 11:50:21 [INFO] [TRAIN] epoch: 29, iter: 7150/120000, loss: 0.9601, lr: 0.009462, batch_cost: 0.2085, reader_cost: 0.00103, ips: 14.3901 samples/sec | ETA 06:32:06
- 2022-04-13 11:50:34 [INFO] [TRAIN] epoch: 30, iter: 7200/120000, loss: 0.9453, lr: 0.009458, batch_cost: 0.2618, reader_cost: 0.05368, ips: 11.4571 samples/sec | ETA 08:12:16
- 2022-04-13 11:50:45 [INFO] [TRAIN] epoch: 30, iter: 7250/120000, loss: 0.9733, lr: 0.009455, batch_cost: 0.2189, reader_cost: 0.00128, ips: 13.7066 samples/sec | ETA 06:51:17
- 2022-04-13 11:50:56 [INFO] [TRAIN] epoch: 30, iter: 7300/120000, loss: 0.9996, lr: 0.009451, batch_cost: 0.2097, reader_cost: 0.00069, ips: 14.3081 samples/sec | ETA 06:33:50
- 2022-04-13 11:51:06 [INFO] [TRAIN] epoch: 30, iter: 7350/120000, loss: 1.0446, lr: 0.009447, batch_cost: 0.2079, reader_cost: 0.00070, ips: 14.4304 samples/sec | ETA 06:30:19
- 2022-04-13 11:51:17 [INFO] [TRAIN] epoch: 30, iter: 7400/120000, loss: 0.9337, lr: 0.009443, batch_cost: 0.2214, reader_cost: 0.00419, ips: 13.5511 samples/sec | ETA 06:55:27
- 2022-04-13 11:51:30 [INFO] [TRAIN] epoch: 31, iter: 7450/120000, loss: 0.9842, lr: 0.009440, batch_cost: 0.2606, reader_cost: 0.05575, ips: 11.5121 samples/sec | ETA 08:08:50
- 2022-04-13 11:51:40 [INFO] [TRAIN] epoch: 31, iter: 7500/120000, loss: 0.9868, lr: 0.009436, batch_cost: 0.2069, reader_cost: 0.00236, ips: 14.5014 samples/sec | ETA 06:27:53
- 2022-04-13 11:51:50 [INFO] [TRAIN] epoch: 31, iter: 7550/120000, loss: 1.0282, lr: 0.009432, batch_cost: 0.1980, reader_cost: 0.00054, ips: 15.1549 samples/sec | ETA 06:11:00
- 2022-04-13 11:52:01 [INFO] [TRAIN] epoch: 31, iter: 7600/120000, loss: 0.9442, lr: 0.009428, batch_cost: 0.2048, reader_cost: 0.00071, ips: 14.6491 samples/sec | ETA 06:23:38
- 2022-04-13 11:52:11 [INFO] [TRAIN] epoch: 31, iter: 7650/120000, loss: 0.9681, lr: 0.009424, batch_cost: 0.2025, reader_cost: 0.00069, ips: 14.8158 samples/sec | ETA 06:19:09
- 2022-04-13 11:52:24 [INFO] [TRAIN] epoch: 32, iter: 7700/120000, loss: 0.9617, lr: 0.009421, batch_cost: 0.2641, reader_cost: 0.06385, ips: 11.3604 samples/sec | ETA 08:14:15
- 2022-04-13 11:52:35 [INFO] [TRAIN] epoch: 32, iter: 7750/120000, loss: 0.9398, lr: 0.009417, batch_cost: 0.2239, reader_cost: 0.00076, ips: 13.4011 samples/sec | ETA 06:58:48
- 2022-04-13 11:52:46 [INFO] [TRAIN] epoch: 32, iter: 7800/120000, loss: 1.0001, lr: 0.009413, batch_cost: 0.2150, reader_cost: 0.00058, ips: 13.9506 samples/sec | ETA 06:42:07
- 2022-04-13 11:52:56 [INFO] [TRAIN] epoch: 32, iter: 7850/120000, loss: 0.9674, lr: 0.009409, batch_cost: 0.1963, reader_cost: 0.00286, ips: 15.2804 samples/sec | ETA 06:06:58
- 2022-04-13 11:53:07 [INFO] [TRAIN] epoch: 32, iter: 7900/120000, loss: 0.9753, lr: 0.009406, batch_cost: 0.2158, reader_cost: 0.00084, ips: 13.9016 samples/sec | ETA 06:43:11
- 2022-04-13 11:53:20 [INFO] [TRAIN] epoch: 33, iter: 7950/120000, loss: 0.9576, lr: 0.009402, batch_cost: 0.2673, reader_cost: 0.06140, ips: 11.2220 samples/sec | ETA 08:19:14
- 2022-04-13 11:53:31 [INFO] [TRAIN] epoch: 33, iter: 8000/120000, loss: 0.9796, lr: 0.009398, batch_cost: 0.2132, reader_cost: 0.00129, ips: 14.0724 samples/sec | ETA 06:37:56
- 2022-04-13 11:53:31 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1962 - reader cost: 0.1532
- 2022-04-13 11:53:55 [INFO] [EVAL] #Images: 500 mIoU: 0.6052 Acc: 0.9141 Kappa: 0.8891 Dice: 0.7317
- 2022-04-13 11:53:55 [INFO] [EVAL] Class IoU:
- [0.9292 0.5881 0.8606 0.1693 0.4478 0.3892 0.5549 0.6773 0.8891 0.4514
- 0.91 0.7235 0.461 0.8878 0.4164 0.5528 0.4064 0.4963 0.6885]
- 2022-04-13 11:53:55 [INFO] [EVAL] Class Precision:
- [0.9953 0.6374 0.8872 0.8529 0.5702 0.8508 0.8008 0.9071 0.9349 0.8631
- 0.9632 0.8557 0.7719 0.9515 0.4521 0.9324 0.7952 0.621 0.794 ]
- 2022-04-13 11:53:55 [INFO] [EVAL] Class Recall:
- [0.9333 0.8838 0.9663 0.1744 0.6761 0.4177 0.6438 0.7278 0.9477 0.4862
- 0.9427 0.824 0.5337 0.9299 0.8406 0.5759 0.4539 0.7121 0.8383]
- 2022-04-13 11:53:56 [INFO] [EVAL] The model with the best validation mIoU (0.6525) was saved at iter 4000.
- 2022-04-13 11:54:06 [INFO] [TRAIN] epoch: 33, iter: 8050/120000, loss: 1.0064, lr: 0.009394, batch_cost: 0.2087, reader_cost: 0.00156, ips: 14.3755 samples/sec | ETA 06:29:22
- 2022-04-13 11:54:17 [INFO] [TRAIN] epoch: 33, iter: 8100/120000, loss: 0.9995, lr: 0.009390, batch_cost: 0.2100, reader_cost: 0.00108, ips: 14.2867 samples/sec | ETA 06:31:37
- 2022-04-13 11:54:27 [INFO] [TRAIN] epoch: 33, iter: 8150/120000, loss: 0.9699, lr: 0.009387, batch_cost: 0.1973, reader_cost: 0.00082, ips: 15.2058 samples/sec | ETA 06:07:47
- 2022-04-13 11:54:40 [INFO] [TRAIN] epoch: 34, iter: 8200/120000, loss: 0.9648, lr: 0.009383, batch_cost: 0.2681, reader_cost: 0.06343, ips: 11.1912 samples/sec | ETA 08:19:29
- 2022-04-13 11:54:50 [INFO] [TRAIN] epoch: 34, iter: 8250/120000, loss: 0.9745, lr: 0.009379, batch_cost: 0.2094, reader_cost: 0.00205, ips: 14.3242 samples/sec | ETA 06:30:04
- 2022-04-13 11:55:00 [INFO] [TRAIN] epoch: 34, iter: 8300/120000, loss: 0.9823, lr: 0.009375, batch_cost: 0.1969, reader_cost: 0.00088, ips: 15.2349 samples/sec | ETA 06:06:35
- 2022-04-13 11:55:11 [INFO] [TRAIN] epoch: 34, iter: 8350/120000, loss: 0.9970, lr: 0.009372, batch_cost: 0.2241, reader_cost: 0.00076, ips: 13.3861 samples/sec | ETA 06:57:02
- 2022-04-13 11:55:22 [INFO] [TRAIN] epoch: 34, iter: 8400/120000, loss: 0.9664, lr: 0.009368, batch_cost: 0.2160, reader_cost: 0.00108, ips: 13.8913 samples/sec | ETA 06:41:41
- 2022-04-13 11:55:36 [INFO] [TRAIN] epoch: 35, iter: 8450/120000, loss: 0.9809, lr: 0.009364, batch_cost: 0.2699, reader_cost: 0.06507, ips: 11.1133 samples/sec | ETA 08:21:52
- 2022-04-13 11:55:46 [INFO] [TRAIN] epoch: 35, iter: 8500/120000, loss: 0.9478, lr: 0.009360, batch_cost: 0.1981, reader_cost: 0.00084, ips: 15.1431 samples/sec | ETA 06:08:09
- 2022-04-13 11:55:56 [INFO] [TRAIN] epoch: 35, iter: 8550/120000, loss: 0.9199, lr: 0.009356, batch_cost: 0.2046, reader_cost: 0.00166, ips: 14.6655 samples/sec | ETA 06:19:58
- 2022-04-13 11:56:07 [INFO] [TRAIN] epoch: 35, iter: 8600/120000, loss: 1.0528, lr: 0.009353, batch_cost: 0.2133, reader_cost: 0.00087, ips: 14.0638 samples/sec | ETA 06:36:03
- 2022-04-13 11:56:17 [INFO] [TRAIN] epoch: 35, iter: 8650/120000, loss: 1.0156, lr: 0.009349, batch_cost: 0.2099, reader_cost: 0.00151, ips: 14.2901 samples/sec | ETA 06:29:36
- 2022-04-13 11:56:32 [INFO] [TRAIN] epoch: 36, iter: 8700/120000, loss: 0.9487, lr: 0.009345, batch_cost: 0.2921, reader_cost: 0.05812, ips: 10.2716 samples/sec | ETA 09:01:46
- 2022-04-13 11:56:42 [INFO] [TRAIN] epoch: 36, iter: 8750/120000, loss: 0.9502, lr: 0.009341, batch_cost: 0.2128, reader_cost: 0.00176, ips: 14.0954 samples/sec | ETA 06:34:37
- 2022-04-13 11:56:52 [INFO] [TRAIN] epoch: 36, iter: 8800/120000, loss: 0.9573, lr: 0.009338, batch_cost: 0.1960, reader_cost: 0.00179, ips: 15.3098 samples/sec | ETA 06:03:09
- 2022-04-13 11:57:02 [INFO] [TRAIN] epoch: 36, iter: 8850/120000, loss: 0.9991, lr: 0.009334, batch_cost: 0.2066, reader_cost: 0.00066, ips: 14.5205 samples/sec | ETA 06:22:44
- 2022-04-13 11:57:14 [INFO] [TRAIN] epoch: 36, iter: 8900/120000, loss: 1.0388, lr: 0.009330, batch_cost: 0.2253, reader_cost: 0.00070, ips: 13.3169 samples/sec | ETA 06:57:08
- 2022-04-13 11:57:27 [INFO] [TRAIN] epoch: 37, iter: 8950/120000, loss: 0.9958, lr: 0.009326, batch_cost: 0.2693, reader_cost: 0.05375, ips: 11.1408 samples/sec | ETA 08:18:23
- 2022-04-13 11:57:38 [INFO] [TRAIN] epoch: 37, iter: 9000/120000, loss: 0.9695, lr: 0.009322, batch_cost: 0.2207, reader_cost: 0.00085, ips: 13.5906 samples/sec | ETA 06:48:22
- 2022-04-13 11:57:48 [INFO] [TRAIN] epoch: 37, iter: 9050/120000, loss: 0.9527, lr: 0.009319, batch_cost: 0.2039, reader_cost: 0.00104, ips: 14.7153 samples/sec | ETA 06:16:59
- 2022-04-13 11:57:59 [INFO] [TRAIN] epoch: 37, iter: 9100/120000, loss: 0.9769, lr: 0.009315, batch_cost: 0.2191, reader_cost: 0.00089, ips: 13.6903 samples/sec | ETA 06:45:01
- 2022-04-13 11:58:09 [INFO] [TRAIN] epoch: 37, iter: 9150/120000, loss: 0.9894, lr: 0.009311, batch_cost: 0.2009, reader_cost: 0.00187, ips: 14.9329 samples/sec | ETA 06:11:09
- 2022-04-13 11:58:22 [INFO] [TRAIN] epoch: 38, iter: 9200/120000, loss: 1.0282, lr: 0.009307, batch_cost: 0.2590, reader_cost: 0.05408, ips: 11.5840 samples/sec | ETA 07:58:14
- 2022-04-13 11:58:33 [INFO] [TRAIN] epoch: 38, iter: 9250/120000, loss: 0.9409, lr: 0.009304, batch_cost: 0.2144, reader_cost: 0.00152, ips: 13.9900 samples/sec | ETA 06:35:49
- 2022-04-13 11:58:43 [INFO] [TRAIN] epoch: 38, iter: 9300/120000, loss: 0.9374, lr: 0.009300, batch_cost: 0.2071, reader_cost: 0.00136, ips: 14.4874 samples/sec | ETA 06:22:03
- 2022-04-13 11:58:54 [INFO] [TRAIN] epoch: 38, iter: 9350/120000, loss: 0.9629, lr: 0.009296, batch_cost: 0.2173, reader_cost: 0.00089, ips: 13.8073 samples/sec | ETA 06:40:41
- 2022-04-13 11:59:04 [INFO] [TRAIN] epoch: 38, iter: 9400/120000, loss: 0.9647, lr: 0.009292, batch_cost: 0.1951, reader_cost: 0.00055, ips: 15.3774 samples/sec | ETA 05:59:37
- 2022-04-13 11:59:18 [INFO] [TRAIN] epoch: 39, iter: 9450/120000, loss: 0.9747, lr: 0.009288, batch_cost: 0.2767, reader_cost: 0.06400, ips: 10.8417 samples/sec | ETA 08:29:50
- 2022-04-13 11:59:28 [INFO] [TRAIN] epoch: 39, iter: 9500/120000, loss: 0.9499, lr: 0.009285, batch_cost: 0.2058, reader_cost: 0.00067, ips: 14.5753 samples/sec | ETA 06:19:04
- 2022-04-13 11:59:40 [INFO] [TRAIN] epoch: 39, iter: 9550/120000, loss: 0.9356, lr: 0.009281, batch_cost: 0.2342, reader_cost: 0.00112, ips: 12.8106 samples/sec | ETA 07:11:05
- 2022-04-13 11:59:50 [INFO] [TRAIN] epoch: 39, iter: 9600/120000, loss: 0.9893, lr: 0.009277, batch_cost: 0.2031, reader_cost: 0.00102, ips: 14.7736 samples/sec | ETA 06:13:38
- 2022-04-13 12:00:01 [INFO] [TRAIN] epoch: 39, iter: 9650/120000, loss: 0.9575, lr: 0.009273, batch_cost: 0.2081, reader_cost: 0.00089, ips: 14.4160 samples/sec | ETA 06:22:44
- 2022-04-13 12:00:14 [INFO] [TRAIN] epoch: 40, iter: 9700/120000, loss: 0.9738, lr: 0.009270, batch_cost: 0.2688, reader_cost: 0.05333, ips: 11.1621 samples/sec | ETA 08:14:05
- 2022-04-13 12:00:24 [INFO] [TRAIN] epoch: 40, iter: 9750/120000, loss: 0.9794, lr: 0.009266, batch_cost: 0.2011, reader_cost: 0.00094, ips: 14.9206 samples/sec | ETA 06:09:27
- 2022-04-13 12:00:34 [INFO] [TRAIN] epoch: 40, iter: 9800/120000, loss: 0.9412, lr: 0.009262, batch_cost: 0.1968, reader_cost: 0.00075, ips: 15.2411 samples/sec | ETA 06:01:31
- 2022-04-13 12:00:44 [INFO] [TRAIN] epoch: 40, iter: 9850/120000, loss: 0.9317, lr: 0.009258, batch_cost: 0.2094, reader_cost: 0.00150, ips: 14.3273 samples/sec | ETA 06:24:24
- 2022-04-13 12:00:56 [INFO] [TRAIN] epoch: 40, iter: 9900/120000, loss: 0.9547, lr: 0.009254, batch_cost: 0.2237, reader_cost: 0.00075, ips: 13.4114 samples/sec | ETA 06:50:28
- 2022-04-13 12:01:09 [INFO] [TRAIN] epoch: 41, iter: 9950/120000, loss: 0.9503, lr: 0.009251, batch_cost: 0.2648, reader_cost: 0.05137, ips: 11.3293 samples/sec | ETA 08:05:41
- 2022-04-13 12:01:19 [INFO] [TRAIN] epoch: 41, iter: 10000/120000, loss: 0.9468, lr: 0.009247, batch_cost: 0.2018, reader_cost: 0.00117, ips: 14.8644 samples/sec | ETA 06:10:00
- 2022-04-13 12:01:29 [INFO] [TRAIN] epoch: 41, iter: 10050/120000, loss: 0.9912, lr: 0.009243, batch_cost: 0.2042, reader_cost: 0.00090, ips: 14.6888 samples/sec | ETA 06:14:15
- 2022-04-13 12:01:40 [INFO] [TRAIN] epoch: 41, iter: 10100/120000, loss: 0.9794, lr: 0.009239, batch_cost: 0.2200, reader_cost: 0.00067, ips: 13.6379 samples/sec | ETA 06:42:55
- 2022-04-13 12:01:50 [INFO] [TRAIN] epoch: 41, iter: 10150/120000, loss: 0.9610, lr: 0.009236, batch_cost: 0.1986, reader_cost: 0.00124, ips: 15.1040 samples/sec | ETA 06:03:38
- 2022-04-13 12:02:04 [INFO] [TRAIN] epoch: 42, iter: 10200/120000, loss: 0.9515, lr: 0.009232, batch_cost: 0.2706, reader_cost: 0.05461, ips: 11.0870 samples/sec | ETA 08:15:10
- 2022-04-13 12:02:14 [INFO] [TRAIN] epoch: 42, iter: 10250/120000, loss: 0.9373, lr: 0.009228, batch_cost: 0.2141, reader_cost: 0.00097, ips: 14.0103 samples/sec | ETA 06:31:40
- 2022-04-13 12:02:26 [INFO] [TRAIN] epoch: 42, iter: 10300/120000, loss: 0.9442, lr: 0.009224, batch_cost: 0.2294, reader_cost: 0.00083, ips: 13.0798 samples/sec | ETA 06:59:20
- 2022-04-13 12:02:36 [INFO] [TRAIN] epoch: 42, iter: 10350/120000, loss: 0.9290, lr: 0.009220, batch_cost: 0.1979, reader_cost: 0.00093, ips: 15.1583 samples/sec | ETA 06:01:41
- 2022-04-13 12:02:46 [INFO] [TRAIN] epoch: 42, iter: 10400/120000, loss: 0.9558, lr: 0.009217, batch_cost: 0.2035, reader_cost: 0.00138, ips: 14.7432 samples/sec | ETA 06:11:41
- 2022-04-13 12:02:59 [INFO] [TRAIN] epoch: 43, iter: 10450/120000, loss: 0.9767, lr: 0.009213, batch_cost: 0.2707, reader_cost: 0.06182, ips: 11.0808 samples/sec | ETA 08:14:19
- 2022-04-13 12:03:09 [INFO] [TRAIN] epoch: 43, iter: 10500/120000, loss: 0.9646, lr: 0.009209, batch_cost: 0.2018, reader_cost: 0.00154, ips: 14.8640 samples/sec | ETA 06:08:20
- 2022-04-13 12:03:21 [INFO] [TRAIN] epoch: 43, iter: 10550/120000, loss: 0.9782, lr: 0.009205, batch_cost: 0.2237, reader_cost: 0.00269, ips: 13.4132 samples/sec | ETA 06:47:59
- 2022-04-13 12:03:31 [INFO] [TRAIN] epoch: 43, iter: 10600/120000, loss: 0.9265, lr: 0.009201, batch_cost: 0.2173, reader_cost: 0.00068, ips: 13.8075 samples/sec | ETA 06:36:09
- 2022-04-13 12:03:42 [INFO] [TRAIN] epoch: 43, iter: 10650/120000, loss: 1.0360, lr: 0.009198, batch_cost: 0.2106, reader_cost: 0.00063, ips: 14.2449 samples/sec | ETA 06:23:49
- 2022-04-13 12:03:56 [INFO] [TRAIN] epoch: 44, iter: 10700/120000, loss: 0.9336, lr: 0.009194, batch_cost: 0.2741, reader_cost: 0.05952, ips: 10.9449 samples/sec | ETA 08:19:19
- 2022-04-13 12:04:06 [INFO] [TRAIN] epoch: 44, iter: 10750/120000, loss: 1.0138, lr: 0.009190, batch_cost: 0.1991, reader_cost: 0.00072, ips: 15.0650 samples/sec | ETA 06:02:35
- 2022-04-13 12:04:19 [INFO] [TRAIN] epoch: 44, iter: 10800/120000, loss: 0.9833, lr: 0.009186, batch_cost: 0.2762, reader_cost: 0.00103, ips: 10.8625 samples/sec | ETA 08:22:38
- 2022-04-13 12:04:31 [INFO] [TRAIN] epoch: 44, iter: 10850/120000, loss: 0.9849, lr: 0.009183, batch_cost: 0.2217, reader_cost: 0.00092, ips: 13.5323 samples/sec | ETA 06:43:17
- 2022-04-13 12:04:42 [INFO] [TRAIN] epoch: 44, iter: 10900/120000, loss: 0.9718, lr: 0.009179, batch_cost: 0.2236, reader_cost: 0.00082, ips: 13.4171 samples/sec | ETA 06:46:34
- 2022-04-13 12:04:56 [INFO] [TRAIN] epoch: 45, iter: 10950/120000, loss: 0.9947, lr: 0.009175, batch_cost: 0.2835, reader_cost: 0.05236, ips: 10.5830 samples/sec | ETA 08:35:12
- 2022-04-13 12:05:06 [INFO] [TRAIN] epoch: 45, iter: 11000/120000, loss: 0.9728, lr: 0.009171, batch_cost: 0.2068, reader_cost: 0.00085, ips: 14.5048 samples/sec | ETA 06:15:44
- 2022-04-13 12:05:17 [INFO] [TRAIN] epoch: 45, iter: 11050/120000, loss: 0.9500, lr: 0.009167, batch_cost: 0.2060, reader_cost: 0.00100, ips: 14.5656 samples/sec | ETA 06:13:59
- 2022-04-13 12:05:26 [INFO] [TRAIN] epoch: 45, iter: 11100/120000, loss: 0.9868, lr: 0.009164, batch_cost: 0.1957, reader_cost: 0.00059, ips: 15.3306 samples/sec | ETA 05:55:10
- 2022-04-13 12:05:36 [INFO] [TRAIN] epoch: 45, iter: 11150/120000, loss: 0.9633, lr: 0.009160, batch_cost: 0.1984, reader_cost: 0.00078, ips: 15.1219 samples/sec | ETA 05:59:54
- 2022-04-13 12:05:50 [INFO] [TRAIN] epoch: 46, iter: 11200/120000, loss: 0.9737, lr: 0.009156, batch_cost: 0.2783, reader_cost: 0.04913, ips: 10.7790 samples/sec | ETA 08:24:41
- 2022-04-13 12:06:02 [INFO] [TRAIN] epoch: 46, iter: 11250/120000, loss: 0.9262, lr: 0.009152, batch_cost: 0.2342, reader_cost: 0.00063, ips: 12.8095 samples/sec | ETA 07:04:29
- 2022-04-13 12:06:12 [INFO] [TRAIN] epoch: 46, iter: 11300/120000, loss: 0.9417, lr: 0.009148, batch_cost: 0.2059, reader_cost: 0.00102, ips: 14.5705 samples/sec | ETA 06:13:00
- 2022-04-13 12:06:24 [INFO] [TRAIN] epoch: 46, iter: 11350/120000, loss: 0.9427, lr: 0.009145, batch_cost: 0.2354, reader_cost: 0.00061, ips: 12.7468 samples/sec | ETA 07:06:11
- 2022-04-13 12:06:34 [INFO] [TRAIN] epoch: 46, iter: 11400/120000, loss: 0.9780, lr: 0.009141, batch_cost: 0.2023, reader_cost: 0.00111, ips: 14.8293 samples/sec | ETA 06:06:10
- 2022-04-13 12:06:48 [INFO] [TRAIN] epoch: 47, iter: 11450/120000, loss: 0.9784, lr: 0.009137, batch_cost: 0.2841, reader_cost: 0.05069, ips: 10.5589 samples/sec | ETA 08:34:01
- 2022-04-13 12:06:58 [INFO] [TRAIN] epoch: 47, iter: 11500/120000, loss: 0.9609, lr: 0.009133, batch_cost: 0.2027, reader_cost: 0.00075, ips: 14.8035 samples/sec | ETA 06:06:28
- 2022-04-13 12:07:08 [INFO] [TRAIN] epoch: 47, iter: 11550/120000, loss: 0.9323, lr: 0.009130, batch_cost: 0.1985, reader_cost: 0.00093, ips: 15.1150 samples/sec | ETA 05:58:45
- 2022-04-13 12:07:19 [INFO] [TRAIN] epoch: 47, iter: 11600/120000, loss: 0.9510, lr: 0.009126, batch_cost: 0.2082, reader_cost: 0.00079, ips: 14.4085 samples/sec | ETA 06:16:09
- 2022-04-13 12:07:29 [INFO] [TRAIN] epoch: 47, iter: 11650/120000, loss: 0.9921, lr: 0.009122, batch_cost: 0.2013, reader_cost: 0.00099, ips: 14.9032 samples/sec | ETA 06:03:30
- 2022-04-13 12:07:43 [INFO] [TRAIN] epoch: 48, iter: 11700/120000, loss: 0.9601, lr: 0.009118, batch_cost: 0.2818, reader_cost: 0.05573, ips: 10.6453 samples/sec | ETA 08:28:40
- 2022-04-13 12:07:53 [INFO] [TRAIN] epoch: 48, iter: 11750/120000, loss: 0.9881, lr: 0.009114, batch_cost: 0.2039, reader_cost: 0.00135, ips: 14.7159 samples/sec | ETA 06:07:47
- 2022-04-13 12:08:04 [INFO] [TRAIN] epoch: 48, iter: 11800/120000, loss: 0.9594, lr: 0.009111, batch_cost: 0.2149, reader_cost: 0.00044, ips: 13.9622 samples/sec | ETA 06:27:28
- 2022-04-13 12:08:14 [INFO] [TRAIN] epoch: 48, iter: 11850/120000, loss: 0.9921, lr: 0.009107, batch_cost: 0.2052, reader_cost: 0.00159, ips: 14.6188 samples/sec | ETA 06:09:54
- 2022-04-13 12:08:25 [INFO] [TRAIN] epoch: 48, iter: 11900/120000, loss: 0.9535, lr: 0.009103, batch_cost: 0.2271, reader_cost: 0.00128, ips: 13.2102 samples/sec | ETA 06:49:09
- 2022-04-13 12:08:39 [INFO] [TRAIN] epoch: 49, iter: 11950/120000, loss: 0.9489, lr: 0.009099, batch_cost: 0.2696, reader_cost: 0.05321, ips: 11.1273 samples/sec | ETA 08:05:30
- 2022-04-13 12:08:49 [INFO] [TRAIN] epoch: 49, iter: 12000/120000, loss: 0.9335, lr: 0.009095, batch_cost: 0.2063, reader_cost: 0.00129, ips: 14.5392 samples/sec | ETA 06:11:24
- 2022-04-13 12:08:49 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1922 - reader cost: 0.1261
- 2022-04-13 12:09:13 [INFO] [EVAL] #Images: 500 mIoU: 0.7155 Acc: 0.9488 Kappa: 0.9334 Dice: 0.8245
- 2022-04-13 12:09:13 [INFO] [EVAL] Class IoU:
- [0.9692 0.775 0.9086 0.461 0.5135 0.5834 0.6114 0.7325 0.9122 0.5841
- 0.9403 0.7832 0.5377 0.9304 0.6786 0.7239 0.6788 0.5415 0.7294]
- 2022-04-13 12:09:13 [INFO] [EVAL] Class Precision:
- [0.9778 0.9081 0.9489 0.8024 0.7756 0.7607 0.8706 0.8955 0.943 0.7668
- 0.9598 0.8515 0.785 0.9555 0.8942 0.9736 0.9168 0.6588 0.8584]
- 2022-04-13 12:09:13 [INFO] [EVAL] Class Recall:
- [0.991 0.8409 0.9553 0.5201 0.6031 0.7144 0.6725 0.801 0.9654 0.7102
- 0.9788 0.9071 0.6306 0.9726 0.7379 0.7384 0.7233 0.7526 0.8292]
- 2022-04-13 12:09:14 [INFO] [EVAL] The model with the best validation mIoU (0.7155) was saved at iter 12000.
- 2022-04-13 12:09:24 [INFO] [TRAIN] epoch: 49, iter: 12050/120000, loss: 0.9503, lr: 0.009092, batch_cost: 0.2021, reader_cost: 0.00101, ips: 14.8465 samples/sec | ETA 06:03:33
- 2022-04-13 12:09:34 [INFO] [TRAIN] epoch: 49, iter: 12100/120000, loss: 0.9347, lr: 0.009088, batch_cost: 0.2015, reader_cost: 0.00076, ips: 14.8898 samples/sec | ETA 06:02:19
- 2022-04-13 12:09:45 [INFO] [TRAIN] epoch: 49, iter: 12150/120000, loss: 0.9794, lr: 0.009084, batch_cost: 0.2130, reader_cost: 0.00050, ips: 14.0840 samples/sec | ETA 06:22:52
- 2022-04-13 12:09:59 [INFO] [TRAIN] epoch: 50, iter: 12200/120000, loss: 0.9335, lr: 0.009080, batch_cost: 0.2758, reader_cost: 0.05457, ips: 10.8789 samples/sec | ETA 08:15:27
- 2022-04-13 12:10:09 [INFO] [TRAIN] epoch: 50, iter: 12250/120000, loss: 0.9342, lr: 0.009076, batch_cost: 0.2072, reader_cost: 0.00117, ips: 14.4793 samples/sec | ETA 06:12:05
- 2022-04-13 12:10:20 [INFO] [TRAIN] epoch: 50, iter: 12300/120000, loss: 0.9371, lr: 0.009073, batch_cost: 0.2172, reader_cost: 0.00067, ips: 13.8095 samples/sec | ETA 06:29:56
- 2022-04-13 12:10:32 [INFO] [TRAIN] epoch: 50, iter: 12350/120000, loss: 0.9684, lr: 0.009069, batch_cost: 0.2300, reader_cost: 0.00112, ips: 13.0462 samples/sec | ETA 06:52:34
- 2022-04-13 12:10:41 [INFO] [TRAIN] epoch: 50, iter: 12400/120000, loss: 0.9358, lr: 0.009065, batch_cost: 0.1817, reader_cost: 0.00091, ips: 16.5111 samples/sec | ETA 05:25:50
- 2022-04-13 12:10:55 [INFO] [TRAIN] epoch: 51, iter: 12450/120000, loss: 0.9366, lr: 0.009061, batch_cost: 0.2752, reader_cost: 0.04831, ips: 10.9024 samples/sec | ETA 08:13:14
- 2022-04-13 12:11:05 [INFO] [TRAIN] epoch: 51, iter: 12500/120000, loss: 0.9748, lr: 0.009057, batch_cost: 0.2050, reader_cost: 0.00116, ips: 14.6319 samples/sec | ETA 06:07:20
- 2022-04-13 12:11:15 [INFO] [TRAIN] epoch: 51, iter: 12550/120000, loss: 0.9552, lr: 0.009054, batch_cost: 0.2015, reader_cost: 0.00091, ips: 14.8858 samples/sec | ETA 06:00:54
- 2022-04-13 12:11:25 [INFO] [TRAIN] epoch: 51, iter: 12600/120000, loss: 0.9314, lr: 0.009050, batch_cost: 0.2112, reader_cost: 0.00079, ips: 14.2075 samples/sec | ETA 06:17:58
- 2022-04-13 12:11:38 [INFO] [TRAIN] epoch: 52, iter: 12650/120000, loss: 0.9800, lr: 0.009046, batch_cost: 0.2512, reader_cost: 0.05426, ips: 11.9414 samples/sec | ETA 07:29:29
- 2022-04-13 12:11:49 [INFO] [TRAIN] epoch: 52, iter: 12700/120000, loss: 0.9549, lr: 0.009042, batch_cost: 0.2159, reader_cost: 0.00155, ips: 13.8941 samples/sec | ETA 06:26:08
- 2022-04-13 12:11:59 [INFO] [TRAIN] epoch: 52, iter: 12750/120000, loss: 0.9669, lr: 0.009039, batch_cost: 0.2004, reader_cost: 0.00103, ips: 14.9698 samples/sec | ETA 05:58:13
- 2022-04-13 12:12:10 [INFO] [TRAIN] epoch: 52, iter: 12800/120000, loss: 0.9296, lr: 0.009035, batch_cost: 0.2246, reader_cost: 0.00090, ips: 13.3549 samples/sec | ETA 06:41:21
- 2022-04-13 12:12:20 [INFO] [TRAIN] epoch: 52, iter: 12850/120000, loss: 0.9639, lr: 0.009031, batch_cost: 0.2027, reader_cost: 0.00100, ips: 14.7990 samples/sec | ETA 06:02:01
- 2022-04-13 12:12:34 [INFO] [TRAIN] epoch: 53, iter: 12900/120000, loss: 0.9827, lr: 0.009027, batch_cost: 0.2721, reader_cost: 0.05623, ips: 11.0240 samples/sec | ETA 08:05:45
- 2022-04-13 12:12:45 [INFO] [TRAIN] epoch: 53, iter: 12950/120000, loss: 0.9578, lr: 0.009023, batch_cost: 0.2186, reader_cost: 0.00180, ips: 13.7247 samples/sec | ETA 06:29:59
- 2022-04-13 12:12:55 [INFO] [TRAIN] epoch: 53, iter: 13000/120000, loss: 0.9330, lr: 0.009020, batch_cost: 0.2033, reader_cost: 0.00129, ips: 14.7556 samples/sec | ETA 06:02:34
- 2022-04-13 12:13:05 [INFO] [TRAIN] epoch: 53, iter: 13050/120000, loss: 0.9439, lr: 0.009016, batch_cost: 0.2047, reader_cost: 0.00145, ips: 14.6586 samples/sec | ETA 06:04:48
- 2022-04-13 12:13:15 [INFO] [TRAIN] epoch: 53, iter: 13100/120000, loss: 0.9879, lr: 0.009012, batch_cost: 0.2053, reader_cost: 0.00057, ips: 14.6100 samples/sec | ETA 06:05:50
- 2022-04-13 12:13:29 [INFO] [TRAIN] epoch: 54, iter: 13150/120000, loss: 0.9333, lr: 0.009008, batch_cost: 0.2799, reader_cost: 0.05832, ips: 10.7179 samples/sec | ETA 08:18:27
- 2022-04-13 12:13:41 [INFO] [TRAIN] epoch: 54, iter: 13200/120000, loss: 0.9831, lr: 0.009004, batch_cost: 0.2337, reader_cost: 0.00097, ips: 12.8360 samples/sec | ETA 06:56:01
- 2022-04-13 12:13:51 [INFO] [TRAIN] epoch: 54, iter: 13250/120000, loss: 0.9886, lr: 0.009001, batch_cost: 0.2081, reader_cost: 0.00065, ips: 14.4182 samples/sec | ETA 06:10:11
- 2022-04-13 12:14:02 [INFO] [TRAIN] epoch: 54, iter: 13300/120000, loss: 0.9397, lr: 0.008997, batch_cost: 0.2096, reader_cost: 0.00113, ips: 14.3144 samples/sec | ETA 06:12:42
- 2022-04-13 12:14:12 [INFO] [TRAIN] epoch: 54, iter: 13350/120000, loss: 0.9389, lr: 0.008993, batch_cost: 0.2014, reader_cost: 0.00107, ips: 14.8941 samples/sec | ETA 05:58:01
- 2022-04-13 12:14:25 [INFO] [TRAIN] epoch: 55, iter: 13400/120000, loss: 0.9248, lr: 0.008989, batch_cost: 0.2591, reader_cost: 0.05411, ips: 11.5765 samples/sec | ETA 07:40:25
- 2022-04-13 12:14:35 [INFO] [TRAIN] epoch: 55, iter: 13450/120000, loss: 0.9413, lr: 0.008985, batch_cost: 0.2077, reader_cost: 0.00083, ips: 14.4417 samples/sec | ETA 06:08:53
- 2022-04-13 12:14:46 [INFO] [TRAIN] epoch: 55, iter: 13500/120000, loss: 0.9190, lr: 0.008982, batch_cost: 0.2113, reader_cost: 0.00087, ips: 14.1969 samples/sec | ETA 06:15:04
- 2022-04-13 12:14:57 [INFO] [TRAIN] epoch: 55, iter: 13550/120000, loss: 0.9395, lr: 0.008978, batch_cost: 0.2218, reader_cost: 0.00141, ips: 13.5238 samples/sec | ETA 06:33:33
- 2022-04-13 12:15:07 [INFO] [TRAIN] epoch: 55, iter: 13600/120000, loss: 0.9588, lr: 0.008974, batch_cost: 0.2085, reader_cost: 0.00118, ips: 14.3908 samples/sec | ETA 06:09:40
- 2022-04-13 12:15:21 [INFO] [TRAIN] epoch: 56, iter: 13650/120000, loss: 0.9665, lr: 0.008970, batch_cost: 0.2643, reader_cost: 0.05388, ips: 11.3492 samples/sec | ETA 07:48:32
- 2022-04-13 12:15:31 [INFO] [TRAIN] epoch: 56, iter: 13700/120000, loss: 0.9316, lr: 0.008966, batch_cost: 0.2162, reader_cost: 0.00104, ips: 13.8779 samples/sec | ETA 06:22:58
- 2022-04-13 12:15:42 [INFO] [TRAIN] epoch: 56, iter: 13750/120000, loss: 0.9689, lr: 0.008963, batch_cost: 0.2020, reader_cost: 0.00058, ips: 14.8504 samples/sec | ETA 05:57:44
- 2022-04-13 12:15:52 [INFO] [TRAIN] epoch: 56, iter: 13800/120000, loss: 0.9559, lr: 0.008959, batch_cost: 0.2068, reader_cost: 0.00050, ips: 14.5098 samples/sec | ETA 06:05:57
- 2022-04-13 12:16:02 [INFO] [TRAIN] epoch: 56, iter: 13850/120000, loss: 0.9209, lr: 0.008955, batch_cost: 0.2035, reader_cost: 0.00118, ips: 14.7390 samples/sec | ETA 06:00:05
- 2022-04-13 12:16:16 [INFO] [TRAIN] epoch: 57, iter: 13900/120000, loss: 0.9353, lr: 0.008951, batch_cost: 0.2724, reader_cost: 0.05374, ips: 11.0113 samples/sec | ETA 08:01:46
- 2022-04-13 12:16:26 [INFO] [TRAIN] epoch: 57, iter: 13950/120000, loss: 0.9333, lr: 0.008947, batch_cost: 0.2106, reader_cost: 0.00066, ips: 14.2475 samples/sec | ETA 06:12:10
- 2022-04-13 12:16:36 [INFO] [TRAIN] epoch: 57, iter: 14000/120000, loss: 0.9483, lr: 0.008944, batch_cost: 0.2042, reader_cost: 0.00041, ips: 14.6908 samples/sec | ETA 06:00:46
- 2022-04-13 12:16:48 [INFO] [TRAIN] epoch: 57, iter: 14050/120000, loss: 0.9688, lr: 0.008940, batch_cost: 0.2360, reader_cost: 0.00144, ips: 12.7131 samples/sec | ETA 06:56:41
- 2022-04-13 12:16:59 [INFO] [TRAIN] epoch: 57, iter: 14100/120000, loss: 0.9910, lr: 0.008936, batch_cost: 0.2180, reader_cost: 0.00089, ips: 13.7642 samples/sec | ETA 06:24:41
- 2022-04-13 12:17:12 [INFO] [TRAIN] epoch: 58, iter: 14150/120000, loss: 0.9151, lr: 0.008932, batch_cost: 0.2660, reader_cost: 0.05528, ips: 11.2768 samples/sec | ETA 07:49:19
- 2022-04-13 12:17:23 [INFO] [TRAIN] epoch: 58, iter: 14200/120000, loss: 0.9292, lr: 0.008928, batch_cost: 0.2054, reader_cost: 0.00132, ips: 14.6050 samples/sec | ETA 06:02:12
- 2022-04-13 12:17:34 [INFO] [TRAIN] epoch: 58, iter: 14250/120000, loss: 0.9342, lr: 0.008925, batch_cost: 0.2165, reader_cost: 0.00096, ips: 13.8587 samples/sec | ETA 06:21:31
- 2022-04-13 12:17:44 [INFO] [TRAIN] epoch: 58, iter: 14300/120000, loss: 0.9623, lr: 0.008921, batch_cost: 0.2070, reader_cost: 0.00093, ips: 14.4906 samples/sec | ETA 06:04:43
- 2022-04-13 12:17:55 [INFO] [TRAIN] epoch: 58, iter: 14350/120000, loss: 0.9609, lr: 0.008917, batch_cost: 0.2132, reader_cost: 0.00137, ips: 14.0685 samples/sec | ETA 06:15:29
- 2022-04-13 12:18:08 [INFO] [TRAIN] epoch: 59, iter: 14400/120000, loss: 0.9443, lr: 0.008913, batch_cost: 0.2671, reader_cost: 0.05686, ips: 11.2326 samples/sec | ETA 07:50:03
- 2022-04-13 12:18:18 [INFO] [TRAIN] epoch: 59, iter: 14450/120000, loss: 0.9776, lr: 0.008909, batch_cost: 0.1985, reader_cost: 0.00103, ips: 15.1159 samples/sec | ETA 05:49:08
- 2022-04-13 12:18:28 [INFO] [TRAIN] epoch: 59, iter: 14500/120000, loss: 0.9218, lr: 0.008906, batch_cost: 0.2060, reader_cost: 0.00075, ips: 14.5652 samples/sec | ETA 06:02:09
- 2022-04-13 12:18:38 [INFO] [TRAIN] epoch: 59, iter: 14550/120000, loss: 0.9459, lr: 0.008902, batch_cost: 0.1980, reader_cost: 0.00044, ips: 15.1547 samples/sec | ETA 05:47:54
- 2022-04-13 12:18:49 [INFO] [TRAIN] epoch: 59, iter: 14600/120000, loss: 0.9447, lr: 0.008898, batch_cost: 0.2249, reader_cost: 0.00128, ips: 13.3401 samples/sec | ETA 06:35:02
- 2022-04-13 12:19:02 [INFO] [TRAIN] epoch: 60, iter: 14650/120000, loss: 0.9433, lr: 0.008894, batch_cost: 0.2637, reader_cost: 0.05832, ips: 11.3760 samples/sec | ETA 07:43:02
- 2022-04-13 12:19:13 [INFO] [TRAIN] epoch: 60, iter: 14700/120000, loss: 0.9344, lr: 0.008890, batch_cost: 0.2093, reader_cost: 0.00095, ips: 14.3349 samples/sec | ETA 06:07:17
- 2022-04-13 12:19:23 [INFO] [TRAIN] epoch: 60, iter: 14750/120000, loss: 0.9566, lr: 0.008887, batch_cost: 0.2031, reader_cost: 0.00169, ips: 14.7725 samples/sec | ETA 05:56:14
- 2022-04-13 12:19:33 [INFO] [TRAIN] epoch: 60, iter: 14800/120000, loss: 0.9743, lr: 0.008883, batch_cost: 0.2041, reader_cost: 0.00066, ips: 14.6980 samples/sec | ETA 05:57:52
- 2022-04-13 12:19:44 [INFO] [TRAIN] epoch: 60, iter: 14850/120000, loss: 0.9375, lr: 0.008879, batch_cost: 0.2068, reader_cost: 0.00116, ips: 14.5067 samples/sec | ETA 06:02:25
- 2022-04-13 12:19:57 [INFO] [TRAIN] epoch: 61, iter: 14900/120000, loss: 0.9626, lr: 0.008875, batch_cost: 0.2639, reader_cost: 0.06422, ips: 11.3695 samples/sec | ETA 07:42:12
- 2022-04-13 12:20:08 [INFO] [TRAIN] epoch: 61, iter: 14950/120000, loss: 0.9560, lr: 0.008871, batch_cost: 0.2172, reader_cost: 0.00132, ips: 13.8147 samples/sec | ETA 06:20:12
- 2022-04-13 12:20:18 [INFO] [TRAIN] epoch: 61, iter: 15000/120000, loss: 0.9379, lr: 0.008868, batch_cost: 0.2090, reader_cost: 0.00083, ips: 14.3541 samples/sec | ETA 06:05:45
- 2022-04-13 12:20:28 [INFO] [TRAIN] epoch: 61, iter: 15050/120000, loss: 0.9629, lr: 0.008864, batch_cost: 0.1958, reader_cost: 0.00082, ips: 15.3207 samples/sec | ETA 05:42:30
- 2022-04-13 12:20:38 [INFO] [TRAIN] epoch: 61, iter: 15100/120000, loss: 0.9441, lr: 0.008860, batch_cost: 0.2039, reader_cost: 0.00058, ips: 14.7133 samples/sec | ETA 05:56:28
- 2022-04-13 12:20:51 [INFO] [TRAIN] epoch: 62, iter: 15150/120000, loss: 0.9492, lr: 0.008856, batch_cost: 0.2663, reader_cost: 0.04737, ips: 11.2670 samples/sec | ETA 07:45:17
- 2022-04-13 12:21:01 [INFO] [TRAIN] epoch: 62, iter: 15200/120000, loss: 0.9663, lr: 0.008852, batch_cost: 0.1996, reader_cost: 0.00131, ips: 15.0292 samples/sec | ETA 05:48:39
- 2022-04-13 12:21:12 [INFO] [TRAIN] epoch: 62, iter: 15250/120000, loss: 0.9636, lr: 0.008849, batch_cost: 0.2027, reader_cost: 0.00071, ips: 14.7971 samples/sec | ETA 05:53:57
- 2022-04-13 12:21:23 [INFO] [TRAIN] epoch: 62, iter: 15300/120000, loss: 0.9822, lr: 0.008845, batch_cost: 0.2207, reader_cost: 0.00107, ips: 13.5943 samples/sec | ETA 06:25:05
- 2022-04-13 12:21:33 [INFO] [TRAIN] epoch: 62, iter: 15350/120000, loss: 0.9970, lr: 0.008841, batch_cost: 0.1989, reader_cost: 0.00123, ips: 15.0795 samples/sec | ETA 05:46:59
- 2022-04-13 12:21:46 [INFO] [TRAIN] epoch: 63, iter: 15400/120000, loss: 0.9290, lr: 0.008837, batch_cost: 0.2629, reader_cost: 0.05742, ips: 11.4106 samples/sec | ETA 07:38:20
- 2022-04-13 12:21:56 [INFO] [TRAIN] epoch: 63, iter: 15450/120000, loss: 0.9550, lr: 0.008833, batch_cost: 0.1972, reader_cost: 0.00089, ips: 15.2135 samples/sec | ETA 05:43:36
- 2022-04-13 12:22:08 [INFO] [TRAIN] epoch: 63, iter: 15500/120000, loss: 0.9393, lr: 0.008830, batch_cost: 0.2421, reader_cost: 0.00117, ips: 12.3927 samples/sec | ETA 07:01:37
- 2022-04-13 12:22:19 [INFO] [TRAIN] epoch: 63, iter: 15550/120000, loss: 0.9381, lr: 0.008826, batch_cost: 0.2214, reader_cost: 0.00069, ips: 13.5481 samples/sec | ETA 06:25:28
- 2022-04-13 12:22:30 [INFO] [TRAIN] epoch: 63, iter: 15600/120000, loss: 0.9275, lr: 0.008822, batch_cost: 0.2191, reader_cost: 0.00104, ips: 13.6947 samples/sec | ETA 06:21:10
- 2022-04-13 12:22:43 [INFO] [TRAIN] epoch: 64, iter: 15650/120000, loss: 0.9355, lr: 0.008818, batch_cost: 0.2697, reader_cost: 0.05477, ips: 11.1229 samples/sec | ETA 07:49:04
- 2022-04-13 12:22:54 [INFO] [TRAIN] epoch: 64, iter: 15700/120000, loss: 0.9546, lr: 0.008814, batch_cost: 0.2147, reader_cost: 0.00091, ips: 13.9756 samples/sec | ETA 06:13:09
- 2022-04-13 12:23:04 [INFO] [TRAIN] epoch: 64, iter: 15750/120000, loss: 0.9582, lr: 0.008811, batch_cost: 0.1999, reader_cost: 0.00080, ips: 15.0055 samples/sec | ETA 05:47:22
- 2022-04-13 12:23:15 [INFO] [TRAIN] epoch: 64, iter: 15800/120000, loss: 0.9516, lr: 0.008807, batch_cost: 0.2142, reader_cost: 0.00110, ips: 14.0060 samples/sec | ETA 06:11:59
- 2022-04-13 12:23:25 [INFO] [TRAIN] epoch: 64, iter: 15850/120000, loss: 0.9524, lr: 0.008803, batch_cost: 0.1986, reader_cost: 0.00080, ips: 15.1024 samples/sec | ETA 05:44:48
- 2022-04-13 12:23:38 [INFO] [TRAIN] epoch: 65, iter: 15900/120000, loss: 0.9092, lr: 0.008799, batch_cost: 0.2625, reader_cost: 0.05702, ips: 11.4301 samples/sec | ETA 07:35:22
- 2022-04-13 12:23:48 [INFO] [TRAIN] epoch: 65, iter: 15950/120000, loss: 0.9107, lr: 0.008795, batch_cost: 0.2086, reader_cost: 0.00155, ips: 14.3849 samples/sec | ETA 06:01:39
- 2022-04-13 12:23:58 [INFO] [TRAIN] epoch: 65, iter: 16000/120000, loss: 0.9675, lr: 0.008792, batch_cost: 0.2017, reader_cost: 0.00144, ips: 14.8749 samples/sec | ETA 05:49:34
- 2022-04-13 12:23:58 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1950 - reader cost: 0.1501
- 2022-04-13 12:24:23 [INFO] [EVAL] #Images: 500 mIoU: 0.7008 Acc: 0.9462 Kappa: 0.9301 Dice: 0.8114
- 2022-04-13 12:24:23 [INFO] [EVAL] Class IoU:
- [0.9714 0.791 0.9047 0.3214 0.5354 0.5856 0.6072 0.7375 0.8958 0.5565
- 0.9353 0.7742 0.5559 0.9323 0.6327 0.7362 0.6413 0.4865 0.7146]
- 2022-04-13 12:24:23 [INFO] [EVAL] Class Precision:
- [0.9869 0.8731 0.9542 0.7212 0.6673 0.7765 0.8716 0.8921 0.9244 0.7304
- 0.9576 0.8708 0.7163 0.9647 0.9509 0.8492 0.7571 0.7216 0.8499]
- 2022-04-13 12:24:23 [INFO] [EVAL] Class Recall:
- [0.9841 0.8937 0.9458 0.367 0.7304 0.7043 0.6668 0.8098 0.9666 0.7003
- 0.9756 0.8747 0.7128 0.9653 0.6541 0.847 0.8074 0.5989 0.8178]
- 2022-04-13 12:24:23 [INFO] [EVAL] The model with the best validation mIoU (0.7155) was saved at iter 12000.
- 2022-04-13 12:24:34 [INFO] [TRAIN] epoch: 65, iter: 16050/120000, loss: 0.9290, lr: 0.008788, batch_cost: 0.2138, reader_cost: 0.00134, ips: 14.0317 samples/sec | ETA 06:10:24
- 2022-04-13 12:24:44 [INFO] [TRAIN] epoch: 65, iter: 16100/120000, loss: 0.9934, lr: 0.008784, batch_cost: 0.2040, reader_cost: 0.00077, ips: 14.7087 samples/sec | ETA 05:53:11
- 2022-04-13 12:24:58 [INFO] [TRAIN] epoch: 66, iter: 16150/120000, loss: 0.9603, lr: 0.008780, batch_cost: 0.2748, reader_cost: 0.05052, ips: 10.9158 samples/sec | ETA 07:55:41
- 2022-04-13 12:25:09 [INFO] [TRAIN] epoch: 66, iter: 16200/120000, loss: 0.9470, lr: 0.008776, batch_cost: 0.2214, reader_cost: 0.00128, ips: 13.5518 samples/sec | ETA 06:22:58
- 2022-04-13 12:25:20 [INFO] [TRAIN] epoch: 66, iter: 16250/120000, loss: 0.9611, lr: 0.008773, batch_cost: 0.2157, reader_cost: 0.00088, ips: 13.9088 samples/sec | ETA 06:12:57
- 2022-04-13 12:25:31 [INFO] [TRAIN] epoch: 66, iter: 16300/120000, loss: 0.9583, lr: 0.008769, batch_cost: 0.2209, reader_cost: 0.00081, ips: 13.5832 samples/sec | ETA 06:21:43
- 2022-04-13 12:25:41 [INFO] [TRAIN] epoch: 66, iter: 16350/120000, loss: 0.9687, lr: 0.008765, batch_cost: 0.2026, reader_cost: 0.00117, ips: 14.8084 samples/sec | ETA 05:49:58
- 2022-04-13 12:25:55 [INFO] [TRAIN] epoch: 67, iter: 16400/120000, loss: 0.9257, lr: 0.008761, batch_cost: 0.2733, reader_cost: 0.06311, ips: 10.9770 samples/sec | ETA 07:51:53
- 2022-04-13 12:26:05 [INFO] [TRAIN] epoch: 67, iter: 16450/120000, loss: 0.9727, lr: 0.008757, batch_cost: 0.2057, reader_cost: 0.00155, ips: 14.5829 samples/sec | ETA 05:55:02
- 2022-04-13 12:26:16 [INFO] [TRAIN] epoch: 67, iter: 16500/120000, loss: 0.9373, lr: 0.008754, batch_cost: 0.2130, reader_cost: 0.00045, ips: 14.0840 samples/sec | ETA 06:07:26
- 2022-04-13 12:26:26 [INFO] [TRAIN] epoch: 67, iter: 16550/120000, loss: 0.9456, lr: 0.008750, batch_cost: 0.2159, reader_cost: 0.00060, ips: 13.8946 samples/sec | ETA 06:12:16
- 2022-04-13 12:26:37 [INFO] [TRAIN] epoch: 67, iter: 16600/120000, loss: 0.9429, lr: 0.008746, batch_cost: 0.2190, reader_cost: 0.00079, ips: 13.6996 samples/sec | ETA 06:17:23
- 2022-04-13 12:26:50 [INFO] [TRAIN] epoch: 68, iter: 16650/120000, loss: 0.9548, lr: 0.008742, batch_cost: 0.2562, reader_cost: 0.04967, ips: 11.7091 samples/sec | ETA 07:21:19
- 2022-04-13 12:27:00 [INFO] [TRAIN] epoch: 68, iter: 16700/120000, loss: 0.9569, lr: 0.008738, batch_cost: 0.2006, reader_cost: 0.00119, ips: 14.9522 samples/sec | ETA 05:45:26
- 2022-04-13 12:27:11 [INFO] [TRAIN] epoch: 68, iter: 16750/120000, loss: 0.9688, lr: 0.008735, batch_cost: 0.2133, reader_cost: 0.00055, ips: 14.0660 samples/sec | ETA 06:07:01
- 2022-04-13 12:27:22 [INFO] [TRAIN] epoch: 68, iter: 16800/120000, loss: 0.9659, lr: 0.008731, batch_cost: 0.2233, reader_cost: 0.00074, ips: 13.4354 samples/sec | ETA 06:24:03
- 2022-04-13 12:27:32 [INFO] [TRAIN] epoch: 68, iter: 16850/120000, loss: 0.9527, lr: 0.008727, batch_cost: 0.2063, reader_cost: 0.00179, ips: 14.5424 samples/sec | ETA 05:54:39
- 2022-04-13 12:27:46 [INFO] [TRAIN] epoch: 69, iter: 16900/120000, loss: 0.9494, lr: 0.008723, batch_cost: 0.2663, reader_cost: 0.05431, ips: 11.2654 samples/sec | ETA 07:37:35
- 2022-04-13 12:27:55 [INFO] [TRAIN] epoch: 69, iter: 16950/120000, loss: 0.9689, lr: 0.008719, batch_cost: 0.1932, reader_cost: 0.00116, ips: 15.5266 samples/sec | ETA 05:31:50
- 2022-04-13 12:28:05 [INFO] [TRAIN] epoch: 69, iter: 17000/120000, loss: 0.9746, lr: 0.008716, batch_cost: 0.1994, reader_cost: 0.00199, ips: 15.0465 samples/sec | ETA 05:42:16
- 2022-04-13 12:28:17 [INFO] [TRAIN] epoch: 69, iter: 17050/120000, loss: 0.9224, lr: 0.008712, batch_cost: 0.2373, reader_cost: 0.00084, ips: 12.6418 samples/sec | ETA 06:47:10
- 2022-04-13 12:28:27 [INFO] [TRAIN] epoch: 69, iter: 17100/120000, loss: 0.9260, lr: 0.008708, batch_cost: 0.2038, reader_cost: 0.00103, ips: 14.7183 samples/sec | ETA 05:49:33
- 2022-04-13 12:28:40 [INFO] [TRAIN] epoch: 70, iter: 17150/120000, loss: 0.9720, lr: 0.008704, batch_cost: 0.2628, reader_cost: 0.06070, ips: 11.4170 samples/sec | ETA 07:30:25
- 2022-04-13 12:28:51 [INFO] [TRAIN] epoch: 70, iter: 17200/120000, loss: 0.9371, lr: 0.008700, batch_cost: 0.2020, reader_cost: 0.00060, ips: 14.8529 samples/sec | ETA 05:46:03
- 2022-04-13 12:29:01 [INFO] [TRAIN] epoch: 70, iter: 17250/120000, loss: 0.9051, lr: 0.008696, batch_cost: 0.2053, reader_cost: 0.00068, ips: 14.6146 samples/sec | ETA 05:51:31
- 2022-04-13 12:29:11 [INFO] [TRAIN] epoch: 70, iter: 17300/120000, loss: 0.9322, lr: 0.008693, batch_cost: 0.2126, reader_cost: 0.00105, ips: 14.1090 samples/sec | ETA 06:03:57
- 2022-04-13 12:29:22 [INFO] [TRAIN] epoch: 70, iter: 17350/120000, loss: 0.9223, lr: 0.008689, batch_cost: 0.2052, reader_cost: 0.00089, ips: 14.6212 samples/sec | ETA 05:51:01
- 2022-04-13 12:29:35 [INFO] [TRAIN] epoch: 71, iter: 17400/120000, loss: 0.9379, lr: 0.008685, batch_cost: 0.2688, reader_cost: 0.06287, ips: 11.1628 samples/sec | ETA 07:39:33
- 2022-04-13 12:29:47 [INFO] [TRAIN] epoch: 71, iter: 17450/120000, loss: 0.9570, lr: 0.008681, batch_cost: 0.2311, reader_cost: 0.00089, ips: 12.9840 samples/sec | ETA 06:34:54
- 2022-04-13 12:29:58 [INFO] [TRAIN] epoch: 71, iter: 17500/120000, loss: 0.9421, lr: 0.008677, batch_cost: 0.2182, reader_cost: 0.00071, ips: 13.7469 samples/sec | ETA 06:12:48
- 2022-04-13 12:30:08 [INFO] [TRAIN] epoch: 71, iter: 17550/120000, loss: 0.9516, lr: 0.008674, batch_cost: 0.2012, reader_cost: 0.00123, ips: 14.9108 samples/sec | ETA 05:43:32
- 2022-04-13 12:30:18 [INFO] [TRAIN] epoch: 71, iter: 17600/120000, loss: 1.0003, lr: 0.008670, batch_cost: 0.2029, reader_cost: 0.00105, ips: 14.7830 samples/sec | ETA 05:46:20
- 2022-04-13 12:30:31 [INFO] [TRAIN] epoch: 72, iter: 17650/120000, loss: 0.9630, lr: 0.008666, batch_cost: 0.2615, reader_cost: 0.05881, ips: 11.4721 samples/sec | ETA 07:26:04
- 2022-04-13 12:30:41 [INFO] [TRAIN] epoch: 72, iter: 17700/120000, loss: 0.9412, lr: 0.008662, batch_cost: 0.1960, reader_cost: 0.00064, ips: 15.3086 samples/sec | ETA 05:34:07
- 2022-04-13 12:30:51 [INFO] [TRAIN] epoch: 72, iter: 17750/120000, loss: 0.9367, lr: 0.008658, batch_cost: 0.2004, reader_cost: 0.00075, ips: 14.9679 samples/sec | ETA 05:41:33
- 2022-04-13 12:31:02 [INFO] [TRAIN] epoch: 72, iter: 17800/120000, loss: 0.9344, lr: 0.008655, batch_cost: 0.2299, reader_cost: 0.00105, ips: 13.0515 samples/sec | ETA 06:31:31
- 2022-04-13 12:31:12 [INFO] [TRAIN] epoch: 72, iter: 17850/120000, loss: 0.9740, lr: 0.008651, batch_cost: 0.1931, reader_cost: 0.00215, ips: 15.5348 samples/sec | ETA 05:28:46
- 2022-04-13 12:31:26 [INFO] [TRAIN] epoch: 73, iter: 17900/120000, loss: 0.9426, lr: 0.008647, batch_cost: 0.2794, reader_cost: 0.05773, ips: 10.7363 samples/sec | ETA 07:55:29
- 2022-04-13 12:31:36 [INFO] [TRAIN] epoch: 73, iter: 17950/120000, loss: 0.9144, lr: 0.008643, batch_cost: 0.2019, reader_cost: 0.00045, ips: 14.8570 samples/sec | ETA 05:43:26
- 2022-04-13 12:31:47 [INFO] [TRAIN] epoch: 73, iter: 18000/120000, loss: 0.9292, lr: 0.008639, batch_cost: 0.2113, reader_cost: 0.00144, ips: 14.1997 samples/sec | ETA 05:59:09
- 2022-04-13 12:31:57 [INFO] [TRAIN] epoch: 73, iter: 18050/120000, loss: 1.0043, lr: 0.008636, batch_cost: 0.2081, reader_cost: 0.00054, ips: 14.4168 samples/sec | ETA 05:53:34
- 2022-04-13 12:32:07 [INFO] [TRAIN] epoch: 73, iter: 18100/120000, loss: 0.9509, lr: 0.008632, batch_cost: 0.1982, reader_cost: 0.00122, ips: 15.1338 samples/sec | ETA 05:36:39
- 2022-04-13 12:32:20 [INFO] [TRAIN] epoch: 74, iter: 18150/120000, loss: 0.9961, lr: 0.008628, batch_cost: 0.2698, reader_cost: 0.05283, ips: 11.1192 samples/sec | ETA 07:37:59
- 2022-04-13 12:32:31 [INFO] [TRAIN] epoch: 74, iter: 18200/120000, loss: 0.9699, lr: 0.008624, batch_cost: 0.2084, reader_cost: 0.00119, ips: 14.3965 samples/sec | ETA 05:53:33
- 2022-04-13 12:32:41 [INFO] [TRAIN] epoch: 74, iter: 18250/120000, loss: 0.9182, lr: 0.008620, batch_cost: 0.2140, reader_cost: 0.00096, ips: 14.0202 samples/sec | ETA 06:02:52
- 2022-04-13 12:32:51 [INFO] [TRAIN] epoch: 74, iter: 18300/120000, loss: 0.9514, lr: 0.008616, batch_cost: 0.1933, reader_cost: 0.00130, ips: 15.5182 samples/sec | ETA 05:27:40
- 2022-04-13 12:33:01 [INFO] [TRAIN] epoch: 74, iter: 18350/120000, loss: 0.9129, lr: 0.008613, batch_cost: 0.2028, reader_cost: 0.00098, ips: 14.7947 samples/sec | ETA 05:43:32
- 2022-04-13 12:33:15 [INFO] [TRAIN] epoch: 75, iter: 18400/120000, loss: 0.9249, lr: 0.008609, batch_cost: 0.2722, reader_cost: 0.06082, ips: 11.0199 samples/sec | ETA 07:40:59
- 2022-04-13 12:33:26 [INFO] [TRAIN] epoch: 75, iter: 18450/120000, loss: 0.9350, lr: 0.008605, batch_cost: 0.2155, reader_cost: 0.00126, ips: 13.9195 samples/sec | ETA 06:04:46
- 2022-04-13 12:33:36 [INFO] [TRAIN] epoch: 75, iter: 18500/120000, loss: 0.9278, lr: 0.008601, batch_cost: 0.2034, reader_cost: 0.00089, ips: 14.7479 samples/sec | ETA 05:44:07
- 2022-04-13 12:33:46 [INFO] [TRAIN] epoch: 75, iter: 18550/120000, loss: 0.9415, lr: 0.008597, batch_cost: 0.2086, reader_cost: 0.00086, ips: 14.3832 samples/sec | ETA 05:52:40
- 2022-04-13 12:33:56 [INFO] [TRAIN] epoch: 75, iter: 18600/120000, loss: 0.9301, lr: 0.008594, batch_cost: 0.1868, reader_cost: 0.00074, ips: 16.0610 samples/sec | ETA 05:15:40
- 2022-04-13 12:34:10 [INFO] [TRAIN] epoch: 76, iter: 18650/120000, loss: 0.8959, lr: 0.008590, batch_cost: 0.2824, reader_cost: 0.05160, ips: 10.6216 samples/sec | ETA 07:57:05
- 2022-04-13 12:34:20 [INFO] [TRAIN] epoch: 76, iter: 18700/120000, loss: 0.9148, lr: 0.008586, batch_cost: 0.2007, reader_cost: 0.00134, ips: 14.9485 samples/sec | ETA 05:38:49
- 2022-04-13 12:34:30 [INFO] [TRAIN] epoch: 76, iter: 18750/120000, loss: 0.9480, lr: 0.008582, batch_cost: 0.2024, reader_cost: 0.00085, ips: 14.8205 samples/sec | ETA 05:41:35
- 2022-04-13 12:34:40 [INFO] [TRAIN] epoch: 76, iter: 18800/120000, loss: 0.9366, lr: 0.008578, batch_cost: 0.2108, reader_cost: 0.00083, ips: 14.2331 samples/sec | ETA 05:55:30
- 2022-04-13 12:34:54 [INFO] [TRAIN] epoch: 77, iter: 18850/120000, loss: 0.9309, lr: 0.008575, batch_cost: 0.2680, reader_cost: 0.06283, ips: 11.1940 samples/sec | ETA 07:31:48
- 2022-04-13 12:35:05 [INFO] [TRAIN] epoch: 77, iter: 18900/120000, loss: 0.9136, lr: 0.008571, batch_cost: 0.2199, reader_cost: 0.00122, ips: 13.6420 samples/sec | ETA 06:10:32
- 2022-04-13 12:35:15 [INFO] [TRAIN] epoch: 77, iter: 18950/120000, loss: 0.9487, lr: 0.008567, batch_cost: 0.1984, reader_cost: 0.00089, ips: 15.1245 samples/sec | ETA 05:34:03
- 2022-04-13 12:35:25 [INFO] [TRAIN] epoch: 77, iter: 19000/120000, loss: 0.9464, lr: 0.008563, batch_cost: 0.2027, reader_cost: 0.00069, ips: 14.8006 samples/sec | ETA 05:41:12
- 2022-04-13 12:35:35 [INFO] [TRAIN] epoch: 77, iter: 19050/120000, loss: 0.9200, lr: 0.008559, batch_cost: 0.2065, reader_cost: 0.00080, ips: 14.5301 samples/sec | ETA 05:47:22
- 2022-04-13 12:35:48 [INFO] [TRAIN] epoch: 78, iter: 19100/120000, loss: 0.9755, lr: 0.008555, batch_cost: 0.2616, reader_cost: 0.05580, ips: 11.4682 samples/sec | ETA 07:19:54
- 2022-04-13 12:35:58 [INFO] [TRAIN] epoch: 78, iter: 19150/120000, loss: 0.9200, lr: 0.008552, batch_cost: 0.2019, reader_cost: 0.00065, ips: 14.8560 samples/sec | ETA 05:39:25
- 2022-04-13 12:36:09 [INFO] [TRAIN] epoch: 78, iter: 19200/120000, loss: 0.9575, lr: 0.008548, batch_cost: 0.2114, reader_cost: 0.00109, ips: 14.1916 samples/sec | ETA 05:55:08
- 2022-04-13 12:36:20 [INFO] [TRAIN] epoch: 78, iter: 19250/120000, loss: 0.9275, lr: 0.008544, batch_cost: 0.2117, reader_cost: 0.00099, ips: 14.1720 samples/sec | ETA 05:55:27
- 2022-04-13 12:36:30 [INFO] [TRAIN] epoch: 78, iter: 19300/120000, loss: 0.9073, lr: 0.008540, batch_cost: 0.2153, reader_cost: 0.00068, ips: 13.9340 samples/sec | ETA 06:01:20
- 2022-04-13 12:36:44 [INFO] [TRAIN] epoch: 79, iter: 19350/120000, loss: 0.9034, lr: 0.008536, batch_cost: 0.2824, reader_cost: 0.07031, ips: 10.6214 samples/sec | ETA 07:53:48
- 2022-04-13 12:36:55 [INFO] [TRAIN] epoch: 79, iter: 19400/120000, loss: 0.8913, lr: 0.008533, batch_cost: 0.2018, reader_cost: 0.00089, ips: 14.8651 samples/sec | ETA 05:38:22
- 2022-04-13 12:37:05 [INFO] [TRAIN] epoch: 79, iter: 19450/120000, loss: 0.9085, lr: 0.008529, batch_cost: 0.2136, reader_cost: 0.00121, ips: 14.0432 samples/sec | ETA 05:58:00
- 2022-04-13 12:37:16 [INFO] [TRAIN] epoch: 79, iter: 19500/120000, loss: 0.9453, lr: 0.008525, batch_cost: 0.2193, reader_cost: 0.00120, ips: 13.6796 samples/sec | ETA 06:07:20
- 2022-04-13 12:37:27 [INFO] [TRAIN] epoch: 79, iter: 19550/120000, loss: 0.9222, lr: 0.008521, batch_cost: 0.2111, reader_cost: 0.00053, ips: 14.2110 samples/sec | ETA 05:53:25
- 2022-04-13 12:37:40 [INFO] [TRAIN] epoch: 80, iter: 19600/120000, loss: 0.9376, lr: 0.008517, batch_cost: 0.2709, reader_cost: 0.05993, ips: 11.0746 samples/sec | ETA 07:33:17
- 2022-04-13 12:37:51 [INFO] [TRAIN] epoch: 80, iter: 19650/120000, loss: 0.9446, lr: 0.008513, batch_cost: 0.2053, reader_cost: 0.00088, ips: 14.6143 samples/sec | ETA 05:43:19
- 2022-04-13 12:38:01 [INFO] [TRAIN] epoch: 80, iter: 19700/120000, loss: 0.9966, lr: 0.008510, batch_cost: 0.2157, reader_cost: 0.00084, ips: 13.9081 samples/sec | ETA 06:00:34
- 2022-04-13 12:38:12 [INFO] [TRAIN] epoch: 80, iter: 19750/120000, loss: 0.9158, lr: 0.008506, batch_cost: 0.2084, reader_cost: 0.00123, ips: 14.3952 samples/sec | ETA 05:48:12
- 2022-04-13 12:38:22 [INFO] [TRAIN] epoch: 80, iter: 19800/120000, loss: 0.9394, lr: 0.008502, batch_cost: 0.2051, reader_cost: 0.00060, ips: 14.6288 samples/sec | ETA 05:42:28
- 2022-04-13 12:38:35 [INFO] [TRAIN] epoch: 81, iter: 19850/120000, loss: 0.9256, lr: 0.008498, batch_cost: 0.2689, reader_cost: 0.06393, ips: 11.1575 samples/sec | ETA 07:28:47
- 2022-04-13 12:38:46 [INFO] [TRAIN] epoch: 81, iter: 19900/120000, loss: 0.9400, lr: 0.008494, batch_cost: 0.2105, reader_cost: 0.00089, ips: 14.2519 samples/sec | ETA 05:51:10
- 2022-04-13 12:38:57 [INFO] [TRAIN] epoch: 81, iter: 19950/120000, loss: 0.9953, lr: 0.008491, batch_cost: 0.2281, reader_cost: 0.00107, ips: 13.1517 samples/sec | ETA 06:20:22
- 2022-04-13 12:39:09 [INFO] [TRAIN] epoch: 81, iter: 20000/120000, loss: 0.9311, lr: 0.008487, batch_cost: 0.2315, reader_cost: 0.00135, ips: 12.9591 samples/sec | ETA 06:25:49
- 2022-04-13 12:39:09 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1962 - reader cost: 0.1515
- 2022-04-13 12:39:34 [INFO] [EVAL] #Images: 500 mIoU: 0.7028 Acc: 0.9499 Kappa: 0.9349 Dice: 0.8127
- 2022-04-13 12:39:34 [INFO] [EVAL] Class IoU:
- [0.9766 0.7929 0.9088 0.481 0.4621 0.5901 0.6564 0.7493 0.9104 0.437
- 0.941 0.7599 0.4949 0.9377 0.7131 0.7483 0.5628 0.4956 0.7355]
- 2022-04-13 12:39:34 [INFO] [EVAL] Class Precision:
- [0.9872 0.8587 0.9503 0.7735 0.8689 0.7587 0.8444 0.8703 0.9376 0.8997
- 0.9674 0.8063 0.6747 0.974 0.8327 0.9034 0.8962 0.7709 0.8678]
- 2022-04-13 12:39:34 [INFO] [EVAL] Class Recall:
- [0.9891 0.9118 0.9541 0.5599 0.4967 0.7265 0.7467 0.8436 0.9691 0.4593
- 0.9717 0.9296 0.65 0.9617 0.8324 0.8134 0.6021 0.5811 0.8284]
- 2022-04-13 12:39:34 [INFO] [EVAL] The model with the best validation mIoU (0.7155) was saved at iter 12000.
- 2022-04-13 12:39:44 [INFO] [TRAIN] epoch: 81, iter: 20050/120000, loss: 0.9390, lr: 0.008483, batch_cost: 0.2040, reader_cost: 0.00079, ips: 14.7074 samples/sec | ETA 05:39:47
- 2022-04-13 12:39:58 [INFO] [TRAIN] epoch: 82, iter: 20100/120000, loss: 0.9565, lr: 0.008479, batch_cost: 0.2682, reader_cost: 0.05661, ips: 11.1842 samples/sec | ETA 07:26:36
- 2022-04-13 12:40:08 [INFO] [TRAIN] epoch: 82, iter: 20150/120000, loss: 0.9586, lr: 0.008475, batch_cost: 0.2133, reader_cost: 0.00125, ips: 14.0664 samples/sec | ETA 05:54:55
- 2022-04-13 12:40:18 [INFO] [TRAIN] epoch: 82, iter: 20200/120000, loss: 0.9422, lr: 0.008471, batch_cost: 0.1960, reader_cost: 0.00110, ips: 15.3071 samples/sec | ETA 05:25:59
- 2022-04-13 12:40:29 [INFO] [TRAIN] epoch: 82, iter: 20250/120000, loss: 0.9423, lr: 0.008468, batch_cost: 0.2219, reader_cost: 0.00126, ips: 13.5170 samples/sec | ETA 06:08:58
- 2022-04-13 12:40:40 [INFO] [TRAIN] epoch: 82, iter: 20300/120000, loss: 0.9381, lr: 0.008464, batch_cost: 0.2124, reader_cost: 0.00101, ips: 14.1238 samples/sec | ETA 05:52:57
- 2022-04-13 12:40:53 [INFO] [TRAIN] epoch: 83, iter: 20350/120000, loss: 0.8832, lr: 0.008460, batch_cost: 0.2649, reader_cost: 0.05416, ips: 11.3263 samples/sec | ETA 07:19:54
- 2022-04-13 12:41:04 [INFO] [TRAIN] epoch: 83, iter: 20400/120000, loss: 0.9104, lr: 0.008456, batch_cost: 0.2059, reader_cost: 0.00088, ips: 14.5670 samples/sec | ETA 05:41:52
- 2022-04-13 12:41:13 [INFO] [TRAIN] epoch: 83, iter: 20450/120000, loss: 0.9693, lr: 0.008452, batch_cost: 0.1939, reader_cost: 0.00090, ips: 15.4682 samples/sec | ETA 05:21:47
- 2022-04-13 12:41:23 [INFO] [TRAIN] epoch: 83, iter: 20500/120000, loss: 0.9523, lr: 0.008449, batch_cost: 0.1995, reader_cost: 0.00171, ips: 15.0404 samples/sec | ETA 05:30:46
- 2022-04-13 12:41:34 [INFO] [TRAIN] epoch: 83, iter: 20550/120000, loss: 0.9526, lr: 0.008445, batch_cost: 0.2245, reader_cost: 0.00105, ips: 13.3650 samples/sec | ETA 06:12:03
- 2022-04-13 12:41:48 [INFO] [TRAIN] epoch: 84, iter: 20600/120000, loss: 0.9375, lr: 0.008441, batch_cost: 0.2635, reader_cost: 0.04864, ips: 11.3869 samples/sec | ETA 07:16:28
- 2022-04-13 12:41:59 [INFO] [TRAIN] epoch: 84, iter: 20650/120000, loss: 0.9122, lr: 0.008437, batch_cost: 0.2214, reader_cost: 0.00070, ips: 13.5480 samples/sec | ETA 06:06:39
- 2022-04-13 12:42:10 [INFO] [TRAIN] epoch: 84, iter: 20700/120000, loss: 0.9264, lr: 0.008433, batch_cost: 0.2250, reader_cost: 0.00103, ips: 13.3340 samples/sec | ETA 06:12:21
- 2022-04-13 12:42:20 [INFO] [TRAIN] epoch: 84, iter: 20750/120000, loss: 0.9184, lr: 0.008429, batch_cost: 0.2021, reader_cost: 0.00112, ips: 14.8454 samples/sec | ETA 05:34:16
- 2022-04-13 12:42:31 [INFO] [TRAIN] epoch: 84, iter: 20800/120000, loss: 0.9061, lr: 0.008426, batch_cost: 0.2234, reader_cost: 0.00184, ips: 13.4304 samples/sec | ETA 06:09:18
- 2022-04-13 12:42:45 [INFO] [TRAIN] epoch: 85, iter: 20850/120000, loss: 0.9242, lr: 0.008422, batch_cost: 0.2662, reader_cost: 0.05928, ips: 11.2697 samples/sec | ETA 07:19:53
- 2022-04-13 12:42:56 [INFO] [TRAIN] epoch: 85, iter: 20900/120000, loss: 0.9147, lr: 0.008418, batch_cost: 0.2194, reader_cost: 0.00073, ips: 13.6765 samples/sec | ETA 06:02:17
- 2022-04-13 12:43:06 [INFO] [TRAIN] epoch: 85, iter: 20950/120000, loss: 0.9312, lr: 0.008414, batch_cost: 0.2105, reader_cost: 0.00090, ips: 14.2499 samples/sec | ETA 05:47:32
- 2022-04-13 12:43:16 [INFO] [TRAIN] epoch: 85, iter: 21000/120000, loss: 0.9222, lr: 0.008410, batch_cost: 0.2005, reader_cost: 0.00128, ips: 14.9594 samples/sec | ETA 05:30:53
- 2022-04-13 12:43:27 [INFO] [TRAIN] epoch: 85, iter: 21050/120000, loss: 0.9276, lr: 0.008406, batch_cost: 0.2199, reader_cost: 0.00118, ips: 13.6442 samples/sec | ETA 06:02:36
- 2022-04-13 12:43:40 [INFO] [TRAIN] epoch: 86, iter: 21100/120000, loss: 0.9155, lr: 0.008403, batch_cost: 0.2672, reader_cost: 0.05479, ips: 11.2292 samples/sec | ETA 07:20:22
- 2022-04-13 12:43:51 [INFO] [TRAIN] epoch: 86, iter: 21150/120000, loss: 0.9493, lr: 0.008399, batch_cost: 0.2132, reader_cost: 0.00123, ips: 14.0706 samples/sec | ETA 05:51:15
- 2022-04-13 12:44:02 [INFO] [TRAIN] epoch: 86, iter: 21200/120000, loss: 0.9698, lr: 0.008395, batch_cost: 0.2141, reader_cost: 0.00034, ips: 14.0132 samples/sec | ETA 05:52:31
- 2022-04-13 12:44:12 [INFO] [TRAIN] epoch: 86, iter: 21250/120000, loss: 0.9283, lr: 0.008391, batch_cost: 0.1983, reader_cost: 0.00107, ips: 15.1270 samples/sec | ETA 05:26:24
- 2022-04-13 12:44:22 [INFO] [TRAIN] epoch: 86, iter: 21300/120000, loss: 0.9479, lr: 0.008387, batch_cost: 0.2064, reader_cost: 0.00121, ips: 14.5371 samples/sec | ETA 05:39:28
- 2022-04-13 12:44:35 [INFO] [TRAIN] epoch: 87, iter: 21350/120000, loss: 0.9399, lr: 0.008384, batch_cost: 0.2681, reader_cost: 0.05566, ips: 11.1889 samples/sec | ETA 07:20:50
- 2022-04-13 12:44:46 [INFO] [TRAIN] epoch: 87, iter: 21400/120000, loss: 0.9482, lr: 0.008380, batch_cost: 0.2030, reader_cost: 0.00076, ips: 14.7810 samples/sec | ETA 05:33:32
- 2022-04-13 12:44:56 [INFO] [TRAIN] epoch: 87, iter: 21450/120000, loss: 0.9408, lr: 0.008376, batch_cost: 0.2084, reader_cost: 0.00076, ips: 14.3939 samples/sec | ETA 05:42:19
- 2022-04-13 12:45:07 [INFO] [TRAIN] epoch: 87, iter: 21500/120000, loss: 0.9363, lr: 0.008372, batch_cost: 0.2159, reader_cost: 0.00063, ips: 13.8936 samples/sec | ETA 05:54:28
- 2022-04-13 12:45:17 [INFO] [TRAIN] epoch: 87, iter: 21550/120000, loss: 0.9286, lr: 0.008368, batch_cost: 0.2029, reader_cost: 0.00069, ips: 14.7829 samples/sec | ETA 05:32:59
- 2022-04-13 12:45:30 [INFO] [TRAIN] epoch: 88, iter: 21600/120000, loss: 0.8996, lr: 0.008364, batch_cost: 0.2613, reader_cost: 0.05339, ips: 11.4810 samples/sec | ETA 07:08:32
- 2022-04-13 12:45:41 [INFO] [TRAIN] epoch: 88, iter: 21650/120000, loss: 0.9288, lr: 0.008361, batch_cost: 0.2182, reader_cost: 0.00086, ips: 13.7464 samples/sec | ETA 05:57:43
- 2022-04-13 12:45:51 [INFO] [TRAIN] epoch: 88, iter: 21700/120000, loss: 0.9044, lr: 0.008357, batch_cost: 0.2050, reader_cost: 0.00107, ips: 14.6337 samples/sec | ETA 05:35:52
- 2022-04-13 12:46:01 [INFO] [TRAIN] epoch: 88, iter: 21750/120000, loss: 0.9007, lr: 0.008353, batch_cost: 0.2015, reader_cost: 0.00065, ips: 14.8864 samples/sec | ETA 05:29:59
- 2022-04-13 12:46:13 [INFO] [TRAIN] epoch: 88, iter: 21800/120000, loss: 1.0161, lr: 0.008349, batch_cost: 0.2325, reader_cost: 0.00093, ips: 12.9006 samples/sec | ETA 06:20:36
- 2022-04-13 12:46:26 [INFO] [TRAIN] epoch: 89, iter: 21850/120000, loss: 0.8932, lr: 0.008345, batch_cost: 0.2607, reader_cost: 0.05247, ips: 11.5054 samples/sec | ETA 07:06:32
- 2022-04-13 12:46:36 [INFO] [TRAIN] epoch: 89, iter: 21900/120000, loss: 0.9294, lr: 0.008341, batch_cost: 0.2081, reader_cost: 0.00118, ips: 14.4182 samples/sec | ETA 05:40:11
- 2022-04-13 12:46:47 [INFO] [TRAIN] epoch: 89, iter: 21950/120000, loss: 0.9106, lr: 0.008338, batch_cost: 0.2062, reader_cost: 0.00048, ips: 14.5487 samples/sec | ETA 05:36:58
- 2022-04-13 12:46:58 [INFO] [TRAIN] epoch: 89, iter: 22000/120000, loss: 0.9275, lr: 0.008334, batch_cost: 0.2280, reader_cost: 0.00133, ips: 13.1565 samples/sec | ETA 06:12:26
- 2022-04-13 12:47:09 [INFO] [TRAIN] epoch: 89, iter: 22050/120000, loss: 0.9969, lr: 0.008330, batch_cost: 0.2106, reader_cost: 0.00169, ips: 14.2440 samples/sec | ETA 05:43:49
- 2022-04-13 12:47:22 [INFO] [TRAIN] epoch: 90, iter: 22100/120000, loss: 0.9273, lr: 0.008326, batch_cost: 0.2693, reader_cost: 0.04888, ips: 11.1411 samples/sec | ETA 07:19:21
- 2022-04-13 12:47:33 [INFO] [TRAIN] epoch: 90, iter: 22150/120000, loss: 0.9189, lr: 0.008322, batch_cost: 0.2140, reader_cost: 0.00042, ips: 14.0215 samples/sec | ETA 05:48:55
- 2022-04-13 12:47:44 [INFO] [TRAIN] epoch: 90, iter: 22200/120000, loss: 0.9004, lr: 0.008319, batch_cost: 0.2220, reader_cost: 0.00101, ips: 13.5135 samples/sec | ETA 06:01:51
- 2022-04-13 12:47:55 [INFO] [TRAIN] epoch: 90, iter: 22250/120000, loss: 0.9455, lr: 0.008315, batch_cost: 0.2153, reader_cost: 0.00119, ips: 13.9328 samples/sec | ETA 05:50:47
- 2022-04-13 12:48:05 [INFO] [TRAIN] epoch: 90, iter: 22300/120000, loss: 0.9388, lr: 0.008311, batch_cost: 0.2006, reader_cost: 0.00068, ips: 14.9546 samples/sec | ETA 05:26:39
- 2022-04-13 12:48:18 [INFO] [TRAIN] epoch: 91, iter: 22350/120000, loss: 0.9354, lr: 0.008307, batch_cost: 0.2651, reader_cost: 0.05966, ips: 11.3172 samples/sec | ETA 07:11:25
- 2022-04-13 12:48:28 [INFO] [TRAIN] epoch: 91, iter: 22400/120000, loss: 0.9580, lr: 0.008303, batch_cost: 0.2114, reader_cost: 0.00105, ips: 14.1878 samples/sec | ETA 05:43:57
- 2022-04-13 12:48:40 [INFO] [TRAIN] epoch: 91, iter: 22450/120000, loss: 0.9253, lr: 0.008299, batch_cost: 0.2264, reader_cost: 0.00127, ips: 13.2495 samples/sec | ETA 06:08:07
- 2022-04-13 12:48:50 [INFO] [TRAIN] epoch: 91, iter: 22500/120000, loss: 0.9023, lr: 0.008296, batch_cost: 0.2049, reader_cost: 0.00090, ips: 14.6386 samples/sec | ETA 05:33:01
- 2022-04-13 12:49:00 [INFO] [TRAIN] epoch: 91, iter: 22550/120000, loss: 0.8944, lr: 0.008292, batch_cost: 0.2065, reader_cost: 0.00155, ips: 14.5266 samples/sec | ETA 05:35:25
- 2022-04-13 12:49:14 [INFO] [TRAIN] epoch: 92, iter: 22600/120000, loss: 0.9158, lr: 0.008288, batch_cost: 0.2658, reader_cost: 0.05770, ips: 11.2877 samples/sec | ETA 07:11:26
- 2022-04-13 12:49:25 [INFO] [TRAIN] epoch: 92, iter: 22650/120000, loss: 0.9261, lr: 0.008284, batch_cost: 0.2166, reader_cost: 0.00091, ips: 13.8511 samples/sec | ETA 05:51:24
- 2022-04-13 12:49:35 [INFO] [TRAIN] epoch: 92, iter: 22700/120000, loss: 0.9692, lr: 0.008280, batch_cost: 0.2041, reader_cost: 0.00068, ips: 14.7010 samples/sec | ETA 05:30:55
- 2022-04-13 12:49:45 [INFO] [TRAIN] epoch: 92, iter: 22750/120000, loss: 0.9114, lr: 0.008276, batch_cost: 0.2008, reader_cost: 0.00092, ips: 14.9421 samples/sec | ETA 05:25:25
- 2022-04-13 12:49:56 [INFO] [TRAIN] epoch: 92, iter: 22800/120000, loss: 0.9642, lr: 0.008273, batch_cost: 0.2272, reader_cost: 0.00082, ips: 13.2032 samples/sec | ETA 06:08:05
- 2022-04-13 12:50:10 [INFO] [TRAIN] epoch: 93, iter: 22850/120000, loss: 0.9306, lr: 0.008269, batch_cost: 0.2786, reader_cost: 0.06481, ips: 10.7688 samples/sec | ETA 07:31:04
- 2022-04-13 12:50:20 [INFO] [TRAIN] epoch: 93, iter: 22900/120000, loss: 0.8897, lr: 0.008265, batch_cost: 0.2060, reader_cost: 0.00099, ips: 14.5614 samples/sec | ETA 05:33:24
- 2022-04-13 12:50:30 [INFO] [TRAIN] epoch: 93, iter: 22950/120000, loss: 0.9128, lr: 0.008261, batch_cost: 0.1992, reader_cost: 0.00070, ips: 15.0620 samples/sec | ETA 05:22:10
- 2022-04-13 12:50:41 [INFO] [TRAIN] epoch: 93, iter: 23000/120000, loss: 0.9478, lr: 0.008257, batch_cost: 0.2228, reader_cost: 0.00043, ips: 13.4635 samples/sec | ETA 06:00:14
- 2022-04-13 12:50:52 [INFO] [TRAIN] epoch: 93, iter: 23050/120000, loss: 0.9731, lr: 0.008253, batch_cost: 0.2136, reader_cost: 0.00068, ips: 14.0437 samples/sec | ETA 05:45:10
- 2022-04-13 12:51:06 [INFO] [TRAIN] epoch: 94, iter: 23100/120000, loss: 0.9090, lr: 0.008250, batch_cost: 0.2778, reader_cost: 0.06502, ips: 10.7988 samples/sec | ETA 07:28:39
- 2022-04-13 12:51:16 [INFO] [TRAIN] epoch: 94, iter: 23150/120000, loss: 0.9356, lr: 0.008246, batch_cost: 0.2025, reader_cost: 0.00142, ips: 14.8146 samples/sec | ETA 05:26:52
- 2022-04-13 12:51:27 [INFO] [TRAIN] epoch: 94, iter: 23200/120000, loss: 0.9730, lr: 0.008242, batch_cost: 0.2172, reader_cost: 0.00055, ips: 13.8109 samples/sec | ETA 05:50:26
- 2022-04-13 12:51:38 [INFO] [TRAIN] epoch: 94, iter: 23250/120000, loss: 0.9292, lr: 0.008238, batch_cost: 0.2167, reader_cost: 0.00074, ips: 13.8409 samples/sec | ETA 05:49:30
- 2022-04-13 12:51:48 [INFO] [TRAIN] epoch: 94, iter: 23300/120000, loss: 0.9489, lr: 0.008234, batch_cost: 0.2080, reader_cost: 0.00075, ips: 14.4212 samples/sec | ETA 05:35:16
- 2022-04-13 12:52:02 [INFO] [TRAIN] epoch: 95, iter: 23350/120000, loss: 0.9120, lr: 0.008230, batch_cost: 0.2757, reader_cost: 0.05472, ips: 10.8821 samples/sec | ETA 07:24:04
- 2022-04-13 12:52:12 [INFO] [TRAIN] epoch: 95, iter: 23400/120000, loss: 0.9589, lr: 0.008227, batch_cost: 0.2058, reader_cost: 0.00082, ips: 14.5801 samples/sec | ETA 05:31:16
- 2022-04-13 12:52:23 [INFO] [TRAIN] epoch: 95, iter: 23450/120000, loss: 0.9542, lr: 0.008223, batch_cost: 0.2162, reader_cost: 0.00063, ips: 13.8779 samples/sec | ETA 05:47:51
- 2022-04-13 12:52:33 [INFO] [TRAIN] epoch: 95, iter: 23500/120000, loss: 0.9364, lr: 0.008219, batch_cost: 0.2068, reader_cost: 0.00092, ips: 14.5103 samples/sec | ETA 05:32:31
- 2022-04-13 12:52:43 [INFO] [TRAIN] epoch: 95, iter: 23550/120000, loss: 0.9581, lr: 0.008215, batch_cost: 0.1990, reader_cost: 0.00100, ips: 15.0725 samples/sec | ETA 05:19:57
- 2022-04-13 12:52:57 [INFO] [TRAIN] epoch: 96, iter: 23600/120000, loss: 0.9414, lr: 0.008211, batch_cost: 0.2741, reader_cost: 0.05363, ips: 10.9442 samples/sec | ETA 07:20:24
- 2022-04-13 12:53:07 [INFO] [TRAIN] epoch: 96, iter: 23650/120000, loss: 0.9838, lr: 0.008207, batch_cost: 0.2006, reader_cost: 0.00088, ips: 14.9542 samples/sec | ETA 05:22:09
- 2022-04-13 12:53:17 [INFO] [TRAIN] epoch: 96, iter: 23700/120000, loss: 0.9332, lr: 0.008204, batch_cost: 0.2001, reader_cost: 0.00115, ips: 14.9909 samples/sec | ETA 05:21:11
- 2022-04-13 12:53:28 [INFO] [TRAIN] epoch: 96, iter: 23750/120000, loss: 0.9716, lr: 0.008200, batch_cost: 0.2170, reader_cost: 0.00074, ips: 13.8230 samples/sec | ETA 05:48:09
- 2022-04-13 12:53:39 [INFO] [TRAIN] epoch: 96, iter: 23800/120000, loss: 0.9392, lr: 0.008196, batch_cost: 0.2165, reader_cost: 0.00149, ips: 13.8563 samples/sec | ETA 05:47:08
- 2022-04-13 12:53:52 [INFO] [TRAIN] epoch: 97, iter: 23850/120000, loss: 0.9406, lr: 0.008192, batch_cost: 0.2656, reader_cost: 0.04652, ips: 11.2965 samples/sec | ETA 07:05:34
- 2022-04-13 12:54:03 [INFO] [TRAIN] epoch: 97, iter: 23900/120000, loss: 0.9514, lr: 0.008188, batch_cost: 0.2195, reader_cost: 0.00092, ips: 13.6649 samples/sec | ETA 05:51:37
- 2022-04-13 12:54:15 [INFO] [TRAIN] epoch: 97, iter: 23950/120000, loss: 0.9383, lr: 0.008184, batch_cost: 0.2291, reader_cost: 0.00099, ips: 13.0974 samples/sec | ETA 06:06:40
- 2022-04-13 12:54:25 [INFO] [TRAIN] epoch: 97, iter: 24000/120000, loss: 0.9358, lr: 0.008181, batch_cost: 0.2042, reader_cost: 0.00173, ips: 14.6925 samples/sec | ETA 05:26:41
- 2022-04-13 12:54:25 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1963 - reader cost: 0.0669
- 2022-04-13 12:54:50 [INFO] [EVAL] #Images: 500 mIoU: 0.7037 Acc: 0.9483 Kappa: 0.9327 Dice: 0.8113
- 2022-04-13 12:54:50 [INFO] [EVAL] Class IoU:
- [0.974 0.8041 0.9006 0.288 0.5427 0.5614 0.6619 0.7481 0.9107 0.5089
- 0.9378 0.7565 0.5778 0.9273 0.5875 0.8149 0.6879 0.4419 0.7375]
- 2022-04-13 12:54:50 [INFO] [EVAL] Class Precision:
- [0.9857 0.9036 0.9353 0.8668 0.7059 0.8265 0.7904 0.898 0.9476 0.8504
- 0.9645 0.8025 0.6848 0.9469 0.7703 0.9017 0.9065 0.7456 0.8364]
- 2022-04-13 12:54:50 [INFO] [EVAL] Class Recall:
- [0.9879 0.8796 0.9604 0.3014 0.7012 0.6364 0.8028 0.8175 0.9591 0.559
- 0.9713 0.9296 0.7871 0.9782 0.7123 0.8943 0.7405 0.5204 0.8618]
- 2022-04-13 12:54:50 [INFO] [EVAL] The model with the best validation mIoU (0.7155) was saved at iter 12000.
- 2022-04-13 12:55:01 [INFO] [TRAIN] epoch: 97, iter: 24050/120000, loss: 0.9545, lr: 0.008177, batch_cost: 0.2171, reader_cost: 0.00135, ips: 13.8185 samples/sec | ETA 05:47:10
- 2022-04-13 12:55:14 [INFO] [TRAIN] epoch: 98, iter: 24100/120000, loss: 0.9056, lr: 0.008173, batch_cost: 0.2705, reader_cost: 0.05821, ips: 11.0904 samples/sec | ETA 07:12:21
- 2022-04-13 12:55:25 [INFO] [TRAIN] epoch: 98, iter: 24150/120000, loss: 0.9135, lr: 0.008169, batch_cost: 0.2034, reader_cost: 0.00079, ips: 14.7525 samples/sec | ETA 05:24:51
- 2022-04-13 12:55:35 [INFO] [TRAIN] epoch: 98, iter: 24200/120000, loss: 0.9398, lr: 0.008165, batch_cost: 0.2045, reader_cost: 0.00064, ips: 14.6664 samples/sec | ETA 05:26:35
- 2022-04-13 12:55:45 [INFO] [TRAIN] epoch: 98, iter: 24250/120000, loss: 0.9269, lr: 0.008161, batch_cost: 0.2093, reader_cost: 0.00098, ips: 14.3326 samples/sec | ETA 05:34:01
- 2022-04-13 12:55:55 [INFO] [TRAIN] epoch: 98, iter: 24300/120000, loss: 0.9491, lr: 0.008158, batch_cost: 0.1970, reader_cost: 0.00044, ips: 15.2269 samples/sec | ETA 05:14:14
- 2022-04-13 12:56:09 [INFO] [TRAIN] epoch: 99, iter: 24350/120000, loss: 0.9367, lr: 0.008154, batch_cost: 0.2757, reader_cost: 0.05812, ips: 10.8795 samples/sec | ETA 07:19:35
- 2022-04-13 12:56:20 [INFO] [TRAIN] epoch: 99, iter: 24400/120000, loss: 0.9634, lr: 0.008150, batch_cost: 0.2125, reader_cost: 0.00085, ips: 14.1200 samples/sec | ETA 05:38:31
- 2022-04-13 12:56:30 [INFO] [TRAIN] epoch: 99, iter: 24450/120000, loss: 0.9146, lr: 0.008146, batch_cost: 0.2169, reader_cost: 0.00111, ips: 13.8317 samples/sec | ETA 05:45:24
- 2022-04-13 12:56:42 [INFO] [TRAIN] epoch: 99, iter: 24500/120000, loss: 0.9052, lr: 0.008142, batch_cost: 0.2317, reader_cost: 0.00110, ips: 12.9485 samples/sec | ETA 06:08:46
- 2022-04-13 12:56:52 [INFO] [TRAIN] epoch: 99, iter: 24550/120000, loss: 0.9474, lr: 0.008138, batch_cost: 0.2030, reader_cost: 0.00076, ips: 14.7755 samples/sec | ETA 05:23:00
- 2022-04-13 12:57:07 [INFO] [TRAIN] epoch: 100, iter: 24600/120000, loss: 0.8964, lr: 0.008135, batch_cost: 0.2914, reader_cost: 0.05506, ips: 10.2956 samples/sec | ETA 07:43:18
- 2022-04-13 12:57:18 [INFO] [TRAIN] epoch: 100, iter: 24650/120000, loss: 0.9428, lr: 0.008131, batch_cost: 0.2297, reader_cost: 0.00116, ips: 13.0624 samples/sec | ETA 06:04:58
- 2022-04-13 12:57:28 [INFO] [TRAIN] epoch: 100, iter: 24700/120000, loss: 0.9247, lr: 0.008127, batch_cost: 0.2044, reader_cost: 0.00058, ips: 14.6743 samples/sec | ETA 05:24:43
- 2022-04-13 12:57:40 [INFO] [TRAIN] epoch: 100, iter: 24750/120000, loss: 0.9432, lr: 0.008123, batch_cost: 0.2213, reader_cost: 0.00096, ips: 13.5563 samples/sec | ETA 05:51:18
- 2022-04-13 12:57:49 [INFO] [TRAIN] epoch: 100, iter: 24800/120000, loss: 0.9406, lr: 0.008119, batch_cost: 0.1956, reader_cost: 0.00090, ips: 15.3352 samples/sec | ETA 05:10:23
- 2022-04-13 12:58:04 [INFO] [TRAIN] epoch: 101, iter: 24850/120000, loss: 0.9229, lr: 0.008115, batch_cost: 0.2861, reader_cost: 0.05873, ips: 10.4866 samples/sec | ETA 07:33:40
- 2022-04-13 12:58:14 [INFO] [TRAIN] epoch: 101, iter: 24900/120000, loss: 0.9559, lr: 0.008112, batch_cost: 0.2062, reader_cost: 0.00086, ips: 14.5496 samples/sec | ETA 05:26:48
- 2022-04-13 12:58:25 [INFO] [TRAIN] epoch: 101, iter: 24950/120000, loss: 0.9272, lr: 0.008108, batch_cost: 0.2177, reader_cost: 0.00065, ips: 13.7818 samples/sec | ETA 05:44:50
- 2022-04-13 12:58:35 [INFO] [TRAIN] epoch: 101, iter: 25000/120000, loss: 0.9282, lr: 0.008104, batch_cost: 0.2063, reader_cost: 0.00090, ips: 14.5411 samples/sec | ETA 05:26:39
- 2022-04-13 12:58:48 [INFO] [TRAIN] epoch: 102, iter: 25050/120000, loss: 0.9097, lr: 0.008100, batch_cost: 0.2611, reader_cost: 0.05050, ips: 11.4911 samples/sec | ETA 06:53:08
- 2022-04-13 12:59:00 [INFO] [TRAIN] epoch: 102, iter: 25100/120000, loss: 0.8928, lr: 0.008096, batch_cost: 0.2276, reader_cost: 0.00121, ips: 13.1835 samples/sec | ETA 05:59:55
- 2022-04-13 12:59:11 [INFO] [TRAIN] epoch: 102, iter: 25150/120000, loss: 0.9397, lr: 0.008092, batch_cost: 0.2260, reader_cost: 0.00104, ips: 13.2761 samples/sec | ETA 05:57:13
- 2022-04-13 12:59:22 [INFO] [TRAIN] epoch: 102, iter: 25200/120000, loss: 0.9445, lr: 0.008089, batch_cost: 0.2161, reader_cost: 0.00075, ips: 13.8816 samples/sec | ETA 05:41:27
- 2022-04-13 12:59:32 [INFO] [TRAIN] epoch: 102, iter: 25250/120000, loss: 0.9562, lr: 0.008085, batch_cost: 0.2022, reader_cost: 0.00068, ips: 14.8387 samples/sec | ETA 05:19:15
- 2022-04-13 12:59:45 [INFO] [TRAIN] epoch: 103, iter: 25300/120000, loss: 0.9541, lr: 0.008081, batch_cost: 0.2615, reader_cost: 0.05848, ips: 11.4717 samples/sec | ETA 06:52:45
- 2022-04-13 12:59:56 [INFO] [TRAIN] epoch: 103, iter: 25350/120000, loss: 0.9171, lr: 0.008077, batch_cost: 0.2219, reader_cost: 0.00075, ips: 13.5169 samples/sec | ETA 05:50:07
- 2022-04-13 13:00:06 [INFO] [TRAIN] epoch: 103, iter: 25400/120000, loss: 0.9676, lr: 0.008073, batch_cost: 0.2044, reader_cost: 0.00069, ips: 14.6762 samples/sec | ETA 05:22:17
- 2022-04-13 13:00:16 [INFO] [TRAIN] epoch: 103, iter: 25450/120000, loss: 0.9660, lr: 0.008069, batch_cost: 0.2055, reader_cost: 0.00042, ips: 14.5961 samples/sec | ETA 05:23:53
- 2022-04-13 13:00:27 [INFO] [TRAIN] epoch: 103, iter: 25500/120000, loss: 0.9417, lr: 0.008065, batch_cost: 0.2170, reader_cost: 0.00143, ips: 13.8233 samples/sec | ETA 05:41:48
- 2022-04-13 13:00:41 [INFO] [TRAIN] epoch: 104, iter: 25550/120000, loss: 0.9203, lr: 0.008062, batch_cost: 0.2680, reader_cost: 0.05107, ips: 11.1952 samples/sec | ETA 07:01:49
- 2022-04-13 13:00:52 [INFO] [TRAIN] epoch: 104, iter: 25600/120000, loss: 0.9296, lr: 0.008058, batch_cost: 0.2197, reader_cost: 0.00075, ips: 13.6545 samples/sec | ETA 05:45:40
- 2022-04-13 13:01:03 [INFO] [TRAIN] epoch: 104, iter: 25650/120000, loss: 0.9324, lr: 0.008054, batch_cost: 0.2277, reader_cost: 0.00158, ips: 13.1761 samples/sec | ETA 05:58:02
- 2022-04-13 13:01:13 [INFO] [TRAIN] epoch: 104, iter: 25700/120000, loss: 0.8774, lr: 0.008050, batch_cost: 0.2028, reader_cost: 0.00071, ips: 14.7960 samples/sec | ETA 05:18:40
- 2022-04-13 13:01:24 [INFO] [TRAIN] epoch: 104, iter: 25750/120000, loss: 0.9050, lr: 0.008046, batch_cost: 0.2192, reader_cost: 0.00052, ips: 13.6877 samples/sec | ETA 05:44:17
- 2022-04-13 13:01:39 [INFO] [TRAIN] epoch: 105, iter: 25800/120000, loss: 0.9171, lr: 0.008042, batch_cost: 0.2907, reader_cost: 0.06835, ips: 10.3205 samples/sec | ETA 07:36:22
- 2022-04-13 13:01:50 [INFO] [TRAIN] epoch: 105, iter: 25850/120000, loss: 0.9538, lr: 0.008039, batch_cost: 0.2182, reader_cost: 0.00207, ips: 13.7483 samples/sec | ETA 05:42:24
- 2022-04-13 13:02:00 [INFO] [TRAIN] epoch: 105, iter: 25900/120000, loss: 0.9796, lr: 0.008035, batch_cost: 0.2040, reader_cost: 0.00080, ips: 14.7036 samples/sec | ETA 05:19:59
- 2022-04-13 13:02:12 [INFO] [TRAIN] epoch: 105, iter: 25950/120000, loss: 0.9443, lr: 0.008031, batch_cost: 0.2337, reader_cost: 0.00068, ips: 12.8371 samples/sec | ETA 06:06:19
- 2022-04-13 13:02:22 [INFO] [TRAIN] epoch: 105, iter: 26000/120000, loss: 0.9158, lr: 0.008027, batch_cost: 0.2161, reader_cost: 0.00106, ips: 13.8804 samples/sec | ETA 05:38:36
- 2022-04-13 13:02:36 [INFO] [TRAIN] epoch: 106, iter: 26050/120000, loss: 0.9204, lr: 0.008023, batch_cost: 0.2810, reader_cost: 0.05789, ips: 10.6747 samples/sec | ETA 07:20:03
- 2022-04-13 13:02:46 [INFO] [TRAIN] epoch: 106, iter: 26100/120000, loss: 0.8994, lr: 0.008019, batch_cost: 0.2003, reader_cost: 0.00112, ips: 14.9763 samples/sec | ETA 05:13:29
- 2022-04-13 13:02:57 [INFO] [TRAIN] epoch: 106, iter: 26150/120000, loss: 0.9266, lr: 0.008016, batch_cost: 0.2139, reader_cost: 0.00096, ips: 14.0241 samples/sec | ETA 05:34:36
- 2022-04-13 13:03:07 [INFO] [TRAIN] epoch: 106, iter: 26200/120000, loss: 0.8925, lr: 0.008012, batch_cost: 0.2064, reader_cost: 0.00068, ips: 14.5342 samples/sec | ETA 05:22:41
- 2022-04-13 13:03:18 [INFO] [TRAIN] epoch: 106, iter: 26250/120000, loss: 0.9475, lr: 0.008008, batch_cost: 0.2153, reader_cost: 0.00067, ips: 13.9349 samples/sec | ETA 05:36:23
- 2022-04-13 13:03:31 [INFO] [TRAIN] epoch: 107, iter: 26300/120000, loss: 0.9278, lr: 0.008004, batch_cost: 0.2598, reader_cost: 0.05257, ips: 11.5454 samples/sec | ETA 06:45:47
- 2022-04-13 13:03:42 [INFO] [TRAIN] epoch: 107, iter: 26350/120000, loss: 0.9512, lr: 0.008000, batch_cost: 0.2175, reader_cost: 0.00138, ips: 13.7901 samples/sec | ETA 05:39:33
- 2022-04-13 13:03:53 [INFO] [TRAIN] epoch: 107, iter: 26400/120000, loss: 0.9566, lr: 0.007996, batch_cost: 0.2189, reader_cost: 0.00068, ips: 13.7056 samples/sec | ETA 05:41:27
- 2022-04-13 13:04:05 [INFO] [TRAIN] epoch: 107, iter: 26450/120000, loss: 0.9308, lr: 0.007992, batch_cost: 0.2300, reader_cost: 0.00132, ips: 13.0424 samples/sec | ETA 05:58:38
- 2022-04-13 13:04:15 [INFO] [TRAIN] epoch: 107, iter: 26500/120000, loss: 0.9154, lr: 0.007989, batch_cost: 0.2097, reader_cost: 0.00115, ips: 14.3086 samples/sec | ETA 05:26:43
- 2022-04-13 13:04:29 [INFO] [TRAIN] epoch: 108, iter: 26550/120000, loss: 0.9090, lr: 0.007985, batch_cost: 0.2756, reader_cost: 0.05517, ips: 10.8845 samples/sec | ETA 07:09:16
- 2022-04-13 13:04:40 [INFO] [TRAIN] epoch: 108, iter: 26600/120000, loss: 0.9397, lr: 0.007981, batch_cost: 0.2172, reader_cost: 0.00141, ips: 13.8111 samples/sec | ETA 05:38:08
- 2022-04-13 13:04:51 [INFO] [TRAIN] epoch: 108, iter: 26650/120000, loss: 0.9113, lr: 0.007977, batch_cost: 0.2233, reader_cost: 0.00223, ips: 13.4335 samples/sec | ETA 05:47:27
- 2022-04-13 13:05:01 [INFO] [TRAIN] epoch: 108, iter: 26700/120000, loss: 0.9154, lr: 0.007973, batch_cost: 0.2101, reader_cost: 0.00073, ips: 14.2790 samples/sec | ETA 05:26:42
- 2022-04-13 13:05:11 [INFO] [TRAIN] epoch: 108, iter: 26750/120000, loss: 0.9241, lr: 0.007969, batch_cost: 0.1999, reader_cost: 0.00068, ips: 15.0084 samples/sec | ETA 05:10:39
- 2022-04-13 13:05:24 [INFO] [TRAIN] epoch: 109, iter: 26800/120000, loss: 1.0134, lr: 0.007966, batch_cost: 0.2620, reader_cost: 0.05938, ips: 11.4502 samples/sec | ETA 06:46:58
- 2022-04-13 13:05:35 [INFO] [TRAIN] epoch: 109, iter: 26850/120000, loss: 0.9372, lr: 0.007962, batch_cost: 0.2172, reader_cost: 0.00078, ips: 13.8148 samples/sec | ETA 05:37:08
- 2022-04-13 13:05:46 [INFO] [TRAIN] epoch: 109, iter: 26900/120000, loss: 0.9130, lr: 0.007958, batch_cost: 0.2093, reader_cost: 0.00118, ips: 14.3332 samples/sec | ETA 05:24:46
- 2022-04-13 13:05:57 [INFO] [TRAIN] epoch: 109, iter: 26950/120000, loss: 0.9143, lr: 0.007954, batch_cost: 0.2166, reader_cost: 0.00126, ips: 13.8496 samples/sec | ETA 05:35:55
- 2022-04-13 13:06:08 [INFO] [TRAIN] epoch: 109, iter: 27000/120000, loss: 0.9272, lr: 0.007950, batch_cost: 0.2207, reader_cost: 0.00102, ips: 13.5947 samples/sec | ETA 05:42:02
- 2022-04-13 13:06:21 [INFO] [TRAIN] epoch: 110, iter: 27050/120000, loss: 0.9029, lr: 0.007946, batch_cost: 0.2646, reader_cost: 0.05759, ips: 11.3368 samples/sec | ETA 06:49:56
- 2022-04-13 13:06:32 [INFO] [TRAIN] epoch: 110, iter: 27100/120000, loss: 0.9028, lr: 0.007942, batch_cost: 0.2124, reader_cost: 0.00153, ips: 14.1211 samples/sec | ETA 05:28:56
- 2022-04-13 13:06:42 [INFO] [TRAIN] epoch: 110, iter: 27150/120000, loss: 0.9331, lr: 0.007939, batch_cost: 0.2023, reader_cost: 0.00156, ips: 14.8299 samples/sec | ETA 05:13:02
- 2022-04-13 13:06:53 [INFO] [TRAIN] epoch: 110, iter: 27200/120000, loss: 0.9516, lr: 0.007935, batch_cost: 0.2218, reader_cost: 0.00086, ips: 13.5271 samples/sec | ETA 05:43:00
- 2022-04-13 13:07:03 [INFO] [TRAIN] epoch: 110, iter: 27250/120000, loss: 0.8955, lr: 0.007931, batch_cost: 0.2144, reader_cost: 0.00081, ips: 13.9915 samples/sec | ETA 05:31:27
- 2022-04-13 13:07:17 [INFO] [TRAIN] epoch: 111, iter: 27300/120000, loss: 0.9188, lr: 0.007927, batch_cost: 0.2654, reader_cost: 0.05509, ips: 11.3053 samples/sec | ETA 06:49:58
- 2022-04-13 13:07:28 [INFO] [TRAIN] epoch: 111, iter: 27350/120000, loss: 0.9074, lr: 0.007923, batch_cost: 0.2270, reader_cost: 0.00176, ips: 13.2176 samples/sec | ETA 05:50:28
- 2022-04-13 13:07:39 [INFO] [TRAIN] epoch: 111, iter: 27400/120000, loss: 0.9430, lr: 0.007919, batch_cost: 0.2095, reader_cost: 0.00085, ips: 14.3215 samples/sec | ETA 05:23:17
- 2022-04-13 13:07:49 [INFO] [TRAIN] epoch: 111, iter: 27450/120000, loss: 0.8881, lr: 0.007916, batch_cost: 0.2123, reader_cost: 0.00126, ips: 14.1334 samples/sec | ETA 05:27:24
- 2022-04-13 13:08:00 [INFO] [TRAIN] epoch: 111, iter: 27500/120000, loss: 0.9138, lr: 0.007912, batch_cost: 0.2260, reader_cost: 0.00068, ips: 13.2759 samples/sec | ETA 05:48:22
- 2022-04-13 13:08:14 [INFO] [TRAIN] epoch: 112, iter: 27550/120000, loss: 0.8872, lr: 0.007908, batch_cost: 0.2702, reader_cost: 0.04812, ips: 11.1021 samples/sec | ETA 06:56:21
- 2022-04-13 13:08:25 [INFO] [TRAIN] epoch: 112, iter: 27600/120000, loss: 0.9215, lr: 0.007904, batch_cost: 0.2128, reader_cost: 0.00112, ips: 14.0976 samples/sec | ETA 05:27:42
- 2022-04-13 13:08:36 [INFO] [TRAIN] epoch: 112, iter: 27650/120000, loss: 0.9020, lr: 0.007900, batch_cost: 0.2209, reader_cost: 0.00074, ips: 13.5786 samples/sec | ETA 05:40:03
- 2022-04-13 13:08:46 [INFO] [TRAIN] epoch: 112, iter: 27700/120000, loss: 0.9401, lr: 0.007896, batch_cost: 0.2058, reader_cost: 0.00083, ips: 14.5807 samples/sec | ETA 05:16:30
- 2022-04-13 13:08:56 [INFO] [TRAIN] epoch: 112, iter: 27750/120000, loss: 0.9595, lr: 0.007892, batch_cost: 0.2103, reader_cost: 0.00116, ips: 14.2664 samples/sec | ETA 05:23:18
- 2022-04-13 13:09:10 [INFO] [TRAIN] epoch: 113, iter: 27800/120000, loss: 0.9190, lr: 0.007889, batch_cost: 0.2784, reader_cost: 0.06076, ips: 10.7742 samples/sec | ETA 07:07:52
- 2022-04-13 13:09:21 [INFO] [TRAIN] epoch: 113, iter: 27850/120000, loss: 0.8897, lr: 0.007885, batch_cost: 0.2073, reader_cost: 0.00042, ips: 14.4735 samples/sec | ETA 05:18:20
- 2022-04-13 13:09:34 [INFO] [TRAIN] epoch: 113, iter: 27900/120000, loss: 0.9058, lr: 0.007881, batch_cost: 0.2643, reader_cost: 0.00055, ips: 11.3512 samples/sec | ETA 06:45:41
- 2022-04-13 13:09:45 [INFO] [TRAIN] epoch: 113, iter: 27950/120000, loss: 0.9164, lr: 0.007877, batch_cost: 0.2224, reader_cost: 0.00072, ips: 13.4863 samples/sec | ETA 05:41:16
- 2022-04-13 13:09:56 [INFO] [TRAIN] epoch: 113, iter: 28000/120000, loss: 0.8969, lr: 0.007873, batch_cost: 0.2218, reader_cost: 0.00147, ips: 13.5250 samples/sec | ETA 05:40:06
- 2022-04-13 13:09:56 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1968 - reader cost: 0.1083
- 2022-04-13 13:10:21 [INFO] [EVAL] #Images: 500 mIoU: 0.7276 Acc: 0.9516 Kappa: 0.9373 Dice: 0.8318
- 2022-04-13 13:10:21 [INFO] [EVAL] Class IoU:
- [0.9766 0.8138 0.9126 0.4438 0.5349 0.599 0.6374 0.7519 0.913 0.5318
- 0.9411 0.7881 0.5617 0.9305 0.751 0.8204 0.691 0.497 0.7278]
- 2022-04-13 13:10:21 [INFO] [EVAL] Class Precision:
- [0.9901 0.8787 0.9573 0.785 0.6657 0.7496 0.7033 0.8566 0.9532 0.7594
- 0.9578 0.8605 0.7644 0.9557 0.8232 0.882 0.7714 0.6079 0.8725]
- 2022-04-13 13:10:21 [INFO] [EVAL] Class Recall:
- [0.9863 0.9168 0.9513 0.5052 0.7314 0.7488 0.8719 0.8601 0.9558 0.6395
- 0.9818 0.9035 0.6793 0.9725 0.8954 0.9215 0.869 0.7316 0.8144]
- 2022-04-13 13:10:22 [INFO] [EVAL] The model with the best validation mIoU (0.7276) was saved at iter 28000.
- 2022-04-13 13:10:35 [INFO] [TRAIN] epoch: 114, iter: 28050/120000, loss: 0.9080, lr: 0.007869, batch_cost: 0.2597, reader_cost: 0.05136, ips: 11.5539 samples/sec | ETA 06:37:55
- 2022-04-13 13:10:45 [INFO] [TRAIN] epoch: 114, iter: 28100/120000, loss: 0.9557, lr: 0.007865, batch_cost: 0.1964, reader_cost: 0.00103, ips: 15.2786 samples/sec | ETA 05:00:44
- 2022-04-13 13:10:56 [INFO] [TRAIN] epoch: 114, iter: 28150/120000, loss: 0.9259, lr: 0.007862, batch_cost: 0.2258, reader_cost: 0.00100, ips: 13.2888 samples/sec | ETA 05:45:35
- 2022-04-13 13:11:07 [INFO] [TRAIN] epoch: 114, iter: 28200/120000, loss: 0.9341, lr: 0.007858, batch_cost: 0.2198, reader_cost: 0.00115, ips: 13.6509 samples/sec | ETA 05:36:14
- 2022-04-13 13:11:18 [INFO] [TRAIN] epoch: 114, iter: 28250/120000, loss: 0.9516, lr: 0.007854, batch_cost: 0.2075, reader_cost: 0.00101, ips: 14.4582 samples/sec | ETA 05:17:17
- 2022-04-13 13:11:32 [INFO] [TRAIN] epoch: 115, iter: 28300/120000, loss: 0.9526, lr: 0.007850, batch_cost: 0.2920, reader_cost: 0.05935, ips: 10.2734 samples/sec | ETA 07:26:17
- 2022-04-13 13:11:43 [INFO] [TRAIN] epoch: 115, iter: 28350/120000, loss: 0.9375, lr: 0.007846, batch_cost: 0.2148, reader_cost: 0.00102, ips: 13.9662 samples/sec | ETA 05:28:06
- 2022-04-13 13:11:54 [INFO] [TRAIN] epoch: 115, iter: 28400/120000, loss: 0.9453, lr: 0.007842, batch_cost: 0.2175, reader_cost: 0.00110, ips: 13.7923 samples/sec | ETA 05:32:04
- 2022-04-13 13:12:05 [INFO] [TRAIN] epoch: 115, iter: 28450/120000, loss: 0.9159, lr: 0.007839, batch_cost: 0.2104, reader_cost: 0.00118, ips: 14.2567 samples/sec | ETA 05:21:04
- 2022-04-13 13:12:16 [INFO] [TRAIN] epoch: 115, iter: 28500/120000, loss: 0.9003, lr: 0.007835, batch_cost: 0.2257, reader_cost: 0.00103, ips: 13.2911 samples/sec | ETA 05:44:12
- 2022-04-13 13:12:30 [INFO] [TRAIN] epoch: 116, iter: 28550/120000, loss: 0.8876, lr: 0.007831, batch_cost: 0.2802, reader_cost: 0.05258, ips: 10.7062 samples/sec | ETA 07:07:05
- 2022-04-13 13:12:41 [INFO] [TRAIN] epoch: 116, iter: 28600/120000, loss: 0.9130, lr: 0.007827, batch_cost: 0.2151, reader_cost: 0.00138, ips: 13.9449 samples/sec | ETA 05:27:43
- 2022-04-13 13:12:51 [INFO] [TRAIN] epoch: 116, iter: 28650/120000, loss: 0.9275, lr: 0.007823, batch_cost: 0.2169, reader_cost: 0.00109, ips: 13.8343 samples/sec | ETA 05:30:09
- 2022-04-13 13:13:02 [INFO] [TRAIN] epoch: 116, iter: 28700/120000, loss: 0.8905, lr: 0.007819, batch_cost: 0.2104, reader_cost: 0.00089, ips: 14.2593 samples/sec | ETA 05:20:08
- 2022-04-13 13:13:13 [INFO] [TRAIN] epoch: 116, iter: 28750/120000, loss: 0.9013, lr: 0.007815, batch_cost: 0.2294, reader_cost: 0.00104, ips: 13.0750 samples/sec | ETA 05:48:56
- 2022-04-13 13:13:27 [INFO] [TRAIN] epoch: 117, iter: 28800/120000, loss: 0.9666, lr: 0.007812, batch_cost: 0.2659, reader_cost: 0.06010, ips: 11.2845 samples/sec | ETA 06:44:05
- 2022-04-13 13:13:37 [INFO] [TRAIN] epoch: 117, iter: 28850/120000, loss: 0.8884, lr: 0.007808, batch_cost: 0.2147, reader_cost: 0.00127, ips: 13.9715 samples/sec | ETA 05:26:11
- 2022-04-13 13:13:48 [INFO] [TRAIN] epoch: 117, iter: 28900/120000, loss: 0.9155, lr: 0.007804, batch_cost: 0.2093, reader_cost: 0.00078, ips: 14.3336 samples/sec | ETA 05:17:47
- 2022-04-13 13:13:59 [INFO] [TRAIN] epoch: 117, iter: 28950/120000, loss: 0.9333, lr: 0.007800, batch_cost: 0.2140, reader_cost: 0.00127, ips: 14.0178 samples/sec | ETA 05:24:45
- 2022-04-13 13:14:10 [INFO] [TRAIN] epoch: 117, iter: 29000/120000, loss: 0.8843, lr: 0.007796, batch_cost: 0.2183, reader_cost: 0.00055, ips: 13.7456 samples/sec | ETA 05:31:00
- 2022-04-13 13:14:23 [INFO] [TRAIN] epoch: 118, iter: 29050/120000, loss: 0.9160, lr: 0.007792, batch_cost: 0.2675, reader_cost: 0.05741, ips: 11.2146 samples/sec | ETA 06:45:29
- 2022-04-13 13:14:33 [INFO] [TRAIN] epoch: 118, iter: 29100/120000, loss: 0.9109, lr: 0.007788, batch_cost: 0.2102, reader_cost: 0.00156, ips: 14.2743 samples/sec | ETA 05:18:24
- 2022-04-13 13:14:45 [INFO] [TRAIN] epoch: 118, iter: 29150/120000, loss: 0.9242, lr: 0.007785, batch_cost: 0.2359, reader_cost: 0.00084, ips: 12.7187 samples/sec | ETA 05:57:09
- 2022-04-13 13:14:56 [INFO] [TRAIN] epoch: 118, iter: 29200/120000, loss: 0.9262, lr: 0.007781, batch_cost: 0.2180, reader_cost: 0.00129, ips: 13.7634 samples/sec | ETA 05:29:51
- 2022-04-13 13:15:07 [INFO] [TRAIN] epoch: 118, iter: 29250/120000, loss: 0.9147, lr: 0.007777, batch_cost: 0.2072, reader_cost: 0.00147, ips: 14.4820 samples/sec | ETA 05:13:19
- 2022-04-13 13:15:20 [INFO] [TRAIN] epoch: 119, iter: 29300/120000, loss: 0.9424, lr: 0.007773, batch_cost: 0.2717, reader_cost: 0.06194, ips: 11.0402 samples/sec | ETA 06:50:46
- 2022-04-13 13:15:30 [INFO] [TRAIN] epoch: 119, iter: 29350/120000, loss: 0.9516, lr: 0.007769, batch_cost: 0.2042, reader_cost: 0.00132, ips: 14.6932 samples/sec | ETA 05:08:28
- 2022-04-13 13:15:41 [INFO] [TRAIN] epoch: 119, iter: 29400/120000, loss: 0.9013, lr: 0.007765, batch_cost: 0.2210, reader_cost: 0.00117, ips: 13.5771 samples/sec | ETA 05:33:38
- 2022-04-13 13:15:52 [INFO] [TRAIN] epoch: 119, iter: 29450/120000, loss: 0.9489, lr: 0.007761, batch_cost: 0.2058, reader_cost: 0.00112, ips: 14.5770 samples/sec | ETA 05:10:35
- 2022-04-13 13:16:02 [INFO] [TRAIN] epoch: 119, iter: 29500/120000, loss: 0.9095, lr: 0.007758, batch_cost: 0.2023, reader_cost: 0.00126, ips: 14.8296 samples/sec | ETA 05:05:07
- 2022-04-13 13:16:15 [INFO] [TRAIN] epoch: 120, iter: 29550/120000, loss: 0.9144, lr: 0.007754, batch_cost: 0.2699, reader_cost: 0.06041, ips: 11.1137 samples/sec | ETA 06:46:55
- 2022-04-13 13:16:26 [INFO] [TRAIN] epoch: 120, iter: 29600/120000, loss: 0.8958, lr: 0.007750, batch_cost: 0.2084, reader_cost: 0.00132, ips: 14.3928 samples/sec | ETA 05:14:02
- 2022-04-13 13:16:36 [INFO] [TRAIN] epoch: 120, iter: 29650/120000, loss: 0.9060, lr: 0.007746, batch_cost: 0.2032, reader_cost: 0.00076, ips: 14.7653 samples/sec | ETA 05:05:57
- 2022-04-13 13:16:46 [INFO] [TRAIN] epoch: 120, iter: 29700/120000, loss: 0.9023, lr: 0.007742, batch_cost: 0.2091, reader_cost: 0.00185, ips: 14.3457 samples/sec | ETA 05:14:43
- 2022-04-13 13:16:57 [INFO] [TRAIN] epoch: 120, iter: 29750/120000, loss: 0.9709, lr: 0.007738, batch_cost: 0.2080, reader_cost: 0.00085, ips: 14.4251 samples/sec | ETA 05:12:49
- 2022-04-13 13:17:11 [INFO] [TRAIN] epoch: 121, iter: 29800/120000, loss: 0.9245, lr: 0.007734, batch_cost: 0.2816, reader_cost: 0.06140, ips: 10.6536 samples/sec | ETA 07:03:19
- 2022-04-13 13:17:21 [INFO] [TRAIN] epoch: 121, iter: 29850/120000, loss: 0.8899, lr: 0.007731, batch_cost: 0.2045, reader_cost: 0.00049, ips: 14.6722 samples/sec | ETA 05:07:12
- 2022-04-13 13:17:32 [INFO] [TRAIN] epoch: 121, iter: 29900/120000, loss: 0.9126, lr: 0.007727, batch_cost: 0.2264, reader_cost: 0.00044, ips: 13.2536 samples/sec | ETA 05:39:54
- 2022-04-13 13:17:43 [INFO] [TRAIN] epoch: 121, iter: 29950/120000, loss: 0.9106, lr: 0.007723, batch_cost: 0.2223, reader_cost: 0.00111, ips: 13.4956 samples/sec | ETA 05:33:37
- 2022-04-13 13:17:55 [INFO] [TRAIN] epoch: 121, iter: 30000/120000, loss: 0.9098, lr: 0.007719, batch_cost: 0.2274, reader_cost: 0.00040, ips: 13.1931 samples/sec | ETA 05:41:05
- 2022-04-13 13:18:08 [INFO] [TRAIN] epoch: 122, iter: 30050/120000, loss: 0.9104, lr: 0.007715, batch_cost: 0.2688, reader_cost: 0.05403, ips: 11.1595 samples/sec | ETA 06:43:01
- 2022-04-13 13:18:19 [INFO] [TRAIN] epoch: 122, iter: 30100/120000, loss: 0.9233, lr: 0.007711, batch_cost: 0.2128, reader_cost: 0.00074, ips: 14.0963 samples/sec | ETA 05:18:52
- 2022-04-13 13:18:30 [INFO] [TRAIN] epoch: 122, iter: 30150/120000, loss: 0.9112, lr: 0.007707, batch_cost: 0.2266, reader_cost: 0.00042, ips: 13.2381 samples/sec | ETA 05:39:21
- 2022-04-13 13:18:40 [INFO] [TRAIN] epoch: 122, iter: 30200/120000, loss: 0.9109, lr: 0.007704, batch_cost: 0.2009, reader_cost: 0.00095, ips: 14.9344 samples/sec | ETA 05:00:38
- 2022-04-13 13:18:50 [INFO] [TRAIN] epoch: 122, iter: 30250/120000, loss: 0.9156, lr: 0.007700, batch_cost: 0.1896, reader_cost: 0.00122, ips: 15.8242 samples/sec | ETA 04:43:35
- 2022-04-13 13:19:04 [INFO] [TRAIN] epoch: 123, iter: 30300/120000, loss: 0.9225, lr: 0.007696, batch_cost: 0.2876, reader_cost: 0.05543, ips: 10.4299 samples/sec | ETA 07:10:00
- 2022-04-13 13:19:15 [INFO] [TRAIN] epoch: 123, iter: 30350/120000, loss: 0.9370, lr: 0.007692, batch_cost: 0.2151, reader_cost: 0.00053, ips: 13.9477 samples/sec | ETA 05:21:22
- 2022-04-13 13:19:26 [INFO] [TRAIN] epoch: 123, iter: 30400/120000, loss: 0.9252, lr: 0.007688, batch_cost: 0.2168, reader_cost: 0.00131, ips: 13.8355 samples/sec | ETA 05:23:48
- 2022-04-13 13:19:37 [INFO] [TRAIN] epoch: 123, iter: 30450/120000, loss: 0.9132, lr: 0.007684, batch_cost: 0.2180, reader_cost: 0.00127, ips: 13.7624 samples/sec | ETA 05:25:20
- 2022-04-13 13:19:47 [INFO] [TRAIN] epoch: 123, iter: 30500/120000, loss: 0.9394, lr: 0.007680, batch_cost: 0.2053, reader_cost: 0.00063, ips: 14.6144 samples/sec | ETA 05:06:12
- 2022-04-13 13:20:02 [INFO] [TRAIN] epoch: 124, iter: 30550/120000, loss: 0.9265, lr: 0.007677, batch_cost: 0.2939, reader_cost: 0.05779, ips: 10.2065 samples/sec | ETA 07:18:12
- 2022-04-13 13:20:12 [INFO] [TRAIN] epoch: 124, iter: 30600/120000, loss: 0.9040, lr: 0.007673, batch_cost: 0.2127, reader_cost: 0.00205, ips: 14.1072 samples/sec | ETA 05:16:51
- 2022-04-13 13:20:22 [INFO] [TRAIN] epoch: 124, iter: 30650/120000, loss: 0.9311, lr: 0.007669, batch_cost: 0.1983, reader_cost: 0.00067, ips: 15.1311 samples/sec | ETA 04:55:15
- 2022-04-13 13:20:33 [INFO] [TRAIN] epoch: 124, iter: 30700/120000, loss: 0.9140, lr: 0.007665, batch_cost: 0.2163, reader_cost: 0.00051, ips: 13.8681 samples/sec | ETA 05:21:57
- 2022-04-13 13:20:43 [INFO] [TRAIN] epoch: 124, iter: 30750/120000, loss: 0.9418, lr: 0.007661, batch_cost: 0.2050, reader_cost: 0.00115, ips: 14.6376 samples/sec | ETA 05:04:51
- 2022-04-13 13:20:57 [INFO] [TRAIN] epoch: 125, iter: 30800/120000, loss: 0.9229, lr: 0.007657, batch_cost: 0.2824, reader_cost: 0.05728, ips: 10.6229 samples/sec | ETA 06:59:50
- 2022-04-13 13:21:10 [INFO] [TRAIN] epoch: 125, iter: 30850/120000, loss: 0.9258, lr: 0.007653, batch_cost: 0.2500, reader_cost: 0.00094, ips: 11.9998 samples/sec | ETA 06:11:27
- 2022-04-13 13:21:21 [INFO] [TRAIN] epoch: 125, iter: 30900/120000, loss: 0.8989, lr: 0.007649, batch_cost: 0.2290, reader_cost: 0.00104, ips: 13.1018 samples/sec | ETA 05:40:01
- 2022-04-13 13:21:32 [INFO] [TRAIN] epoch: 125, iter: 30950/120000, loss: 0.9098, lr: 0.007646, batch_cost: 0.2041, reader_cost: 0.00086, ips: 14.6959 samples/sec | ETA 05:02:58
- 2022-04-13 13:21:42 [INFO] [TRAIN] epoch: 125, iter: 31000/120000, loss: 0.9453, lr: 0.007642, batch_cost: 0.2059, reader_cost: 0.00027, ips: 14.5673 samples/sec | ETA 05:05:28
- 2022-04-13 13:21:55 [INFO] [TRAIN] epoch: 126, iter: 31050/120000, loss: 0.9238, lr: 0.007638, batch_cost: 0.2707, reader_cost: 0.05756, ips: 11.0830 samples/sec | ETA 06:41:17
- 2022-04-13 13:22:05 [INFO] [TRAIN] epoch: 126, iter: 31100/120000, loss: 0.9554, lr: 0.007634, batch_cost: 0.2009, reader_cost: 0.00136, ips: 14.9348 samples/sec | ETA 04:57:37
- 2022-04-13 13:22:16 [INFO] [TRAIN] epoch: 126, iter: 31150/120000, loss: 0.9021, lr: 0.007630, batch_cost: 0.2037, reader_cost: 0.00100, ips: 14.7249 samples/sec | ETA 05:01:41
- 2022-04-13 13:22:26 [INFO] [TRAIN] epoch: 126, iter: 31200/120000, loss: 0.9778, lr: 0.007626, batch_cost: 0.2173, reader_cost: 0.00144, ips: 13.8080 samples/sec | ETA 05:21:33
- 2022-04-13 13:22:40 [INFO] [TRAIN] epoch: 127, iter: 31250/120000, loss: 0.9235, lr: 0.007622, batch_cost: 0.2640, reader_cost: 0.06114, ips: 11.3650 samples/sec | ETA 06:30:27
- 2022-04-13 13:22:50 [INFO] [TRAIN] epoch: 127, iter: 31300/120000, loss: 0.9317, lr: 0.007619, batch_cost: 0.2137, reader_cost: 0.00100, ips: 14.0385 samples/sec | ETA 05:15:54
- 2022-04-13 13:23:00 [INFO] [TRAIN] epoch: 127, iter: 31350/120000, loss: 0.9564, lr: 0.007615, batch_cost: 0.2002, reader_cost: 0.00110, ips: 14.9846 samples/sec | ETA 04:55:48
- 2022-04-13 13:23:11 [INFO] [TRAIN] epoch: 127, iter: 31400/120000, loss: 0.8697, lr: 0.007611, batch_cost: 0.2117, reader_cost: 0.00079, ips: 14.1718 samples/sec | ETA 05:12:35
- 2022-04-13 13:23:22 [INFO] [TRAIN] epoch: 127, iter: 31450/120000, loss: 0.9248, lr: 0.007607, batch_cost: 0.2182, reader_cost: 0.00143, ips: 13.7460 samples/sec | ETA 05:22:05
- 2022-04-13 13:23:36 [INFO] [TRAIN] epoch: 128, iter: 31500/120000, loss: 0.9148, lr: 0.007603, batch_cost: 0.2775, reader_cost: 0.04970, ips: 10.8106 samples/sec | ETA 06:49:19
- 2022-04-13 13:23:46 [INFO] [TRAIN] epoch: 128, iter: 31550/120000, loss: 0.9116, lr: 0.007599, batch_cost: 0.2127, reader_cost: 0.00056, ips: 14.1027 samples/sec | ETA 05:13:35
- 2022-04-13 13:23:57 [INFO] [TRAIN] epoch: 128, iter: 31600/120000, loss: 0.9211, lr: 0.007595, batch_cost: 0.2134, reader_cost: 0.00067, ips: 14.0565 samples/sec | ETA 05:14:26
- 2022-04-13 13:24:08 [INFO] [TRAIN] epoch: 128, iter: 31650/120000, loss: 0.9129, lr: 0.007591, batch_cost: 0.2153, reader_cost: 0.00074, ips: 13.9361 samples/sec | ETA 05:16:58
- 2022-04-13 13:24:18 [INFO] [TRAIN] epoch: 128, iter: 31700/120000, loss: 0.9551, lr: 0.007588, batch_cost: 0.2010, reader_cost: 0.00093, ips: 14.9226 samples/sec | ETA 04:55:51
- 2022-04-13 13:24:32 [INFO] [TRAIN] epoch: 129, iter: 31750/120000, loss: 0.9071, lr: 0.007584, batch_cost: 0.2790, reader_cost: 0.05859, ips: 10.7539 samples/sec | ETA 06:50:19
- 2022-04-13 13:24:42 [INFO] [TRAIN] epoch: 129, iter: 31800/120000, loss: 0.9351, lr: 0.007580, batch_cost: 0.2096, reader_cost: 0.00127, ips: 14.3129 samples/sec | ETA 05:08:06
- 2022-04-13 13:24:52 [INFO] [TRAIN] epoch: 129, iter: 31850/120000, loss: 0.9283, lr: 0.007576, batch_cost: 0.1952, reader_cost: 0.00069, ips: 15.3660 samples/sec | ETA 04:46:50
- 2022-04-13 13:25:03 [INFO] [TRAIN] epoch: 129, iter: 31900/120000, loss: 0.9213, lr: 0.007572, batch_cost: 0.2090, reader_cost: 0.00185, ips: 14.3537 samples/sec | ETA 05:06:53
- 2022-04-13 13:25:13 [INFO] [TRAIN] epoch: 129, iter: 31950/120000, loss: 0.9171, lr: 0.007568, batch_cost: 0.2048, reader_cost: 0.00055, ips: 14.6478 samples/sec | ETA 05:00:33
- 2022-04-13 13:25:26 [INFO] [TRAIN] epoch: 130, iter: 32000/120000, loss: 0.9370, lr: 0.007564, batch_cost: 0.2697, reader_cost: 0.05339, ips: 11.1244 samples/sec | ETA 06:35:31
- 2022-04-13 13:25:26 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1959 - reader cost: 0.1248
- 2022-04-13 13:25:51 [INFO] [EVAL] #Images: 500 mIoU: 0.7067 Acc: 0.9525 Kappa: 0.9383 Dice: 0.8148
- 2022-04-13 13:25:51 [INFO] [EVAL] Class IoU:
- [0.9756 0.8132 0.9124 0.4945 0.4746 0.6119 0.6711 0.766 0.9182 0.601
- 0.9404 0.7998 0.6066 0.9348 0.3411 0.7377 0.5342 0.5638 0.7307]
- 2022-04-13 13:25:51 [INFO] [EVAL] Class Precision:
- [0.9866 0.8952 0.9491 0.8361 0.7556 0.7999 0.7721 0.9027 0.9521 0.8234
- 0.9549 0.8894 0.7548 0.9524 0.9495 0.7841 0.7733 0.7722 0.7918]
- 2022-04-13 13:25:51 [INFO] [EVAL] Class Recall:
- [0.9887 0.8988 0.9594 0.5475 0.5607 0.7226 0.837 0.8349 0.9627 0.6899
- 0.9841 0.8881 0.7556 0.9806 0.3474 0.9259 0.6333 0.6763 0.9044]
- 2022-04-13 13:25:51 [INFO] [EVAL] The model with the best validation mIoU (0.7276) was saved at iter 28000.
- 2022-04-13 13:26:02 [INFO] [TRAIN] epoch: 130, iter: 32050/120000, loss: 0.8980, lr: 0.007561, batch_cost: 0.2091, reader_cost: 0.00092, ips: 14.3493 samples/sec | ETA 05:06:27
- 2022-04-13 13:26:13 [INFO] [TRAIN] epoch: 130, iter: 32100/120000, loss: 0.8838, lr: 0.007557, batch_cost: 0.2179, reader_cost: 0.00078, ips: 13.7661 samples/sec | ETA 05:19:15
- 2022-04-13 13:26:24 [INFO] [TRAIN] epoch: 130, iter: 32150/120000, loss: 0.9222, lr: 0.007553, batch_cost: 0.2267, reader_cost: 0.00092, ips: 13.2343 samples/sec | ETA 05:31:54
- 2022-04-13 13:26:35 [INFO] [TRAIN] epoch: 130, iter: 32200/120000, loss: 0.9233, lr: 0.007549, batch_cost: 0.2087, reader_cost: 0.00069, ips: 14.3744 samples/sec | ETA 05:05:24
- 2022-04-13 13:26:48 [INFO] [TRAIN] epoch: 131, iter: 32250/120000, loss: 0.9239, lr: 0.007545, batch_cost: 0.2734, reader_cost: 0.05753, ips: 10.9723 samples/sec | ETA 06:39:52
- 2022-04-13 13:26:58 [INFO] [TRAIN] epoch: 131, iter: 32300/120000, loss: 0.9316, lr: 0.007541, batch_cost: 0.2033, reader_cost: 0.00187, ips: 14.7546 samples/sec | ETA 04:57:11
- 2022-04-13 13:27:09 [INFO] [TRAIN] epoch: 131, iter: 32350/120000, loss: 0.8934, lr: 0.007537, batch_cost: 0.2023, reader_cost: 0.00030, ips: 14.8303 samples/sec | ETA 04:55:30
- 2022-04-13 13:27:20 [INFO] [TRAIN] epoch: 131, iter: 32400/120000, loss: 0.9151, lr: 0.007533, batch_cost: 0.2323, reader_cost: 0.00057, ips: 12.9124 samples/sec | ETA 05:39:12
- 2022-04-13 13:27:30 [INFO] [TRAIN] epoch: 131, iter: 32450/120000, loss: 0.9143, lr: 0.007530, batch_cost: 0.2008, reader_cost: 0.00079, ips: 14.9375 samples/sec | ETA 04:53:03
- 2022-04-13 13:27:44 [INFO] [TRAIN] epoch: 132, iter: 32500/120000, loss: 0.9489, lr: 0.007526, batch_cost: 0.2760, reader_cost: 0.05643, ips: 10.8688 samples/sec | ETA 06:42:31
- 2022-04-13 13:27:54 [INFO] [TRAIN] epoch: 132, iter: 32550/120000, loss: 0.9428, lr: 0.007522, batch_cost: 0.1955, reader_cost: 0.00123, ips: 15.3487 samples/sec | ETA 04:44:52
- 2022-04-13 13:28:04 [INFO] [TRAIN] epoch: 132, iter: 32600/120000, loss: 0.9196, lr: 0.007518, batch_cost: 0.2080, reader_cost: 0.00106, ips: 14.4243 samples/sec | ETA 05:02:57
- 2022-04-13 13:28:16 [INFO] [TRAIN] epoch: 132, iter: 32650/120000, loss: 0.9164, lr: 0.007514, batch_cost: 0.2340, reader_cost: 0.00155, ips: 12.8219 samples/sec | ETA 05:40:37
- 2022-04-13 13:28:27 [INFO] [TRAIN] epoch: 132, iter: 32700/120000, loss: 0.8841, lr: 0.007510, batch_cost: 0.2284, reader_cost: 0.00093, ips: 13.1342 samples/sec | ETA 05:32:20
- 2022-04-13 13:28:41 [INFO] [TRAIN] epoch: 133, iter: 32750/120000, loss: 0.8602, lr: 0.007506, batch_cost: 0.2700, reader_cost: 0.05456, ips: 11.1131 samples/sec | ETA 06:32:33
- 2022-04-13 13:28:52 [INFO] [TRAIN] epoch: 133, iter: 32800/120000, loss: 0.8955, lr: 0.007503, batch_cost: 0.2228, reader_cost: 0.00212, ips: 13.4642 samples/sec | ETA 05:23:49
- 2022-04-13 13:29:03 [INFO] [TRAIN] epoch: 133, iter: 32850/120000, loss: 0.9099, lr: 0.007499, batch_cost: 0.2166, reader_cost: 0.00112, ips: 13.8474 samples/sec | ETA 05:14:40
- 2022-04-13 13:29:13 [INFO] [TRAIN] epoch: 133, iter: 32900/120000, loss: 0.9529, lr: 0.007495, batch_cost: 0.2019, reader_cost: 0.00137, ips: 14.8556 samples/sec | ETA 04:53:09
- 2022-04-13 13:29:24 [INFO] [TRAIN] epoch: 133, iter: 32950/120000, loss: 0.9457, lr: 0.007491, batch_cost: 0.2224, reader_cost: 0.00143, ips: 13.4867 samples/sec | ETA 05:22:43
- 2022-04-13 13:29:37 [INFO] [TRAIN] epoch: 134, iter: 33000/120000, loss: 0.9445, lr: 0.007487, batch_cost: 0.2675, reader_cost: 0.06679, ips: 11.2148 samples/sec | ETA 06:27:52
- 2022-04-13 13:29:48 [INFO] [TRAIN] epoch: 134, iter: 33050/120000, loss: 0.8810, lr: 0.007483, batch_cost: 0.2108, reader_cost: 0.00134, ips: 14.2294 samples/sec | ETA 05:05:31
- 2022-04-13 13:29:58 [INFO] [TRAIN] epoch: 134, iter: 33100/120000, loss: 0.8981, lr: 0.007479, batch_cost: 0.1984, reader_cost: 0.00119, ips: 15.1188 samples/sec | ETA 04:47:23
- 2022-04-13 13:30:09 [INFO] [TRAIN] epoch: 134, iter: 33150/120000, loss: 0.8929, lr: 0.007475, batch_cost: 0.2167, reader_cost: 0.00064, ips: 13.8467 samples/sec | ETA 05:13:36
- 2022-04-13 13:30:19 [INFO] [TRAIN] epoch: 134, iter: 33200/120000, loss: 0.9410, lr: 0.007472, batch_cost: 0.2080, reader_cost: 0.00120, ips: 14.4260 samples/sec | ETA 05:00:50
- 2022-04-13 13:30:33 [INFO] [TRAIN] epoch: 135, iter: 33250/120000, loss: 0.9318, lr: 0.007468, batch_cost: 0.2773, reader_cost: 0.05526, ips: 10.8196 samples/sec | ETA 06:40:53
- 2022-04-13 13:30:43 [INFO] [TRAIN] epoch: 135, iter: 33300/120000, loss: 0.9243, lr: 0.007464, batch_cost: 0.1940, reader_cost: 0.00145, ips: 15.4630 samples/sec | ETA 04:40:20
- 2022-04-13 13:30:54 [INFO] [TRAIN] epoch: 135, iter: 33350/120000, loss: 0.8905, lr: 0.007460, batch_cost: 0.2177, reader_cost: 0.00143, ips: 13.7801 samples/sec | ETA 05:14:24
- 2022-04-13 13:31:05 [INFO] [TRAIN] epoch: 135, iter: 33400/120000, loss: 0.9288, lr: 0.007456, batch_cost: 0.2262, reader_cost: 0.00076, ips: 13.2651 samples/sec | ETA 05:26:25
- 2022-04-13 13:31:15 [INFO] [TRAIN] epoch: 135, iter: 33450/120000, loss: 0.9259, lr: 0.007452, batch_cost: 0.2020, reader_cost: 0.00072, ips: 14.8527 samples/sec | ETA 04:51:21
- 2022-04-13 13:31:30 [INFO] [TRAIN] epoch: 136, iter: 33500/120000, loss: 0.8967, lr: 0.007448, batch_cost: 0.2947, reader_cost: 0.06133, ips: 10.1803 samples/sec | ETA 07:04:50
- 2022-04-13 13:31:40 [INFO] [TRAIN] epoch: 136, iter: 33550/120000, loss: 0.8926, lr: 0.007444, batch_cost: 0.2119, reader_cost: 0.00180, ips: 14.1575 samples/sec | ETA 05:05:18
- 2022-04-13 13:31:51 [INFO] [TRAIN] epoch: 136, iter: 33600/120000, loss: 0.9133, lr: 0.007441, batch_cost: 0.2153, reader_cost: 0.00066, ips: 13.9342 samples/sec | ETA 05:10:01
- 2022-04-13 13:32:01 [INFO] [TRAIN] epoch: 136, iter: 33650/120000, loss: 0.9422, lr: 0.007437, batch_cost: 0.2073, reader_cost: 0.00068, ips: 14.4736 samples/sec | ETA 04:58:18
- 2022-04-13 13:32:13 [INFO] [TRAIN] epoch: 136, iter: 33700/120000, loss: 0.8862, lr: 0.007433, batch_cost: 0.2347, reader_cost: 0.00074, ips: 12.7832 samples/sec | ETA 05:37:33
- 2022-04-13 13:32:27 [INFO] [TRAIN] epoch: 137, iter: 33750/120000, loss: 0.9074, lr: 0.007429, batch_cost: 0.2711, reader_cost: 0.06392, ips: 11.0646 samples/sec | ETA 06:29:45
- 2022-04-13 13:32:37 [INFO] [TRAIN] epoch: 137, iter: 33800/120000, loss: 0.9086, lr: 0.007425, batch_cost: 0.2009, reader_cost: 0.00108, ips: 14.9308 samples/sec | ETA 04:48:39
- 2022-04-13 13:32:47 [INFO] [TRAIN] epoch: 137, iter: 33850/120000, loss: 0.9122, lr: 0.007421, batch_cost: 0.2020, reader_cost: 0.00096, ips: 14.8512 samples/sec | ETA 04:50:02
- 2022-04-13 13:33:00 [INFO] [TRAIN] epoch: 137, iter: 33900/120000, loss: 0.9535, lr: 0.007417, batch_cost: 0.2536, reader_cost: 0.00063, ips: 11.8309 samples/sec | ETA 06:03:52
- 2022-04-13 13:33:10 [INFO] [TRAIN] epoch: 137, iter: 33950/120000, loss: 0.9068, lr: 0.007413, batch_cost: 0.2061, reader_cost: 0.00061, ips: 14.5532 samples/sec | ETA 04:55:38
- 2022-04-13 13:33:23 [INFO] [TRAIN] epoch: 138, iter: 34000/120000, loss: 0.8892, lr: 0.007410, batch_cost: 0.2697, reader_cost: 0.05051, ips: 11.1223 samples/sec | ETA 06:26:36
- 2022-04-13 13:33:33 [INFO] [TRAIN] epoch: 138, iter: 34050/120000, loss: 0.8833, lr: 0.007406, batch_cost: 0.1933, reader_cost: 0.00081, ips: 15.5163 samples/sec | ETA 04:36:58
- 2022-04-13 13:33:44 [INFO] [TRAIN] epoch: 138, iter: 34100/120000, loss: 0.8755, lr: 0.007402, batch_cost: 0.2231, reader_cost: 0.00133, ips: 13.4453 samples/sec | ETA 05:19:26
- 2022-04-13 13:33:56 [INFO] [TRAIN] epoch: 138, iter: 34150/120000, loss: 0.9024, lr: 0.007398, batch_cost: 0.2305, reader_cost: 0.00115, ips: 13.0146 samples/sec | ETA 05:29:49
- 2022-04-13 13:34:07 [INFO] [TRAIN] epoch: 138, iter: 34200/120000, loss: 0.9396, lr: 0.007394, batch_cost: 0.2171, reader_cost: 0.00055, ips: 13.8171 samples/sec | ETA 05:10:29
- 2022-04-13 13:34:20 [INFO] [TRAIN] epoch: 139, iter: 34250/120000, loss: 0.9166, lr: 0.007390, batch_cost: 0.2683, reader_cost: 0.05631, ips: 11.1830 samples/sec | ETA 06:23:23
- 2022-04-13 13:34:31 [INFO] [TRAIN] epoch: 139, iter: 34300/120000, loss: 0.9336, lr: 0.007386, batch_cost: 0.2110, reader_cost: 0.00174, ips: 14.2199 samples/sec | ETA 05:01:20
- 2022-04-13 13:34:41 [INFO] [TRAIN] epoch: 139, iter: 34350/120000, loss: 0.9349, lr: 0.007382, batch_cost: 0.2084, reader_cost: 0.00055, ips: 14.3959 samples/sec | ETA 04:57:28
- 2022-04-13 13:34:52 [INFO] [TRAIN] epoch: 139, iter: 34400/120000, loss: 0.9233, lr: 0.007378, batch_cost: 0.2226, reader_cost: 0.00111, ips: 13.4769 samples/sec | ETA 05:17:34
- 2022-04-13 13:35:03 [INFO] [TRAIN] epoch: 139, iter: 34450/120000, loss: 0.9151, lr: 0.007375, batch_cost: 0.2229, reader_cost: 0.00085, ips: 13.4607 samples/sec | ETA 05:17:46
- 2022-04-13 13:35:17 [INFO] [TRAIN] epoch: 140, iter: 34500/120000, loss: 0.9190, lr: 0.007371, batch_cost: 0.2761, reader_cost: 0.06608, ips: 10.8673 samples/sec | ETA 06:33:22
- 2022-04-13 13:35:27 [INFO] [TRAIN] epoch: 140, iter: 34550/120000, loss: 0.9010, lr: 0.007367, batch_cost: 0.2020, reader_cost: 0.00119, ips: 14.8491 samples/sec | ETA 04:47:43
- 2022-04-13 13:35:38 [INFO] [TRAIN] epoch: 140, iter: 34600/120000, loss: 0.8886, lr: 0.007363, batch_cost: 0.2201, reader_cost: 0.00112, ips: 13.6322 samples/sec | ETA 05:13:13
- 2022-04-13 13:35:49 [INFO] [TRAIN] epoch: 140, iter: 34650/120000, loss: 0.9311, lr: 0.007359, batch_cost: 0.2136, reader_cost: 0.00105, ips: 14.0424 samples/sec | ETA 05:03:54
- 2022-04-13 13:35:59 [INFO] [TRAIN] epoch: 140, iter: 34700/120000, loss: 0.8931, lr: 0.007355, batch_cost: 0.2062, reader_cost: 0.00172, ips: 14.5471 samples/sec | ETA 04:53:11
- 2022-04-13 13:36:13 [INFO] [TRAIN] epoch: 141, iter: 34750/120000, loss: 0.9340, lr: 0.007351, batch_cost: 0.2731, reader_cost: 0.06267, ips: 10.9839 samples/sec | ETA 06:28:04
- 2022-04-13 13:36:23 [INFO] [TRAIN] epoch: 141, iter: 34800/120000, loss: 0.8927, lr: 0.007347, batch_cost: 0.1993, reader_cost: 0.00034, ips: 15.0522 samples/sec | ETA 04:43:00
- 2022-04-13 13:36:33 [INFO] [TRAIN] epoch: 141, iter: 34850/120000, loss: 0.9278, lr: 0.007344, batch_cost: 0.2100, reader_cost: 0.00113, ips: 14.2831 samples/sec | ETA 04:58:04
- 2022-04-13 13:36:44 [INFO] [TRAIN] epoch: 141, iter: 34900/120000, loss: 0.8833, lr: 0.007340, batch_cost: 0.2059, reader_cost: 0.00078, ips: 14.5709 samples/sec | ETA 04:52:01
- 2022-04-13 13:36:54 [INFO] [TRAIN] epoch: 141, iter: 34950/120000, loss: 0.9186, lr: 0.007336, batch_cost: 0.2133, reader_cost: 0.00079, ips: 14.0627 samples/sec | ETA 05:02:23
- 2022-04-13 13:37:08 [INFO] [TRAIN] epoch: 142, iter: 35000/120000, loss: 0.9530, lr: 0.007332, batch_cost: 0.2786, reader_cost: 0.06288, ips: 10.7677 samples/sec | ETA 06:34:41
- 2022-04-13 13:37:19 [INFO] [TRAIN] epoch: 142, iter: 35050/120000, loss: 0.9174, lr: 0.007328, batch_cost: 0.2100, reader_cost: 0.00136, ips: 14.2876 samples/sec | ETA 04:57:17
- 2022-04-13 13:37:30 [INFO] [TRAIN] epoch: 142, iter: 35100/120000, loss: 0.9394, lr: 0.007324, batch_cost: 0.2279, reader_cost: 0.00141, ips: 13.1622 samples/sec | ETA 05:22:30
- 2022-04-13 13:37:40 [INFO] [TRAIN] epoch: 142, iter: 35150/120000, loss: 0.9187, lr: 0.007320, batch_cost: 0.1988, reader_cost: 0.00080, ips: 15.0943 samples/sec | ETA 04:41:04
- 2022-04-13 13:37:51 [INFO] [TRAIN] epoch: 142, iter: 35200/120000, loss: 0.9574, lr: 0.007316, batch_cost: 0.2199, reader_cost: 0.00123, ips: 13.6411 samples/sec | ETA 05:10:49
- 2022-04-13 13:38:04 [INFO] [TRAIN] epoch: 143, iter: 35250/120000, loss: 0.9409, lr: 0.007313, batch_cost: 0.2684, reader_cost: 0.05068, ips: 11.1773 samples/sec | ETA 06:19:07
- 2022-04-13 13:38:15 [INFO] [TRAIN] epoch: 143, iter: 35300/120000, loss: 0.8770, lr: 0.007309, batch_cost: 0.2075, reader_cost: 0.00092, ips: 14.4600 samples/sec | ETA 04:52:52
- 2022-04-13 13:38:25 [INFO] [TRAIN] epoch: 143, iter: 35350/120000, loss: 0.9113, lr: 0.007305, batch_cost: 0.2135, reader_cost: 0.00124, ips: 14.0494 samples/sec | ETA 05:01:15
- 2022-04-13 13:38:37 [INFO] [TRAIN] epoch: 143, iter: 35400/120000, loss: 0.9531, lr: 0.007301, batch_cost: 0.2260, reader_cost: 0.00114, ips: 13.2770 samples/sec | ETA 05:18:35
- 2022-04-13 13:38:47 [INFO] [TRAIN] epoch: 143, iter: 35450/120000, loss: 0.9351, lr: 0.007297, batch_cost: 0.1989, reader_cost: 0.00110, ips: 15.0829 samples/sec | ETA 04:40:17
- 2022-04-13 13:39:00 [INFO] [TRAIN] epoch: 144, iter: 35500/120000, loss: 0.9276, lr: 0.007293, batch_cost: 0.2693, reader_cost: 0.05116, ips: 11.1384 samples/sec | ETA 06:19:19
- 2022-04-13 13:39:10 [INFO] [TRAIN] epoch: 144, iter: 35550/120000, loss: 0.9087, lr: 0.007289, batch_cost: 0.1972, reader_cost: 0.00132, ips: 15.2093 samples/sec | ETA 04:37:37
- 2022-04-13 13:39:22 [INFO] [TRAIN] epoch: 144, iter: 35600/120000, loss: 0.8790, lr: 0.007285, batch_cost: 0.2322, reader_cost: 0.00052, ips: 12.9225 samples/sec | ETA 05:26:33
- 2022-04-13 13:39:33 [INFO] [TRAIN] epoch: 144, iter: 35650/120000, loss: 0.9207, lr: 0.007281, batch_cost: 0.2280, reader_cost: 0.00083, ips: 13.1607 samples/sec | ETA 05:20:27
- 2022-04-13 13:39:43 [INFO] [TRAIN] epoch: 144, iter: 35700/120000, loss: 0.9425, lr: 0.007278, batch_cost: 0.1986, reader_cost: 0.00066, ips: 15.1032 samples/sec | ETA 04:39:04
- 2022-04-13 13:39:56 [INFO] [TRAIN] epoch: 145, iter: 35750/120000, loss: 0.8931, lr: 0.007274, batch_cost: 0.2657, reader_cost: 0.05713, ips: 11.2928 samples/sec | ETA 06:13:01
- 2022-04-13 13:40:06 [INFO] [TRAIN] epoch: 145, iter: 35800/120000, loss: 0.9724, lr: 0.007270, batch_cost: 0.2036, reader_cost: 0.00113, ips: 14.7349 samples/sec | ETA 04:45:42
- 2022-04-13 13:40:18 [INFO] [TRAIN] epoch: 145, iter: 35850/120000, loss: 0.9273, lr: 0.007266, batch_cost: 0.2306, reader_cost: 0.00066, ips: 13.0075 samples/sec | ETA 05:23:28
- 2022-04-13 13:40:29 [INFO] [TRAIN] epoch: 145, iter: 35900/120000, loss: 0.8797, lr: 0.007262, batch_cost: 0.2160, reader_cost: 0.00071, ips: 13.8920 samples/sec | ETA 05:02:41
- 2022-04-13 13:40:40 [INFO] [TRAIN] epoch: 145, iter: 35950/120000, loss: 0.9009, lr: 0.007258, batch_cost: 0.2309, reader_cost: 0.00076, ips: 12.9923 samples/sec | ETA 05:23:27
- 2022-04-13 13:40:54 [INFO] [TRAIN] epoch: 146, iter: 36000/120000, loss: 0.9024, lr: 0.007254, batch_cost: 0.2711, reader_cost: 0.05486, ips: 11.0678 samples/sec | ETA 06:19:28
- 2022-04-13 13:40:54 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1958 - reader cost: 0.1525
- 2022-04-13 13:41:19 [INFO] [EVAL] #Images: 500 mIoU: 0.7136 Acc: 0.9505 Kappa: 0.9356 Dice: 0.8199
- 2022-04-13 13:41:19 [INFO] [EVAL] Class IoU:
- [0.9758 0.8189 0.9079 0.3787 0.4741 0.6036 0.6744 0.7587 0.9087 0.5565
- 0.9468 0.7921 0.577 0.9318 0.5178 0.808 0.7259 0.4748 0.7276]
- 2022-04-13 13:41:19 [INFO] [EVAL] Class Precision:
- [0.9875 0.9193 0.9454 0.8089 0.7524 0.7943 0.8352 0.9076 0.9408 0.6427
- 0.9681 0.8937 0.7291 0.9602 0.8948 0.8933 0.9427 0.5415 0.7951]
- 2022-04-13 13:41:19 [INFO] [EVAL] Class Recall:
- [0.988 0.8823 0.9582 0.416 0.5617 0.7154 0.7779 0.8222 0.9638 0.806
- 0.9773 0.8745 0.7344 0.9693 0.5513 0.8944 0.7595 0.794 0.8955]
- 2022-04-13 13:41:19 [INFO] [EVAL] The model with the best validation mIoU (0.7276) was saved at iter 28000.
- 2022-04-13 13:41:29 [INFO] [TRAIN] epoch: 146, iter: 36050/120000, loss: 0.9066, lr: 0.007250, batch_cost: 0.1951, reader_cost: 0.00159, ips: 15.3752 samples/sec | ETA 04:33:00
- 2022-04-13 13:41:39 [INFO] [TRAIN] epoch: 146, iter: 36100/120000, loss: 0.9083, lr: 0.007246, batch_cost: 0.2010, reader_cost: 0.00086, ips: 14.9285 samples/sec | ETA 04:41:00
- 2022-04-13 13:41:49 [INFO] [TRAIN] epoch: 146, iter: 36150/120000, loss: 0.9395, lr: 0.007243, batch_cost: 0.2058, reader_cost: 0.00093, ips: 14.5749 samples/sec | ETA 04:47:39
- 2022-04-13 13:41:59 [INFO] [TRAIN] epoch: 146, iter: 36200/120000, loss: 0.9024, lr: 0.007239, batch_cost: 0.1963, reader_cost: 0.00069, ips: 15.2794 samples/sec | ETA 04:34:13
- 2022-04-13 13:42:13 [INFO] [TRAIN] epoch: 147, iter: 36250/120000, loss: 0.9240, lr: 0.007235, batch_cost: 0.2716, reader_cost: 0.05321, ips: 11.0453 samples/sec | ETA 06:19:07
- 2022-04-13 13:42:23 [INFO] [TRAIN] epoch: 147, iter: 36300/120000, loss: 0.8960, lr: 0.007231, batch_cost: 0.2080, reader_cost: 0.00073, ips: 14.4238 samples/sec | ETA 04:50:08
- 2022-04-13 13:42:34 [INFO] [TRAIN] epoch: 147, iter: 36350/120000, loss: 0.8978, lr: 0.007227, batch_cost: 0.2251, reader_cost: 0.00073, ips: 13.3266 samples/sec | ETA 05:13:50
- 2022-04-13 13:42:46 [INFO] [TRAIN] epoch: 147, iter: 36400/120000, loss: 0.9147, lr: 0.007223, batch_cost: 0.2328, reader_cost: 0.00146, ips: 12.8884 samples/sec | ETA 05:24:19
- 2022-04-13 13:42:56 [INFO] [TRAIN] epoch: 147, iter: 36450/120000, loss: 0.9663, lr: 0.007219, batch_cost: 0.2028, reader_cost: 0.00182, ips: 14.7917 samples/sec | ETA 04:42:25
- 2022-04-13 13:43:09 [INFO] [TRAIN] epoch: 148, iter: 36500/120000, loss: 0.8946, lr: 0.007215, batch_cost: 0.2664, reader_cost: 0.06149, ips: 11.2596 samples/sec | ETA 06:10:47
- 2022-04-13 13:43:20 [INFO] [TRAIN] epoch: 148, iter: 36550/120000, loss: 0.9225, lr: 0.007211, batch_cost: 0.2114, reader_cost: 0.00093, ips: 14.1897 samples/sec | ETA 04:54:03
- 2022-04-13 13:43:31 [INFO] [TRAIN] epoch: 148, iter: 36600/120000, loss: 0.8968, lr: 0.007208, batch_cost: 0.2139, reader_cost: 0.00093, ips: 14.0236 samples/sec | ETA 04:57:21
- 2022-04-13 13:43:41 [INFO] [TRAIN] epoch: 148, iter: 36650/120000, loss: 0.9277, lr: 0.007204, batch_cost: 0.2040, reader_cost: 0.00106, ips: 14.7072 samples/sec | ETA 04:43:21
- 2022-04-13 13:43:51 [INFO] [TRAIN] epoch: 148, iter: 36700/120000, loss: 0.9168, lr: 0.007200, batch_cost: 0.2057, reader_cost: 0.00071, ips: 14.5869 samples/sec | ETA 04:45:31
- 2022-04-13 13:44:05 [INFO] [TRAIN] epoch: 149, iter: 36750/120000, loss: 0.9513, lr: 0.007196, batch_cost: 0.2733, reader_cost: 0.05920, ips: 10.9779 samples/sec | ETA 06:19:10
- 2022-04-13 13:44:15 [INFO] [TRAIN] epoch: 149, iter: 36800/120000, loss: 0.9205, lr: 0.007192, batch_cost: 0.2002, reader_cost: 0.00103, ips: 14.9835 samples/sec | ETA 04:37:38
- 2022-04-13 13:44:25 [INFO] [TRAIN] epoch: 149, iter: 36850/120000, loss: 0.9516, lr: 0.007188, batch_cost: 0.2120, reader_cost: 0.00063, ips: 14.1532 samples/sec | ETA 04:53:44
- 2022-04-13 13:44:36 [INFO] [TRAIN] epoch: 149, iter: 36900/120000, loss: 0.8651, lr: 0.007184, batch_cost: 0.2140, reader_cost: 0.00086, ips: 14.0188 samples/sec | ETA 04:56:23
- 2022-04-13 13:44:45 [INFO] [TRAIN] epoch: 149, iter: 36950/120000, loss: 0.9009, lr: 0.007180, batch_cost: 0.1888, reader_cost: 0.00051, ips: 15.8896 samples/sec | ETA 04:21:20
- 2022-04-13 13:45:00 [INFO] [TRAIN] epoch: 150, iter: 37000/120000, loss: 0.9092, lr: 0.007176, batch_cost: 0.2851, reader_cost: 0.06025, ips: 10.5231 samples/sec | ETA 06:34:22
- 2022-04-13 13:45:10 [INFO] [TRAIN] epoch: 150, iter: 37050/120000, loss: 0.9093, lr: 0.007173, batch_cost: 0.2034, reader_cost: 0.00113, ips: 14.7478 samples/sec | ETA 04:41:13
- 2022-04-13 13:45:20 [INFO] [TRAIN] epoch: 150, iter: 37100/120000, loss: 0.9759, lr: 0.007169, batch_cost: 0.2067, reader_cost: 0.00114, ips: 14.5146 samples/sec | ETA 04:45:34
- 2022-04-13 13:45:30 [INFO] [TRAIN] epoch: 150, iter: 37150/120000, loss: 0.9320, lr: 0.007165, batch_cost: 0.2013, reader_cost: 0.00095, ips: 14.9016 samples/sec | ETA 04:37:59
- 2022-04-13 13:45:40 [INFO] [TRAIN] epoch: 150, iter: 37200/120000, loss: 0.9094, lr: 0.007161, batch_cost: 0.1881, reader_cost: 0.00025, ips: 15.9467 samples/sec | ETA 04:19:36
- 2022-04-13 13:45:55 [INFO] [TRAIN] epoch: 151, iter: 37250/120000, loss: 0.8942, lr: 0.007157, batch_cost: 0.2959, reader_cost: 0.05756, ips: 10.1391 samples/sec | ETA 06:48:04
- 2022-04-13 13:46:05 [INFO] [TRAIN] epoch: 151, iter: 37300/120000, loss: 0.9321, lr: 0.007153, batch_cost: 0.2038, reader_cost: 0.00071, ips: 14.7234 samples/sec | ETA 04:40:50
- 2022-04-13 13:46:16 [INFO] [TRAIN] epoch: 151, iter: 37350/120000, loss: 1.0088, lr: 0.007149, batch_cost: 0.2208, reader_cost: 0.00088, ips: 13.5849 samples/sec | ETA 05:04:11
- 2022-04-13 13:46:26 [INFO] [TRAIN] epoch: 151, iter: 37400/120000, loss: 0.9255, lr: 0.007145, batch_cost: 0.2095, reader_cost: 0.00181, ips: 14.3218 samples/sec | ETA 04:48:22
- 2022-04-13 13:46:40 [INFO] [TRAIN] epoch: 152, iter: 37450/120000, loss: 0.9150, lr: 0.007141, batch_cost: 0.2682, reader_cost: 0.04574, ips: 11.1858 samples/sec | ETA 06:08:59
- 2022-04-13 13:46:50 [INFO] [TRAIN] epoch: 152, iter: 37500/120000, loss: 0.9092, lr: 0.007138, batch_cost: 0.2100, reader_cost: 0.00101, ips: 14.2857 samples/sec | ETA 04:48:44
- 2022-04-13 13:47:00 [INFO] [TRAIN] epoch: 152, iter: 37550/120000, loss: 0.8815, lr: 0.007134, batch_cost: 0.2046, reader_cost: 0.00153, ips: 14.6597 samples/sec | ETA 04:41:12
- 2022-04-13 13:47:10 [INFO] [TRAIN] epoch: 152, iter: 37600/120000, loss: 0.8926, lr: 0.007130, batch_cost: 0.2019, reader_cost: 0.00110, ips: 14.8582 samples/sec | ETA 04:37:17
- 2022-04-13 13:47:21 [INFO] [TRAIN] epoch: 152, iter: 37650/120000, loss: 0.8918, lr: 0.007126, batch_cost: 0.2065, reader_cost: 0.00081, ips: 14.5268 samples/sec | ETA 04:43:26
- 2022-04-13 13:47:34 [INFO] [TRAIN] epoch: 153, iter: 37700/120000, loss: 0.9378, lr: 0.007122, batch_cost: 0.2548, reader_cost: 0.05481, ips: 11.7740 samples/sec | ETA 05:49:29
- 2022-04-13 13:47:44 [INFO] [TRAIN] epoch: 153, iter: 37750/120000, loss: 0.9442, lr: 0.007118, batch_cost: 0.2163, reader_cost: 0.00095, ips: 13.8669 samples/sec | ETA 04:56:34
- 2022-04-13 13:47:55 [INFO] [TRAIN] epoch: 153, iter: 37800/120000, loss: 0.9083, lr: 0.007114, batch_cost: 0.2119, reader_cost: 0.00076, ips: 14.1560 samples/sec | ETA 04:50:20
- 2022-04-13 13:48:06 [INFO] [TRAIN] epoch: 153, iter: 37850/120000, loss: 0.9482, lr: 0.007110, batch_cost: 0.2136, reader_cost: 0.00125, ips: 14.0453 samples/sec | ETA 04:52:26
- 2022-04-13 13:48:17 [INFO] [TRAIN] epoch: 153, iter: 37900/120000, loss: 0.9186, lr: 0.007106, batch_cost: 0.2259, reader_cost: 0.00083, ips: 13.2773 samples/sec | ETA 05:09:10
- 2022-04-13 13:48:30 [INFO] [TRAIN] epoch: 154, iter: 37950/120000, loss: 0.9323, lr: 0.007103, batch_cost: 0.2571, reader_cost: 0.06268, ips: 11.6669 samples/sec | ETA 05:51:38
- 2022-04-13 13:48:41 [INFO] [TRAIN] epoch: 154, iter: 38000/120000, loss: 0.8937, lr: 0.007099, batch_cost: 0.2139, reader_cost: 0.00092, ips: 14.0271 samples/sec | ETA 04:52:17
- 2022-04-13 13:48:51 [INFO] [TRAIN] epoch: 154, iter: 38050/120000, loss: 0.9251, lr: 0.007095, batch_cost: 0.2050, reader_cost: 0.00125, ips: 14.6326 samples/sec | ETA 04:40:01
- 2022-04-13 13:49:02 [INFO] [TRAIN] epoch: 154, iter: 38100/120000, loss: 0.9137, lr: 0.007091, batch_cost: 0.2178, reader_cost: 0.00043, ips: 13.7739 samples/sec | ETA 04:57:18
- 2022-04-13 13:49:12 [INFO] [TRAIN] epoch: 154, iter: 38150/120000, loss: 0.9578, lr: 0.007087, batch_cost: 0.2046, reader_cost: 0.00106, ips: 14.6596 samples/sec | ETA 04:39:10
- 2022-04-13 13:49:25 [INFO] [TRAIN] epoch: 155, iter: 38200/120000, loss: 0.9560, lr: 0.007083, batch_cost: 0.2575, reader_cost: 0.05980, ips: 11.6519 samples/sec | ETA 05:51:00
- 2022-04-13 13:49:35 [INFO] [TRAIN] epoch: 155, iter: 38250/120000, loss: 0.9451, lr: 0.007079, batch_cost: 0.2146, reader_cost: 0.00113, ips: 13.9813 samples/sec | ETA 04:52:21
- 2022-04-13 13:49:46 [INFO] [TRAIN] epoch: 155, iter: 38300/120000, loss: 0.8969, lr: 0.007075, batch_cost: 0.2160, reader_cost: 0.00075, ips: 13.8909 samples/sec | ETA 04:54:04
- 2022-04-13 13:49:57 [INFO] [TRAIN] epoch: 155, iter: 38350/120000, loss: 0.9436, lr: 0.007071, batch_cost: 0.2098, reader_cost: 0.00157, ips: 14.2969 samples/sec | ETA 04:45:33
- 2022-04-13 13:50:08 [INFO] [TRAIN] epoch: 155, iter: 38400/120000, loss: 0.9843, lr: 0.007067, batch_cost: 0.2190, reader_cost: 0.00098, ips: 13.6991 samples/sec | ETA 04:57:49
- 2022-04-13 13:50:22 [INFO] [TRAIN] epoch: 156, iter: 38450/120000, loss: 0.9114, lr: 0.007064, batch_cost: 0.2943, reader_cost: 0.06574, ips: 10.1949 samples/sec | ETA 06:39:57
- 2022-04-13 13:50:32 [INFO] [TRAIN] epoch: 156, iter: 38500/120000, loss: 0.8749, lr: 0.007060, batch_cost: 0.1978, reader_cost: 0.00079, ips: 15.1637 samples/sec | ETA 04:28:43
- 2022-04-13 13:50:43 [INFO] [TRAIN] epoch: 156, iter: 38550/120000, loss: 0.9126, lr: 0.007056, batch_cost: 0.2104, reader_cost: 0.00114, ips: 14.2617 samples/sec | ETA 04:45:33
- 2022-04-13 13:50:54 [INFO] [TRAIN] epoch: 156, iter: 38600/120000, loss: 0.9240, lr: 0.007052, batch_cost: 0.2317, reader_cost: 0.00098, ips: 12.9456 samples/sec | ETA 05:14:23
- 2022-04-13 13:51:05 [INFO] [TRAIN] epoch: 156, iter: 38650/120000, loss: 0.9287, lr: 0.007048, batch_cost: 0.2130, reader_cost: 0.00068, ips: 14.0867 samples/sec | ETA 04:48:44
- 2022-04-13 13:51:19 [INFO] [TRAIN] epoch: 157, iter: 38700/120000, loss: 0.9955, lr: 0.007044, batch_cost: 0.2687, reader_cost: 0.05561, ips: 11.1648 samples/sec | ETA 06:04:05
- 2022-04-13 13:51:28 [INFO] [TRAIN] epoch: 157, iter: 38750/120000, loss: 0.8697, lr: 0.007040, batch_cost: 0.1942, reader_cost: 0.00133, ips: 15.4493 samples/sec | ETA 04:22:57
- 2022-04-13 13:51:38 [INFO] [TRAIN] epoch: 157, iter: 38800/120000, loss: 0.9072, lr: 0.007036, batch_cost: 0.2031, reader_cost: 0.00096, ips: 14.7735 samples/sec | ETA 04:34:48
- 2022-04-13 13:51:49 [INFO] [TRAIN] epoch: 157, iter: 38850/120000, loss: 0.9216, lr: 0.007032, batch_cost: 0.2079, reader_cost: 0.00099, ips: 14.4268 samples/sec | ETA 04:41:14
- 2022-04-13 13:52:00 [INFO] [TRAIN] epoch: 157, iter: 38900/120000, loss: 0.9379, lr: 0.007028, batch_cost: 0.2161, reader_cost: 0.00099, ips: 13.8834 samples/sec | ETA 04:52:04
- 2022-04-13 13:52:13 [INFO] [TRAIN] epoch: 158, iter: 38950/120000, loss: 0.9064, lr: 0.007025, batch_cost: 0.2639, reader_cost: 0.05429, ips: 11.3661 samples/sec | ETA 05:56:32
- 2022-04-13 13:52:23 [INFO] [TRAIN] epoch: 158, iter: 39000/120000, loss: 0.8876, lr: 0.007021, batch_cost: 0.2125, reader_cost: 0.00143, ips: 14.1153 samples/sec | ETA 04:46:55
- 2022-04-13 13:52:35 [INFO] [TRAIN] epoch: 158, iter: 39050/120000, loss: 0.8959, lr: 0.007017, batch_cost: 0.2381, reader_cost: 0.00071, ips: 12.6015 samples/sec | ETA 05:21:11
- 2022-04-13 13:52:46 [INFO] [TRAIN] epoch: 158, iter: 39100/120000, loss: 0.8909, lr: 0.007013, batch_cost: 0.2122, reader_cost: 0.00129, ips: 14.1382 samples/sec | ETA 04:46:06
- 2022-04-13 13:52:57 [INFO] [TRAIN] epoch: 158, iter: 39150/120000, loss: 0.8940, lr: 0.007009, batch_cost: 0.2150, reader_cost: 0.00153, ips: 13.9526 samples/sec | ETA 04:49:43
- 2022-04-13 13:53:10 [INFO] [TRAIN] epoch: 159, iter: 39200/120000, loss: 0.8896, lr: 0.007005, batch_cost: 0.2676, reader_cost: 0.05653, ips: 11.2098 samples/sec | ETA 06:00:23
- 2022-04-13 13:53:21 [INFO] [TRAIN] epoch: 159, iter: 39250/120000, loss: 0.8947, lr: 0.007001, batch_cost: 0.2092, reader_cost: 0.00080, ips: 14.3435 samples/sec | ETA 04:41:29
- 2022-04-13 13:53:31 [INFO] [TRAIN] epoch: 159, iter: 39300/120000, loss: 0.9021, lr: 0.006997, batch_cost: 0.2000, reader_cost: 0.00102, ips: 15.0014 samples/sec | ETA 04:28:58
- 2022-04-13 13:53:42 [INFO] [TRAIN] epoch: 159, iter: 39350/120000, loss: 0.8843, lr: 0.006993, batch_cost: 0.2219, reader_cost: 0.00115, ips: 13.5218 samples/sec | ETA 04:58:13
- 2022-04-13 13:53:52 [INFO] [TRAIN] epoch: 159, iter: 39400/120000, loss: 0.9228, lr: 0.006989, batch_cost: 0.2033, reader_cost: 0.00114, ips: 14.7576 samples/sec | ETA 04:33:04
- 2022-04-13 13:54:06 [INFO] [TRAIN] epoch: 160, iter: 39450/120000, loss: 0.9259, lr: 0.006986, batch_cost: 0.2787, reader_cost: 0.05132, ips: 10.7626 samples/sec | ETA 06:14:12
- 2022-04-13 13:54:16 [INFO] [TRAIN] epoch: 160, iter: 39500/120000, loss: 0.9293, lr: 0.006982, batch_cost: 0.2081, reader_cost: 0.00098, ips: 14.4166 samples/sec | ETA 04:39:11
- 2022-04-13 13:54:26 [INFO] [TRAIN] epoch: 160, iter: 39550/120000, loss: 0.8905, lr: 0.006978, batch_cost: 0.1978, reader_cost: 0.00123, ips: 15.1645 samples/sec | ETA 04:25:15
- 2022-04-13 13:54:39 [INFO] [TRAIN] epoch: 160, iter: 39600/120000, loss: 0.9211, lr: 0.006974, batch_cost: 0.2500, reader_cost: 0.00053, ips: 12.0007 samples/sec | ETA 05:34:58
- 2022-04-13 13:54:49 [INFO] [TRAIN] epoch: 160, iter: 39650/120000, loss: 0.9489, lr: 0.006970, batch_cost: 0.2064, reader_cost: 0.00086, ips: 14.5379 samples/sec | ETA 04:36:20
- 2022-04-13 13:55:02 [INFO] [TRAIN] epoch: 161, iter: 39700/120000, loss: 0.8786, lr: 0.006966, batch_cost: 0.2636, reader_cost: 0.05396, ips: 11.3800 samples/sec | ETA 05:52:48
- 2022-04-13 13:55:12 [INFO] [TRAIN] epoch: 161, iter: 39750/120000, loss: 0.9288, lr: 0.006962, batch_cost: 0.1926, reader_cost: 0.00086, ips: 15.5785 samples/sec | ETA 04:17:33
- 2022-04-13 13:55:22 [INFO] [TRAIN] epoch: 161, iter: 39800/120000, loss: 0.9030, lr: 0.006958, batch_cost: 0.2114, reader_cost: 0.00086, ips: 14.1942 samples/sec | ETA 04:42:30
- 2022-04-13 13:55:33 [INFO] [TRAIN] epoch: 161, iter: 39850/120000, loss: 0.8918, lr: 0.006954, batch_cost: 0.2206, reader_cost: 0.00073, ips: 13.6004 samples/sec | ETA 04:54:39
- 2022-04-13 13:55:43 [INFO] [TRAIN] epoch: 161, iter: 39900/120000, loss: 0.8713, lr: 0.006950, batch_cost: 0.2025, reader_cost: 0.00079, ips: 14.8118 samples/sec | ETA 04:30:23
- 2022-04-13 13:55:57 [INFO] [TRAIN] epoch: 162, iter: 39950/120000, loss: 0.9315, lr: 0.006947, batch_cost: 0.2687, reader_cost: 0.04579, ips: 11.1633 samples/sec | ETA 05:58:32
- 2022-04-13 13:56:08 [INFO] [TRAIN] epoch: 162, iter: 40000/120000, loss: 0.9019, lr: 0.006943, batch_cost: 0.2240, reader_cost: 0.00088, ips: 13.3949 samples/sec | ETA 04:58:37
- 2022-04-13 13:56:08 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1951 - reader cost: 0.1517
- 2022-04-13 13:56:33 [INFO] [EVAL] #Images: 500 mIoU: 0.7399 Acc: 0.9552 Kappa: 0.9418 Dice: 0.8413
- 2022-04-13 13:56:33 [INFO] [EVAL] Class IoU:
- [0.976 0.815 0.9175 0.495 0.6035 0.6056 0.6879 0.7674 0.9198 0.6338
- 0.9425 0.7927 0.527 0.9417 0.707 0.8048 0.7075 0.4632 0.7502]
- 2022-04-13 13:56:33 [INFO] [EVAL] Class Precision:
- [0.9869 0.9003 0.9479 0.8132 0.7512 0.8153 0.8084 0.8814 0.9581 0.8113
- 0.9643 0.8591 0.8306 0.9661 0.8783 0.969 0.7999 0.8288 0.8331]
- 2022-04-13 13:56:33 [INFO] [EVAL] Class Recall:
- [0.9888 0.8958 0.9663 0.5585 0.7542 0.7019 0.822 0.8558 0.9584 0.7434
- 0.9766 0.9111 0.5905 0.9738 0.7838 0.8261 0.8596 0.5122 0.8829]
- 2022-04-13 13:56:34 [INFO] [EVAL] The model with the best validation mIoU (0.7399) was saved at iter 40000.
- 2022-04-13 13:56:44 [INFO] [TRAIN] epoch: 162, iter: 40050/120000, loss: 0.8911, lr: 0.006939, batch_cost: 0.2132, reader_cost: 0.00151, ips: 14.0702 samples/sec | ETA 04:44:06
- 2022-04-13 13:56:55 [INFO] [TRAIN] epoch: 162, iter: 40100/120000, loss: 0.8571, lr: 0.006935, batch_cost: 0.2080, reader_cost: 0.00067, ips: 14.4243 samples/sec | ETA 04:36:57
- 2022-04-13 13:57:06 [INFO] [TRAIN] epoch: 162, iter: 40150/120000, loss: 0.9224, lr: 0.006931, batch_cost: 0.2245, reader_cost: 0.00133, ips: 13.3610 samples/sec | ETA 04:58:49
- 2022-04-13 13:57:19 [INFO] [TRAIN] epoch: 163, iter: 40200/120000, loss: 0.9209, lr: 0.006927, batch_cost: 0.2667, reader_cost: 0.05375, ips: 11.2474 samples/sec | ETA 05:54:44
- 2022-04-13 13:57:29 [INFO] [TRAIN] epoch: 163, iter: 40250/120000, loss: 0.9381, lr: 0.006923, batch_cost: 0.2008, reader_cost: 0.00140, ips: 14.9427 samples/sec | ETA 04:26:51
- 2022-04-13 13:57:41 [INFO] [TRAIN] epoch: 163, iter: 40300/120000, loss: 0.8918, lr: 0.006919, batch_cost: 0.2323, reader_cost: 0.00081, ips: 12.9128 samples/sec | ETA 05:08:36
- 2022-04-13 13:57:51 [INFO] [TRAIN] epoch: 163, iter: 40350/120000, loss: 0.9276, lr: 0.006915, batch_cost: 0.2108, reader_cost: 0.00104, ips: 14.2325 samples/sec | ETA 04:39:49
- 2022-04-13 13:58:02 [INFO] [TRAIN] epoch: 163, iter: 40400/120000, loss: 0.8633, lr: 0.006911, batch_cost: 0.2202, reader_cost: 0.00048, ips: 13.6227 samples/sec | ETA 04:52:09
- 2022-04-13 13:58:16 [INFO] [TRAIN] epoch: 164, iter: 40450/120000, loss: 0.9327, lr: 0.006907, batch_cost: 0.2628, reader_cost: 0.05480, ips: 11.4150 samples/sec | ETA 05:48:26
- 2022-04-13 13:58:26 [INFO] [TRAIN] epoch: 164, iter: 40500/120000, loss: 0.9057, lr: 0.006904, batch_cost: 0.2069, reader_cost: 0.00077, ips: 14.5019 samples/sec | ETA 04:34:06
- 2022-04-13 13:58:36 [INFO] [TRAIN] epoch: 164, iter: 40550/120000, loss: 0.9106, lr: 0.006900, batch_cost: 0.1997, reader_cost: 0.00121, ips: 15.0215 samples/sec | ETA 04:24:27
- 2022-04-13 13:58:47 [INFO] [TRAIN] epoch: 164, iter: 40600/120000, loss: 0.9359, lr: 0.006896, batch_cost: 0.2170, reader_cost: 0.00061, ips: 13.8241 samples/sec | ETA 04:47:10
- 2022-04-13 13:58:57 [INFO] [TRAIN] epoch: 164, iter: 40650/120000, loss: 0.8932, lr: 0.006892, batch_cost: 0.2105, reader_cost: 0.00077, ips: 14.2537 samples/sec | ETA 04:38:20
- 2022-04-13 13:59:11 [INFO] [TRAIN] epoch: 165, iter: 40700/120000, loss: 0.9367, lr: 0.006888, batch_cost: 0.2770, reader_cost: 0.05352, ips: 10.8318 samples/sec | ETA 06:06:03
- 2022-04-13 13:59:21 [INFO] [TRAIN] epoch: 165, iter: 40750/120000, loss: 0.8968, lr: 0.006884, batch_cost: 0.2036, reader_cost: 0.00066, ips: 14.7325 samples/sec | ETA 04:28:57
- 2022-04-13 13:59:32 [INFO] [TRAIN] epoch: 165, iter: 40800/120000, loss: 0.9024, lr: 0.006880, batch_cost: 0.2166, reader_cost: 0.00113, ips: 13.8482 samples/sec | ETA 04:45:57
- 2022-04-13 13:59:42 [INFO] [TRAIN] epoch: 165, iter: 40850/120000, loss: 0.8955, lr: 0.006876, batch_cost: 0.2065, reader_cost: 0.00068, ips: 14.5295 samples/sec | ETA 04:32:22
- 2022-04-13 13:59:53 [INFO] [TRAIN] epoch: 165, iter: 40900/120000, loss: 0.9136, lr: 0.006872, batch_cost: 0.2194, reader_cost: 0.00088, ips: 13.6709 samples/sec | ETA 04:49:18
- 2022-04-13 14:00:07 [INFO] [TRAIN] epoch: 166, iter: 40950/120000, loss: 0.8846, lr: 0.006868, batch_cost: 0.2625, reader_cost: 0.06089, ips: 11.4293 samples/sec | ETA 05:45:49
- 2022-04-13 14:00:17 [INFO] [TRAIN] epoch: 166, iter: 41000/120000, loss: 0.8806, lr: 0.006864, batch_cost: 0.2089, reader_cost: 0.00104, ips: 14.3608 samples/sec | ETA 04:35:03
- 2022-04-13 14:00:27 [INFO] [TRAIN] epoch: 166, iter: 41050/120000, loss: 0.9322, lr: 0.006861, batch_cost: 0.2072, reader_cost: 0.00085, ips: 14.4796 samples/sec | ETA 04:32:37
- 2022-04-13 14:00:38 [INFO] [TRAIN] epoch: 166, iter: 41100/120000, loss: 0.9015, lr: 0.006857, batch_cost: 0.2046, reader_cost: 0.00093, ips: 14.6604 samples/sec | ETA 04:29:05
- 2022-04-13 14:00:48 [INFO] [TRAIN] epoch: 166, iter: 41150/120000, loss: 0.9031, lr: 0.006853, batch_cost: 0.2061, reader_cost: 0.00032, ips: 14.5577 samples/sec | ETA 04:30:49
- 2022-04-13 14:01:02 [INFO] [TRAIN] epoch: 167, iter: 41200/120000, loss: 0.9001, lr: 0.006849, batch_cost: 0.2795, reader_cost: 0.05663, ips: 10.7332 samples/sec | ETA 06:07:05
- 2022-04-13 14:01:13 [INFO] [TRAIN] epoch: 167, iter: 41250/120000, loss: 0.8993, lr: 0.006845, batch_cost: 0.2183, reader_cost: 0.00167, ips: 13.7412 samples/sec | ETA 04:46:32
- 2022-04-13 14:01:24 [INFO] [TRAIN] epoch: 167, iter: 41300/120000, loss: 0.8847, lr: 0.006841, batch_cost: 0.2254, reader_cost: 0.00042, ips: 13.3097 samples/sec | ETA 04:55:38
- 2022-04-13 14:01:34 [INFO] [TRAIN] epoch: 167, iter: 41350/120000, loss: 0.9022, lr: 0.006837, batch_cost: 0.1964, reader_cost: 0.00172, ips: 15.2778 samples/sec | ETA 04:17:23
- 2022-04-13 14:01:46 [INFO] [TRAIN] epoch: 167, iter: 41400/120000, loss: 0.8861, lr: 0.006833, batch_cost: 0.2501, reader_cost: 0.00093, ips: 11.9955 samples/sec | ETA 05:27:37
- 2022-04-13 14:01:59 [INFO] [TRAIN] epoch: 168, iter: 41450/120000, loss: 0.9551, lr: 0.006829, batch_cost: 0.2606, reader_cost: 0.06373, ips: 11.5105 samples/sec | ETA 05:41:12
- 2022-04-13 14:02:10 [INFO] [TRAIN] epoch: 168, iter: 41500/120000, loss: 0.9355, lr: 0.006825, batch_cost: 0.2164, reader_cost: 0.00148, ips: 13.8616 samples/sec | ETA 04:43:09
- 2022-04-13 14:02:21 [INFO] [TRAIN] epoch: 168, iter: 41550/120000, loss: 0.9312, lr: 0.006821, batch_cost: 0.2097, reader_cost: 0.00084, ips: 14.3044 samples/sec | ETA 04:34:12
- 2022-04-13 14:02:33 [INFO] [TRAIN] epoch: 168, iter: 41600/120000, loss: 0.8692, lr: 0.006818, batch_cost: 0.2502, reader_cost: 0.00077, ips: 11.9899 samples/sec | ETA 05:26:56
- 2022-04-13 14:02:44 [INFO] [TRAIN] epoch: 168, iter: 41650/120000, loss: 0.9013, lr: 0.006814, batch_cost: 0.2160, reader_cost: 0.00165, ips: 13.8900 samples/sec | ETA 04:42:02
- 2022-04-13 14:02:58 [INFO] [TRAIN] epoch: 169, iter: 41700/120000, loss: 0.9251, lr: 0.006810, batch_cost: 0.2856, reader_cost: 0.05742, ips: 10.5051 samples/sec | ETA 06:12:40
- 2022-04-13 14:03:09 [INFO] [TRAIN] epoch: 169, iter: 41750/120000, loss: 0.9273, lr: 0.006806, batch_cost: 0.2053, reader_cost: 0.00163, ips: 14.6159 samples/sec | ETA 04:27:41
- 2022-04-13 14:03:19 [INFO] [TRAIN] epoch: 169, iter: 41800/120000, loss: 0.9259, lr: 0.006802, batch_cost: 0.2179, reader_cost: 0.00031, ips: 13.7647 samples/sec | ETA 04:44:03
- 2022-04-13 14:03:30 [INFO] [TRAIN] epoch: 169, iter: 41850/120000, loss: 0.8975, lr: 0.006798, batch_cost: 0.2079, reader_cost: 0.00125, ips: 14.4300 samples/sec | ETA 04:30:47
- 2022-04-13 14:03:41 [INFO] [TRAIN] epoch: 169, iter: 41900/120000, loss: 0.8820, lr: 0.006794, batch_cost: 0.2296, reader_cost: 0.00148, ips: 13.0656 samples/sec | ETA 04:58:52
- 2022-04-13 14:03:55 [INFO] [TRAIN] epoch: 170, iter: 41950/120000, loss: 0.8878, lr: 0.006790, batch_cost: 0.2729, reader_cost: 0.04997, ips: 10.9927 samples/sec | ETA 05:55:00
- 2022-04-13 14:04:05 [INFO] [TRAIN] epoch: 170, iter: 42000/120000, loss: 0.8821, lr: 0.006786, batch_cost: 0.2040, reader_cost: 0.00127, ips: 14.7025 samples/sec | ETA 04:25:15
- 2022-04-13 14:04:16 [INFO] [TRAIN] epoch: 170, iter: 42050/120000, loss: 0.9433, lr: 0.006782, batch_cost: 0.2191, reader_cost: 0.00122, ips: 13.6947 samples/sec | ETA 04:44:35
- 2022-04-13 14:04:26 [INFO] [TRAIN] epoch: 170, iter: 42100/120000, loss: 0.8869, lr: 0.006778, batch_cost: 0.2029, reader_cost: 0.00058, ips: 14.7869 samples/sec | ETA 04:23:24
- 2022-04-13 14:04:36 [INFO] [TRAIN] epoch: 170, iter: 42150/120000, loss: 0.9285, lr: 0.006774, batch_cost: 0.1985, reader_cost: 0.00055, ips: 15.1135 samples/sec | ETA 04:17:33
- 2022-04-13 14:04:50 [INFO] [TRAIN] epoch: 171, iter: 42200/120000, loss: 0.9144, lr: 0.006771, batch_cost: 0.2717, reader_cost: 0.05156, ips: 11.0426 samples/sec | ETA 05:52:16
- 2022-04-13 14:05:00 [INFO] [TRAIN] epoch: 171, iter: 42250/120000, loss: 0.9157, lr: 0.006767, batch_cost: 0.1972, reader_cost: 0.00144, ips: 15.2157 samples/sec | ETA 04:15:29
- 2022-04-13 14:05:10 [INFO] [TRAIN] epoch: 171, iter: 42300/120000, loss: 0.9048, lr: 0.006763, batch_cost: 0.2059, reader_cost: 0.00060, ips: 14.5683 samples/sec | ETA 04:26:40
- 2022-04-13 14:05:21 [INFO] [TRAIN] epoch: 171, iter: 42350/120000, loss: 0.8864, lr: 0.006759, batch_cost: 0.2194, reader_cost: 0.00182, ips: 13.6739 samples/sec | ETA 04:43:56
- 2022-04-13 14:05:31 [INFO] [TRAIN] epoch: 171, iter: 42400/120000, loss: 0.9014, lr: 0.006755, batch_cost: 0.2077, reader_cost: 0.00125, ips: 14.4441 samples/sec | ETA 04:28:37
- 2022-04-13 14:05:46 [INFO] [TRAIN] epoch: 172, iter: 42450/120000, loss: 0.9110, lr: 0.006751, batch_cost: 0.2899, reader_cost: 0.05430, ips: 10.3484 samples/sec | ETA 06:14:41
- 2022-04-13 14:05:56 [INFO] [TRAIN] epoch: 172, iter: 42500/120000, loss: 0.9145, lr: 0.006747, batch_cost: 0.2072, reader_cost: 0.00098, ips: 14.4802 samples/sec | ETA 04:27:36
- 2022-04-13 14:06:08 [INFO] [TRAIN] epoch: 172, iter: 42550/120000, loss: 0.8747, lr: 0.006743, batch_cost: 0.2310, reader_cost: 0.00095, ips: 12.9874 samples/sec | ETA 04:58:10
- 2022-04-13 14:06:19 [INFO] [TRAIN] epoch: 172, iter: 42600/120000, loss: 0.9196, lr: 0.006739, batch_cost: 0.2308, reader_cost: 0.00185, ips: 12.9979 samples/sec | ETA 04:57:44
- 2022-04-13 14:06:30 [INFO] [TRAIN] epoch: 172, iter: 42650/120000, loss: 0.8932, lr: 0.006735, batch_cost: 0.2154, reader_cost: 0.00089, ips: 13.9255 samples/sec | ETA 04:37:43
- 2022-04-13 14:06:45 [INFO] [TRAIN] epoch: 173, iter: 42700/120000, loss: 0.9198, lr: 0.006731, batch_cost: 0.2927, reader_cost: 0.05951, ips: 10.2488 samples/sec | ETA 06:17:07
- 2022-04-13 14:06:55 [INFO] [TRAIN] epoch: 173, iter: 42750/120000, loss: 0.8805, lr: 0.006727, batch_cost: 0.2082, reader_cost: 0.00076, ips: 14.4081 samples/sec | ETA 04:28:04
- 2022-04-13 14:07:05 [INFO] [TRAIN] epoch: 173, iter: 42800/120000, loss: 0.8982, lr: 0.006724, batch_cost: 0.2002, reader_cost: 0.00086, ips: 14.9881 samples/sec | ETA 04:17:32
- 2022-04-13 14:07:15 [INFO] [TRAIN] epoch: 173, iter: 42850/120000, loss: 0.8679, lr: 0.006720, batch_cost: 0.2037, reader_cost: 0.00082, ips: 14.7258 samples/sec | ETA 04:21:57
- 2022-04-13 14:07:25 [INFO] [TRAIN] epoch: 173, iter: 42900/120000, loss: 0.9034, lr: 0.006716, batch_cost: 0.1924, reader_cost: 0.00062, ips: 15.5922 samples/sec | ETA 04:07:14
- 2022-04-13 14:07:38 [INFO] [TRAIN] epoch: 174, iter: 42950/120000, loss: 0.8745, lr: 0.006712, batch_cost: 0.2669, reader_cost: 0.06244, ips: 11.2412 samples/sec | ETA 05:42:42
- 2022-04-13 14:07:50 [INFO] [TRAIN] epoch: 174, iter: 43000/120000, loss: 0.9326, lr: 0.006708, batch_cost: 0.2293, reader_cost: 0.00065, ips: 13.0835 samples/sec | ETA 04:54:15
- 2022-04-13 14:08:00 [INFO] [TRAIN] epoch: 174, iter: 43050/120000, loss: 0.9242, lr: 0.006704, batch_cost: 0.2146, reader_cost: 0.00123, ips: 13.9764 samples/sec | ETA 04:35:17
- 2022-04-13 14:08:10 [INFO] [TRAIN] epoch: 174, iter: 43100/120000, loss: 0.9210, lr: 0.006700, batch_cost: 0.1999, reader_cost: 0.00143, ips: 15.0085 samples/sec | ETA 04:16:11
- 2022-04-13 14:08:20 [INFO] [TRAIN] epoch: 174, iter: 43150/120000, loss: 0.9042, lr: 0.006696, batch_cost: 0.1888, reader_cost: 0.00076, ips: 15.8895 samples/sec | ETA 04:01:49
- 2022-04-13 14:08:34 [INFO] [TRAIN] epoch: 175, iter: 43200/120000, loss: 0.8887, lr: 0.006692, batch_cost: 0.2752, reader_cost: 0.05299, ips: 10.9013 samples/sec | ETA 05:52:15
- 2022-04-13 14:08:44 [INFO] [TRAIN] epoch: 175, iter: 43250/120000, loss: 0.9034, lr: 0.006688, batch_cost: 0.2050, reader_cost: 0.00062, ips: 14.6336 samples/sec | ETA 04:22:14
- 2022-04-13 14:08:54 [INFO] [TRAIN] epoch: 175, iter: 43300/120000, loss: 0.9330, lr: 0.006684, batch_cost: 0.2026, reader_cost: 0.00105, ips: 14.8066 samples/sec | ETA 04:19:00
- 2022-04-13 14:09:04 [INFO] [TRAIN] epoch: 175, iter: 43350/120000, loss: 0.9133, lr: 0.006680, batch_cost: 0.1995, reader_cost: 0.00077, ips: 15.0354 samples/sec | ETA 04:14:53
- 2022-04-13 14:09:13 [INFO] [TRAIN] epoch: 175, iter: 43400/120000, loss: 0.9105, lr: 0.006676, batch_cost: 0.1876, reader_cost: 0.00049, ips: 15.9948 samples/sec | ETA 03:59:27
- 2022-04-13 14:09:27 [INFO] [TRAIN] epoch: 176, iter: 43450/120000, loss: 0.9018, lr: 0.006673, batch_cost: 0.2711, reader_cost: 0.05160, ips: 11.0678 samples/sec | ETA 05:45:49
- 2022-04-13 14:09:38 [INFO] [TRAIN] epoch: 176, iter: 43500/120000, loss: 0.8839, lr: 0.006669, batch_cost: 0.2260, reader_cost: 0.00155, ips: 13.2771 samples/sec | ETA 04:48:05
- 2022-04-13 14:09:49 [INFO] [TRAIN] epoch: 176, iter: 43550/120000, loss: 0.8986, lr: 0.006665, batch_cost: 0.2107, reader_cost: 0.00060, ips: 14.2416 samples/sec | ETA 04:28:24
- 2022-04-13 14:10:00 [INFO] [TRAIN] epoch: 176, iter: 43600/120000, loss: 0.9123, lr: 0.006661, batch_cost: 0.2319, reader_cost: 0.00066, ips: 12.9339 samples/sec | ETA 04:55:20
- 2022-04-13 14:10:14 [INFO] [TRAIN] epoch: 177, iter: 43650/120000, loss: 0.9047, lr: 0.006657, batch_cost: 0.2684, reader_cost: 0.05894, ips: 11.1755 samples/sec | ETA 05:41:35
- 2022-04-13 14:10:24 [INFO] [TRAIN] epoch: 177, iter: 43700/120000, loss: 0.9109, lr: 0.006653, batch_cost: 0.2020, reader_cost: 0.00091, ips: 14.8547 samples/sec | ETA 04:16:49
- 2022-04-13 14:10:34 [INFO] [TRAIN] epoch: 177, iter: 43750/120000, loss: 0.9262, lr: 0.006649, batch_cost: 0.2075, reader_cost: 0.00158, ips: 14.4598 samples/sec | ETA 04:23:39
- 2022-04-13 14:10:46 [INFO] [TRAIN] epoch: 177, iter: 43800/120000, loss: 0.9010, lr: 0.006645, batch_cost: 0.2316, reader_cost: 0.00080, ips: 12.9552 samples/sec | ETA 04:54:05
- 2022-04-13 14:10:57 [INFO] [TRAIN] epoch: 177, iter: 43850/120000, loss: 0.8945, lr: 0.006641, batch_cost: 0.2164, reader_cost: 0.00106, ips: 13.8643 samples/sec | ETA 04:34:37
- 2022-04-13 14:11:10 [INFO] [TRAIN] epoch: 178, iter: 43900/120000, loss: 0.8863, lr: 0.006637, batch_cost: 0.2713, reader_cost: 0.05590, ips: 11.0569 samples/sec | ETA 05:44:07
- 2022-04-13 14:11:21 [INFO] [TRAIN] epoch: 178, iter: 43950/120000, loss: 0.9293, lr: 0.006633, batch_cost: 0.2092, reader_cost: 0.00098, ips: 14.3406 samples/sec | ETA 04:25:09
- 2022-04-13 14:11:31 [INFO] [TRAIN] epoch: 178, iter: 44000/120000, loss: 0.8926, lr: 0.006629, batch_cost: 0.2147, reader_cost: 0.00093, ips: 13.9729 samples/sec | ETA 04:31:57
- 2022-04-13 14:11:31 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1975 - reader cost: 0.1538
- 2022-04-13 14:11:56 [INFO] [EVAL] #Images: 500 mIoU: 0.7181 Acc: 0.9501 Kappa: 0.9351 Dice: 0.8251
- 2022-04-13 14:11:56 [INFO] [EVAL] Class IoU:
- [0.9716 0.7861 0.9064 0.4101 0.4936 0.6062 0.6607 0.7731 0.9158 0.574
- 0.9434 0.7919 0.551 0.9334 0.6464 0.7663 0.6085 0.5508 0.7541]
- 2022-04-13 14:11:56 [INFO] [EVAL] Class Precision:
- [0.9837 0.8749 0.9381 0.825 0.753 0.8284 0.8498 0.8786 0.9527 0.8959
- 0.9626 0.8606 0.7926 0.973 0.6986 0.8366 0.7844 0.7366 0.8492]
- 2022-04-13 14:11:56 [INFO] [EVAL] Class Recall:
- [0.9875 0.8856 0.9641 0.4491 0.5889 0.6933 0.748 0.8655 0.9594 0.615
- 0.9793 0.9084 0.6438 0.9583 0.8964 0.9013 0.7307 0.6859 0.8708]
- 2022-04-13 14:11:57 [INFO] [EVAL] The model with the best validation mIoU (0.7399) was saved at iter 40000.
- 2022-04-13 14:12:07 [INFO] [TRAIN] epoch: 178, iter: 44050/120000, loss: 0.9101, lr: 0.006625, batch_cost: 0.2046, reader_cost: 0.00198, ips: 14.6637 samples/sec | ETA 04:18:58
- 2022-04-13 14:12:18 [INFO] [TRAIN] epoch: 178, iter: 44100/120000, loss: 0.8860, lr: 0.006622, batch_cost: 0.2204, reader_cost: 0.00114, ips: 13.6136 samples/sec | ETA 04:38:45
- 2022-04-13 14:12:32 [INFO] [TRAIN] epoch: 179, iter: 44150/120000, loss: 0.8942, lr: 0.006618, batch_cost: 0.2700, reader_cost: 0.05547, ips: 11.1109 samples/sec | ETA 05:41:19
- 2022-04-13 14:12:43 [INFO] [TRAIN] epoch: 179, iter: 44200/120000, loss: 0.8735, lr: 0.006614, batch_cost: 0.2184, reader_cost: 0.00154, ips: 13.7386 samples/sec | ETA 04:35:51
- 2022-04-13 14:12:54 [INFO] [TRAIN] epoch: 179, iter: 44250/120000, loss: 0.8770, lr: 0.006610, batch_cost: 0.2262, reader_cost: 0.00087, ips: 13.2637 samples/sec | ETA 04:45:33
- 2022-04-13 14:13:04 [INFO] [TRAIN] epoch: 179, iter: 44300/120000, loss: 0.9793, lr: 0.006606, batch_cost: 0.2062, reader_cost: 0.00110, ips: 14.5493 samples/sec | ETA 04:20:09
- 2022-04-13 14:13:15 [INFO] [TRAIN] epoch: 179, iter: 44350/120000, loss: 0.8884, lr: 0.006602, batch_cost: 0.2100, reader_cost: 0.00116, ips: 14.2880 samples/sec | ETA 04:24:44
- 2022-04-13 14:13:28 [INFO] [TRAIN] epoch: 180, iter: 44400/120000, loss: 0.9197, lr: 0.006598, batch_cost: 0.2601, reader_cost: 0.05596, ips: 11.5351 samples/sec | ETA 05:27:41
- 2022-04-13 14:13:38 [INFO] [TRAIN] epoch: 180, iter: 44450/120000, loss: 0.9141, lr: 0.006594, batch_cost: 0.2136, reader_cost: 0.00116, ips: 14.0451 samples/sec | ETA 04:28:57
- 2022-04-13 14:13:50 [INFO] [TRAIN] epoch: 180, iter: 44500/120000, loss: 0.8778, lr: 0.006590, batch_cost: 0.2300, reader_cost: 0.00140, ips: 13.0454 samples/sec | ETA 04:49:22
- 2022-04-13 14:14:00 [INFO] [TRAIN] epoch: 180, iter: 44550/120000, loss: 0.8877, lr: 0.006586, batch_cost: 0.2043, reader_cost: 0.00057, ips: 14.6819 samples/sec | ETA 04:16:56
- 2022-04-13 14:14:10 [INFO] [TRAIN] epoch: 180, iter: 44600/120000, loss: 0.9267, lr: 0.006582, batch_cost: 0.2039, reader_cost: 0.00133, ips: 14.7165 samples/sec | ETA 04:16:10
- 2022-04-13 14:14:24 [INFO] [TRAIN] epoch: 181, iter: 44650/120000, loss: 0.9224, lr: 0.006578, batch_cost: 0.2846, reader_cost: 0.05768, ips: 10.5405 samples/sec | ETA 05:57:25
- 2022-04-13 14:14:35 [INFO] [TRAIN] epoch: 181, iter: 44700/120000, loss: 0.9043, lr: 0.006574, batch_cost: 0.2165, reader_cost: 0.00114, ips: 13.8576 samples/sec | ETA 04:31:41
- 2022-04-13 14:14:46 [INFO] [TRAIN] epoch: 181, iter: 44750/120000, loss: 0.9020, lr: 0.006570, batch_cost: 0.2225, reader_cost: 0.00070, ips: 13.4820 samples/sec | ETA 04:39:04
- 2022-04-13 14:14:57 [INFO] [TRAIN] epoch: 181, iter: 44800/120000, loss: 0.8487, lr: 0.006567, batch_cost: 0.2109, reader_cost: 0.00085, ips: 14.2235 samples/sec | ETA 04:24:21
- 2022-04-13 14:15:08 [INFO] [TRAIN] epoch: 181, iter: 44850/120000, loss: 0.8874, lr: 0.006563, batch_cost: 0.2119, reader_cost: 0.00053, ips: 14.1562 samples/sec | ETA 04:25:25
- 2022-04-13 14:15:21 [INFO] [TRAIN] epoch: 182, iter: 44900/120000, loss: 0.9071, lr: 0.006559, batch_cost: 0.2661, reader_cost: 0.05440, ips: 11.2743 samples/sec | ETA 05:33:03
- 2022-04-13 14:15:32 [INFO] [TRAIN] epoch: 182, iter: 44950/120000, loss: 0.9189, lr: 0.006555, batch_cost: 0.2289, reader_cost: 0.00086, ips: 13.1034 samples/sec | ETA 04:46:22
- 2022-04-13 14:15:42 [INFO] [TRAIN] epoch: 182, iter: 45000/120000, loss: 0.9290, lr: 0.006551, batch_cost: 0.1999, reader_cost: 0.00066, ips: 15.0046 samples/sec | ETA 04:09:55
- 2022-04-13 14:15:53 [INFO] [TRAIN] epoch: 182, iter: 45050/120000, loss: 0.8817, lr: 0.006547, batch_cost: 0.2190, reader_cost: 0.00068, ips: 13.7011 samples/sec | ETA 04:33:31
- 2022-04-13 14:16:04 [INFO] [TRAIN] epoch: 182, iter: 45100/120000, loss: 0.8632, lr: 0.006543, batch_cost: 0.2066, reader_cost: 0.00056, ips: 14.5230 samples/sec | ETA 04:17:52
- 2022-04-13 14:16:17 [INFO] [TRAIN] epoch: 183, iter: 45150/120000, loss: 0.8975, lr: 0.006539, batch_cost: 0.2686, reader_cost: 0.06060, ips: 11.1686 samples/sec | ETA 05:35:05
- 2022-04-13 14:16:28 [INFO] [TRAIN] epoch: 183, iter: 45200/120000, loss: 0.8983, lr: 0.006535, batch_cost: 0.2180, reader_cost: 0.00133, ips: 13.7640 samples/sec | ETA 04:31:43
- 2022-04-13 14:16:39 [INFO] [TRAIN] epoch: 183, iter: 45250/120000, loss: 0.8808, lr: 0.006531, batch_cost: 0.2207, reader_cost: 0.00108, ips: 13.5909 samples/sec | ETA 04:35:00
- 2022-04-13 14:16:49 [INFO] [TRAIN] epoch: 183, iter: 45300/120000, loss: 0.8749, lr: 0.006527, batch_cost: 0.2003, reader_cost: 0.00103, ips: 14.9776 samples/sec | ETA 04:09:22
- 2022-04-13 14:16:59 [INFO] [TRAIN] epoch: 183, iter: 45350/120000, loss: 0.9113, lr: 0.006523, batch_cost: 0.2052, reader_cost: 0.00093, ips: 14.6218 samples/sec | ETA 04:15:16
- 2022-04-13 14:17:13 [INFO] [TRAIN] epoch: 184, iter: 45400/120000, loss: 0.9283, lr: 0.006519, batch_cost: 0.2841, reader_cost: 0.05564, ips: 10.5604 samples/sec | ETA 05:53:12
- 2022-04-13 14:17:23 [INFO] [TRAIN] epoch: 184, iter: 45450/120000, loss: 0.9210, lr: 0.006515, batch_cost: 0.2003, reader_cost: 0.00043, ips: 14.9757 samples/sec | ETA 04:08:54
- 2022-04-13 14:17:34 [INFO] [TRAIN] epoch: 184, iter: 45500/120000, loss: 0.8962, lr: 0.006512, batch_cost: 0.2112, reader_cost: 0.00074, ips: 14.2072 samples/sec | ETA 04:22:11
- 2022-04-13 14:17:44 [INFO] [TRAIN] epoch: 184, iter: 45550/120000, loss: 0.9080, lr: 0.006508, batch_cost: 0.2047, reader_cost: 0.00101, ips: 14.6527 samples/sec | ETA 04:14:02
- 2022-04-13 14:17:56 [INFO] [TRAIN] epoch: 184, iter: 45600/120000, loss: 0.8822, lr: 0.006504, batch_cost: 0.2351, reader_cost: 0.00108, ips: 12.7606 samples/sec | ETA 04:51:31
- 2022-04-13 14:18:10 [INFO] [TRAIN] epoch: 185, iter: 45650/120000, loss: 0.9109, lr: 0.006500, batch_cost: 0.2701, reader_cost: 0.05888, ips: 11.1067 samples/sec | ETA 05:34:42
- 2022-04-13 14:18:20 [INFO] [TRAIN] epoch: 185, iter: 45700/120000, loss: 0.9250, lr: 0.006496, batch_cost: 0.2123, reader_cost: 0.00094, ips: 14.1309 samples/sec | ETA 04:22:53
- 2022-04-13 14:18:31 [INFO] [TRAIN] epoch: 185, iter: 45750/120000, loss: 0.8815, lr: 0.006492, batch_cost: 0.2116, reader_cost: 0.00098, ips: 14.1756 samples/sec | ETA 04:21:53
- 2022-04-13 14:18:43 [INFO] [TRAIN] epoch: 185, iter: 45800/120000, loss: 0.9013, lr: 0.006488, batch_cost: 0.2400, reader_cost: 0.00113, ips: 12.4986 samples/sec | ETA 04:56:49
- 2022-04-13 14:18:54 [INFO] [TRAIN] epoch: 185, iter: 45850/120000, loss: 0.9547, lr: 0.006484, batch_cost: 0.2199, reader_cost: 0.00123, ips: 13.6424 samples/sec | ETA 04:31:45
- 2022-04-13 14:19:07 [INFO] [TRAIN] epoch: 186, iter: 45900/120000, loss: 0.9334, lr: 0.006480, batch_cost: 0.2719, reader_cost: 0.05331, ips: 11.0338 samples/sec | ETA 05:35:47
- 2022-04-13 14:19:18 [INFO] [TRAIN] epoch: 186, iter: 45950/120000, loss: 0.8717, lr: 0.006476, batch_cost: 0.2094, reader_cost: 0.00101, ips: 14.3233 samples/sec | ETA 04:18:29
- 2022-04-13 14:19:29 [INFO] [TRAIN] epoch: 186, iter: 46000/120000, loss: 0.9071, lr: 0.006472, batch_cost: 0.2262, reader_cost: 0.00063, ips: 13.2607 samples/sec | ETA 04:39:01
- 2022-04-13 14:19:40 [INFO] [TRAIN] epoch: 186, iter: 46050/120000, loss: 0.9210, lr: 0.006468, batch_cost: 0.2140, reader_cost: 0.00121, ips: 14.0214 samples/sec | ETA 04:23:42
- 2022-04-13 14:19:50 [INFO] [TRAIN] epoch: 186, iter: 46100/120000, loss: 0.9441, lr: 0.006464, batch_cost: 0.2089, reader_cost: 0.00127, ips: 14.3626 samples/sec | ETA 04:17:15
- 2022-04-13 14:20:04 [INFO] [TRAIN] epoch: 187, iter: 46150/120000, loss: 0.8955, lr: 0.006460, batch_cost: 0.2700, reader_cost: 0.05377, ips: 11.1126 samples/sec | ETA 05:32:16
- 2022-04-13 14:20:15 [INFO] [TRAIN] epoch: 187, iter: 46200/120000, loss: 0.8923, lr: 0.006456, batch_cost: 0.2200, reader_cost: 0.00077, ips: 13.6392 samples/sec | ETA 04:30:32
- 2022-04-13 14:20:26 [INFO] [TRAIN] epoch: 187, iter: 46250/120000, loss: 0.8948, lr: 0.006452, batch_cost: 0.2280, reader_cost: 0.00132, ips: 13.1601 samples/sec | ETA 04:40:12
- 2022-04-13 14:20:37 [INFO] [TRAIN] epoch: 187, iter: 46300/120000, loss: 0.8629, lr: 0.006449, batch_cost: 0.2193, reader_cost: 0.00068, ips: 13.6795 samples/sec | ETA 04:29:22
- 2022-04-13 14:20:49 [INFO] [TRAIN] epoch: 187, iter: 46350/120000, loss: 0.9524, lr: 0.006445, batch_cost: 0.2421, reader_cost: 0.00069, ips: 12.3933 samples/sec | ETA 04:57:08
- 2022-04-13 14:21:03 [INFO] [TRAIN] epoch: 188, iter: 46400/120000, loss: 0.8861, lr: 0.006441, batch_cost: 0.2805, reader_cost: 0.05897, ips: 10.6944 samples/sec | ETA 05:44:06
- 2022-04-13 14:21:14 [INFO] [TRAIN] epoch: 188, iter: 46450/120000, loss: 0.8832, lr: 0.006437, batch_cost: 0.2216, reader_cost: 0.00108, ips: 13.5405 samples/sec | ETA 04:31:35
- 2022-04-13 14:21:24 [INFO] [TRAIN] epoch: 188, iter: 46500/120000, loss: 0.9100, lr: 0.006433, batch_cost: 0.2018, reader_cost: 0.00089, ips: 14.8686 samples/sec | ETA 04:07:09
- 2022-04-13 14:21:34 [INFO] [TRAIN] epoch: 188, iter: 46550/120000, loss: 0.9332, lr: 0.006429, batch_cost: 0.1990, reader_cost: 0.00074, ips: 15.0764 samples/sec | ETA 04:03:35
- 2022-04-13 14:21:45 [INFO] [TRAIN] epoch: 188, iter: 46600/120000, loss: 0.8749, lr: 0.006425, batch_cost: 0.2188, reader_cost: 0.00091, ips: 13.7129 samples/sec | ETA 04:27:37
- 2022-04-13 14:21:59 [INFO] [TRAIN] epoch: 189, iter: 46650/120000, loss: 0.8915, lr: 0.006421, batch_cost: 0.2748, reader_cost: 0.05923, ips: 10.9161 samples/sec | ETA 05:35:58
- 2022-04-13 14:22:11 [INFO] [TRAIN] epoch: 189, iter: 46700/120000, loss: 0.9120, lr: 0.006417, batch_cost: 0.2340, reader_cost: 0.00084, ips: 12.8227 samples/sec | ETA 04:45:49
- 2022-04-13 14:22:21 [INFO] [TRAIN] epoch: 189, iter: 46750/120000, loss: 0.9193, lr: 0.006413, batch_cost: 0.2054, reader_cost: 0.00101, ips: 14.6067 samples/sec | ETA 04:10:44
- 2022-04-13 14:22:32 [INFO] [TRAIN] epoch: 189, iter: 46800/120000, loss: 0.8977, lr: 0.006409, batch_cost: 0.2158, reader_cost: 0.00076, ips: 13.8989 samples/sec | ETA 04:23:19
- 2022-04-13 14:22:42 [INFO] [TRAIN] epoch: 189, iter: 46850/120000, loss: 0.8529, lr: 0.006405, batch_cost: 0.2052, reader_cost: 0.00123, ips: 14.6192 samples/sec | ETA 04:10:11
- 2022-04-13 14:22:55 [INFO] [TRAIN] epoch: 190, iter: 46900/120000, loss: 0.9228, lr: 0.006401, batch_cost: 0.2634, reader_cost: 0.05831, ips: 11.3875 samples/sec | ETA 05:20:57
- 2022-04-13 14:23:06 [INFO] [TRAIN] epoch: 190, iter: 46950/120000, loss: 0.8661, lr: 0.006397, batch_cost: 0.2076, reader_cost: 0.00071, ips: 14.4491 samples/sec | ETA 04:12:47
- 2022-04-13 14:23:16 [INFO] [TRAIN] epoch: 190, iter: 47000/120000, loss: 0.8739, lr: 0.006393, batch_cost: 0.2095, reader_cost: 0.00092, ips: 14.3181 samples/sec | ETA 04:14:55
- 2022-04-13 14:23:27 [INFO] [TRAIN] epoch: 190, iter: 47050/120000, loss: 0.9556, lr: 0.006389, batch_cost: 0.2160, reader_cost: 0.00078, ips: 13.8913 samples/sec | ETA 04:22:34
- 2022-04-13 14:23:37 [INFO] [TRAIN] epoch: 190, iter: 47100/120000, loss: 0.9327, lr: 0.006386, batch_cost: 0.2108, reader_cost: 0.00081, ips: 14.2338 samples/sec | ETA 04:16:04
- 2022-04-13 14:23:51 [INFO] [TRAIN] epoch: 191, iter: 47150/120000, loss: 0.9083, lr: 0.006382, batch_cost: 0.2623, reader_cost: 0.05743, ips: 11.4368 samples/sec | ETA 05:18:29
- 2022-04-13 14:24:01 [INFO] [TRAIN] epoch: 191, iter: 47200/120000, loss: 0.9265, lr: 0.006378, batch_cost: 0.2006, reader_cost: 0.00066, ips: 14.9568 samples/sec | ETA 04:03:22
- 2022-04-13 14:24:11 [INFO] [TRAIN] epoch: 191, iter: 47250/120000, loss: 0.9129, lr: 0.006374, batch_cost: 0.2090, reader_cost: 0.00138, ips: 14.3509 samples/sec | ETA 04:13:28
- 2022-04-13 14:24:21 [INFO] [TRAIN] epoch: 191, iter: 47300/120000, loss: 0.8870, lr: 0.006370, batch_cost: 0.2029, reader_cost: 0.00062, ips: 14.7864 samples/sec | ETA 04:05:49
- 2022-04-13 14:24:32 [INFO] [TRAIN] epoch: 191, iter: 47350/120000, loss: 0.8869, lr: 0.006366, batch_cost: 0.2131, reader_cost: 0.00118, ips: 14.0771 samples/sec | ETA 04:18:02
- 2022-04-13 14:24:45 [INFO] [TRAIN] epoch: 192, iter: 47400/120000, loss: 0.9193, lr: 0.006362, batch_cost: 0.2644, reader_cost: 0.04941, ips: 11.3461 samples/sec | ETA 05:19:56
- 2022-04-13 14:24:56 [INFO] [TRAIN] epoch: 192, iter: 47450/120000, loss: 0.8781, lr: 0.006358, batch_cost: 0.2230, reader_cost: 0.00130, ips: 13.4513 samples/sec | ETA 04:29:40
- 2022-04-13 14:25:06 [INFO] [TRAIN] epoch: 192, iter: 47500/120000, loss: 0.9176, lr: 0.006354, batch_cost: 0.1984, reader_cost: 0.00067, ips: 15.1195 samples/sec | ETA 03:59:45
- 2022-04-13 14:25:16 [INFO] [TRAIN] epoch: 192, iter: 47550/120000, loss: 0.9006, lr: 0.006350, batch_cost: 0.2048, reader_cost: 0.00094, ips: 14.6458 samples/sec | ETA 04:07:20
- 2022-04-13 14:25:28 [INFO] [TRAIN] epoch: 192, iter: 47600/120000, loss: 0.8777, lr: 0.006346, batch_cost: 0.2380, reader_cost: 0.00083, ips: 12.6072 samples/sec | ETA 04:47:08
- 2022-04-13 14:25:42 [INFO] [TRAIN] epoch: 193, iter: 47650/120000, loss: 0.8825, lr: 0.006342, batch_cost: 0.2727, reader_cost: 0.05796, ips: 11.0004 samples/sec | ETA 05:28:51
- 2022-04-13 14:25:53 [INFO] [TRAIN] epoch: 193, iter: 47700/120000, loss: 0.9177, lr: 0.006338, batch_cost: 0.2139, reader_cost: 0.00158, ips: 14.0266 samples/sec | ETA 04:17:43
- 2022-04-13 14:26:03 [INFO] [TRAIN] epoch: 193, iter: 47750/120000, loss: 0.9260, lr: 0.006334, batch_cost: 0.2113, reader_cost: 0.00109, ips: 14.1990 samples/sec | ETA 04:14:25
- 2022-04-13 14:26:14 [INFO] [TRAIN] epoch: 193, iter: 47800/120000, loss: 0.9223, lr: 0.006330, batch_cost: 0.2152, reader_cost: 0.00091, ips: 13.9408 samples/sec | ETA 04:18:57
- 2022-04-13 14:26:26 [INFO] [TRAIN] epoch: 193, iter: 47850/120000, loss: 0.8868, lr: 0.006326, batch_cost: 0.2337, reader_cost: 0.00084, ips: 12.8387 samples/sec | ETA 04:40:59
- 2022-04-13 14:26:40 [INFO] [TRAIN] epoch: 194, iter: 47900/120000, loss: 0.8817, lr: 0.006322, batch_cost: 0.2798, reader_cost: 0.05360, ips: 10.7209 samples/sec | ETA 05:36:15
- 2022-04-13 14:26:51 [INFO] [TRAIN] epoch: 194, iter: 47950/120000, loss: 0.8968, lr: 0.006318, batch_cost: 0.2358, reader_cost: 0.00102, ips: 12.7249 samples/sec | ETA 04:43:06
- 2022-04-13 14:27:02 [INFO] [TRAIN] epoch: 194, iter: 48000/120000, loss: 0.8819, lr: 0.006315, batch_cost: 0.2034, reader_cost: 0.00121, ips: 14.7468 samples/sec | ETA 04:04:07
- 2022-04-13 14:27:02 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1972 - reader cost: 0.1323
- 2022-04-13 14:27:26 [INFO] [EVAL] #Images: 500 mIoU: 0.7494 Acc: 0.9559 Kappa: 0.9427 Dice: 0.8484
- 2022-04-13 14:27:26 [INFO] [EVAL] Class IoU:
- [0.9768 0.8175 0.9149 0.5141 0.5767 0.6229 0.6907 0.7739 0.9225 0.6589
- 0.941 0.7798 0.5287 0.9425 0.6708 0.8469 0.7765 0.5285 0.7557]
- 2022-04-13 14:27:26 [INFO] [EVAL] Class Precision:
- [0.9858 0.912 0.9424 0.8544 0.7972 0.8113 0.8754 0.9003 0.9614 0.8394
- 0.9598 0.836 0.7742 0.9634 0.9039 0.9728 0.9445 0.8679 0.8819]
- 2022-04-13 14:27:26 [INFO] [EVAL] Class Recall:
- [0.9907 0.8875 0.9691 0.5635 0.6759 0.7285 0.766 0.8465 0.9579 0.7539
- 0.9796 0.9207 0.625 0.9775 0.7223 0.8674 0.8136 0.5747 0.8408]
- 2022-04-13 14:27:27 [INFO] [EVAL] The model with the best validation mIoU (0.7494) was saved at iter 48000.
- 2022-04-13 14:27:38 [INFO] [TRAIN] epoch: 194, iter: 48050/120000, loss: 0.8901, lr: 0.006311, batch_cost: 0.2017, reader_cost: 0.00144, ips: 14.8740 samples/sec | ETA 04:01:51
- 2022-04-13 14:27:48 [INFO] [TRAIN] epoch: 194, iter: 48100/120000, loss: 0.8687, lr: 0.006307, batch_cost: 0.2046, reader_cost: 0.00067, ips: 14.6659 samples/sec | ETA 04:05:07
- 2022-04-13 14:28:02 [INFO] [TRAIN] epoch: 195, iter: 48150/120000, loss: 0.9105, lr: 0.006303, batch_cost: 0.2861, reader_cost: 0.06037, ips: 10.4858 samples/sec | ETA 05:42:36
- 2022-04-13 14:28:13 [INFO] [TRAIN] epoch: 195, iter: 48200/120000, loss: 0.8730, lr: 0.006299, batch_cost: 0.2086, reader_cost: 0.00128, ips: 14.3806 samples/sec | ETA 04:09:38
- 2022-04-13 14:28:23 [INFO] [TRAIN] epoch: 195, iter: 48250/120000, loss: 0.9073, lr: 0.006295, batch_cost: 0.2004, reader_cost: 0.00113, ips: 14.9666 samples/sec | ETA 03:59:42
- 2022-04-13 14:28:35 [INFO] [TRAIN] epoch: 195, iter: 48300/120000, loss: 0.8829, lr: 0.006291, batch_cost: 0.2445, reader_cost: 0.00068, ips: 12.2686 samples/sec | ETA 04:52:12
- 2022-04-13 14:28:45 [INFO] [TRAIN] epoch: 195, iter: 48350/120000, loss: 0.9249, lr: 0.006287, batch_cost: 0.2126, reader_cost: 0.00135, ips: 14.1129 samples/sec | ETA 04:13:50
- 2022-04-13 14:28:59 [INFO] [TRAIN] epoch: 196, iter: 48400/120000, loss: 0.8939, lr: 0.006283, batch_cost: 0.2792, reader_cost: 0.05538, ips: 10.7464 samples/sec | ETA 05:33:08
- 2022-04-13 14:29:11 [INFO] [TRAIN] epoch: 196, iter: 48450/120000, loss: 0.8828, lr: 0.006279, batch_cost: 0.2251, reader_cost: 0.00124, ips: 13.3271 samples/sec | ETA 04:28:26
- 2022-04-13 14:29:21 [INFO] [TRAIN] epoch: 196, iter: 48500/120000, loss: 0.9085, lr: 0.006275, batch_cost: 0.2106, reader_cost: 0.00108, ips: 14.2427 samples/sec | ETA 04:11:00
- 2022-04-13 14:29:32 [INFO] [TRAIN] epoch: 196, iter: 48550/120000, loss: 0.8833, lr: 0.006271, batch_cost: 0.2091, reader_cost: 0.00171, ips: 14.3500 samples/sec | ETA 04:08:57
- 2022-04-13 14:29:42 [INFO] [TRAIN] epoch: 196, iter: 48600/120000, loss: 0.8962, lr: 0.006267, batch_cost: 0.2137, reader_cost: 0.00104, ips: 14.0373 samples/sec | ETA 04:14:19
- 2022-04-13 14:29:56 [INFO] [TRAIN] epoch: 197, iter: 48650/120000, loss: 0.8942, lr: 0.006263, batch_cost: 0.2836, reader_cost: 0.05987, ips: 10.5789 samples/sec | ETA 05:37:13
- 2022-04-13 14:30:07 [INFO] [TRAIN] epoch: 197, iter: 48700/120000, loss: 0.9198, lr: 0.006259, batch_cost: 0.2106, reader_cost: 0.00125, ips: 14.2423 samples/sec | ETA 04:10:18
- 2022-04-13 14:30:17 [INFO] [TRAIN] epoch: 197, iter: 48750/120000, loss: 0.8736, lr: 0.006255, batch_cost: 0.1984, reader_cost: 0.00124, ips: 15.1244 samples/sec | ETA 03:55:32
- 2022-04-13 14:30:29 [INFO] [TRAIN] epoch: 197, iter: 48800/120000, loss: 0.8769, lr: 0.006251, batch_cost: 0.2322, reader_cost: 0.00086, ips: 12.9221 samples/sec | ETA 04:35:29
- 2022-04-13 14:30:38 [INFO] [TRAIN] epoch: 197, iter: 48850/120000, loss: 0.8700, lr: 0.006247, batch_cost: 0.1981, reader_cost: 0.00113, ips: 15.1473 samples/sec | ETA 03:54:51
- 2022-04-13 14:30:52 [INFO] [TRAIN] epoch: 198, iter: 48900/120000, loss: 0.9097, lr: 0.006243, batch_cost: 0.2808, reader_cost: 0.05337, ips: 10.6847 samples/sec | ETA 05:32:43
- 2022-04-13 14:31:03 [INFO] [TRAIN] epoch: 198, iter: 48950/120000, loss: 0.9287, lr: 0.006240, batch_cost: 0.2073, reader_cost: 0.00131, ips: 14.4723 samples/sec | ETA 04:05:28
- 2022-04-13 14:31:14 [INFO] [TRAIN] epoch: 198, iter: 49000/120000, loss: 0.8816, lr: 0.006236, batch_cost: 0.2186, reader_cost: 0.00060, ips: 13.7215 samples/sec | ETA 04:18:43
- 2022-04-13 14:31:24 [INFO] [TRAIN] epoch: 198, iter: 49050/120000, loss: 0.9416, lr: 0.006232, batch_cost: 0.2042, reader_cost: 0.00090, ips: 14.6944 samples/sec | ETA 04:01:25
- 2022-04-13 14:31:35 [INFO] [TRAIN] epoch: 198, iter: 49100/120000, loss: 0.8766, lr: 0.006228, batch_cost: 0.2128, reader_cost: 0.00087, ips: 14.0987 samples/sec | ETA 04:11:26
- 2022-04-13 14:31:49 [INFO] [TRAIN] epoch: 199, iter: 49150/120000, loss: 0.9045, lr: 0.006224, batch_cost: 0.2783, reader_cost: 0.05064, ips: 10.7794 samples/sec | ETA 05:28:38
- 2022-04-13 14:31:59 [INFO] [TRAIN] epoch: 199, iter: 49200/120000, loss: 0.8733, lr: 0.006220, batch_cost: 0.2100, reader_cost: 0.00085, ips: 14.2886 samples/sec | ETA 04:07:45
- 2022-04-13 14:32:09 [INFO] [TRAIN] epoch: 199, iter: 49250/120000, loss: 0.9137, lr: 0.006216, batch_cost: 0.2087, reader_cost: 0.00081, ips: 14.3765 samples/sec | ETA 04:06:03
- 2022-04-13 14:32:21 [INFO] [TRAIN] epoch: 199, iter: 49300/120000, loss: 0.9014, lr: 0.006212, batch_cost: 0.2239, reader_cost: 0.00055, ips: 13.3987 samples/sec | ETA 04:23:49
- 2022-04-13 14:32:30 [INFO] [TRAIN] epoch: 199, iter: 49350/120000, loss: 0.9142, lr: 0.006208, batch_cost: 0.1904, reader_cost: 0.00114, ips: 15.7578 samples/sec | ETA 03:44:10
- 2022-04-13 14:32:44 [INFO] [TRAIN] epoch: 200, iter: 49400/120000, loss: 0.8874, lr: 0.006204, batch_cost: 0.2702, reader_cost: 0.05486, ips: 11.1031 samples/sec | ETA 05:17:55
- 2022-04-13 14:32:54 [INFO] [TRAIN] epoch: 200, iter: 49450/120000, loss: 0.9228, lr: 0.006200, batch_cost: 0.2104, reader_cost: 0.00105, ips: 14.2580 samples/sec | ETA 04:07:24
- 2022-04-13 14:33:05 [INFO] [TRAIN] epoch: 200, iter: 49500/120000, loss: 0.9334, lr: 0.006196, batch_cost: 0.2214, reader_cost: 0.00109, ips: 13.5531 samples/sec | ETA 04:20:05
- 2022-04-13 14:33:16 [INFO] [TRAIN] epoch: 200, iter: 49550/120000, loss: 0.8867, lr: 0.006192, batch_cost: 0.2080, reader_cost: 0.00307, ips: 14.4255 samples/sec | ETA 04:04:11
- 2022-04-13 14:33:25 [INFO] [TRAIN] epoch: 200, iter: 49600/120000, loss: 0.9004, lr: 0.006188, batch_cost: 0.1891, reader_cost: 0.00042, ips: 15.8640 samples/sec | ETA 03:41:53
- 2022-04-13 14:33:39 [INFO] [TRAIN] epoch: 201, iter: 49650/120000, loss: 0.8831, lr: 0.006184, batch_cost: 0.2748, reader_cost: 0.05885, ips: 10.9164 samples/sec | ETA 05:22:13
- 2022-04-13 14:33:50 [INFO] [TRAIN] epoch: 201, iter: 49700/120000, loss: 0.9157, lr: 0.006180, batch_cost: 0.2202, reader_cost: 0.00158, ips: 13.6209 samples/sec | ETA 04:18:03
- 2022-04-13 14:34:00 [INFO] [TRAIN] epoch: 201, iter: 49750/120000, loss: 0.9474, lr: 0.006176, batch_cost: 0.2037, reader_cost: 0.00031, ips: 14.7290 samples/sec | ETA 03:58:28
- 2022-04-13 14:34:11 [INFO] [TRAIN] epoch: 201, iter: 49800/120000, loss: 0.9126, lr: 0.006172, batch_cost: 0.2186, reader_cost: 0.00126, ips: 13.7224 samples/sec | ETA 04:15:47
- 2022-04-13 14:34:24 [INFO] [TRAIN] epoch: 202, iter: 49850/120000, loss: 0.9006, lr: 0.006168, batch_cost: 0.2673, reader_cost: 0.06624, ips: 11.2226 samples/sec | ETA 05:12:32
- 2022-04-13 14:34:35 [INFO] [TRAIN] epoch: 202, iter: 49900/120000, loss: 0.9291, lr: 0.006164, batch_cost: 0.2100, reader_cost: 0.00112, ips: 14.2874 samples/sec | ETA 04:05:19
- 2022-04-13 14:34:46 [INFO] [TRAIN] epoch: 202, iter: 49950/120000, loss: 0.9320, lr: 0.006160, batch_cost: 0.2189, reader_cost: 0.00032, ips: 13.7020 samples/sec | ETA 04:15:37
- 2022-04-13 14:34:56 [INFO] [TRAIN] epoch: 202, iter: 50000/120000, loss: 0.8977, lr: 0.006156, batch_cost: 0.1985, reader_cost: 0.00044, ips: 15.1111 samples/sec | ETA 03:51:37
- 2022-04-13 14:35:06 [INFO] [TRAIN] epoch: 202, iter: 50050/120000, loss: 0.9183, lr: 0.006152, batch_cost: 0.2059, reader_cost: 0.00059, ips: 14.5698 samples/sec | ETA 04:00:03
- 2022-04-13 14:35:19 [INFO] [TRAIN] epoch: 203, iter: 50100/120000, loss: 0.8833, lr: 0.006149, batch_cost: 0.2664, reader_cost: 0.05631, ips: 11.2603 samples/sec | ETA 05:10:22
- 2022-04-13 14:35:31 [INFO] [TRAIN] epoch: 203, iter: 50150/120000, loss: 0.9312, lr: 0.006145, batch_cost: 0.2301, reader_cost: 0.00090, ips: 13.0389 samples/sec | ETA 04:27:51
- 2022-04-13 14:35:42 [INFO] [TRAIN] epoch: 203, iter: 50200/120000, loss: 0.8915, lr: 0.006141, batch_cost: 0.2235, reader_cost: 0.00059, ips: 13.4235 samples/sec | ETA 04:19:59
- 2022-04-13 14:35:52 [INFO] [TRAIN] epoch: 203, iter: 50250/120000, loss: 0.9305, lr: 0.006137, batch_cost: 0.2012, reader_cost: 0.00096, ips: 14.9072 samples/sec | ETA 03:53:56
- 2022-04-13 14:36:03 [INFO] [TRAIN] epoch: 203, iter: 50300/120000, loss: 0.8820, lr: 0.006133, batch_cost: 0.2129, reader_cost: 0.00098, ips: 14.0931 samples/sec | ETA 04:07:17
- 2022-04-13 14:36:16 [INFO] [TRAIN] epoch: 204, iter: 50350/120000, loss: 0.8782, lr: 0.006129, batch_cost: 0.2607, reader_cost: 0.05037, ips: 11.5070 samples/sec | ETA 05:02:38
- 2022-04-13 14:36:26 [INFO] [TRAIN] epoch: 204, iter: 50400/120000, loss: 0.8832, lr: 0.006125, batch_cost: 0.2087, reader_cost: 0.00110, ips: 14.3736 samples/sec | ETA 04:02:06
- 2022-04-13 14:36:36 [INFO] [TRAIN] epoch: 204, iter: 50450/120000, loss: 0.8831, lr: 0.006121, batch_cost: 0.1960, reader_cost: 0.00088, ips: 15.3038 samples/sec | ETA 03:47:13
- 2022-04-13 14:36:47 [INFO] [TRAIN] epoch: 204, iter: 50500/120000, loss: 0.8656, lr: 0.006117, batch_cost: 0.2109, reader_cost: 0.00054, ips: 14.2271 samples/sec | ETA 04:04:15
- 2022-04-13 14:36:57 [INFO] [TRAIN] epoch: 204, iter: 50550/120000, loss: 0.9098, lr: 0.006113, batch_cost: 0.2009, reader_cost: 0.00121, ips: 14.9340 samples/sec | ETA 03:52:31
- 2022-04-13 14:37:10 [INFO] [TRAIN] epoch: 205, iter: 50600/120000, loss: 0.9047, lr: 0.006109, batch_cost: 0.2653, reader_cost: 0.05609, ips: 11.3079 samples/sec | ETA 05:06:51
- 2022-04-13 14:37:21 [INFO] [TRAIN] epoch: 205, iter: 50650/120000, loss: 0.8986, lr: 0.006105, batch_cost: 0.2181, reader_cost: 0.00076, ips: 13.7556 samples/sec | ETA 04:12:04
- 2022-04-13 14:37:31 [INFO] [TRAIN] epoch: 205, iter: 50700/120000, loss: 0.8936, lr: 0.006101, batch_cost: 0.1982, reader_cost: 0.00082, ips: 15.1375 samples/sec | ETA 03:48:54
- 2022-04-13 14:37:42 [INFO] [TRAIN] epoch: 205, iter: 50750/120000, loss: 0.8741, lr: 0.006097, batch_cost: 0.2243, reader_cost: 0.00137, ips: 13.3771 samples/sec | ETA 04:18:50
- 2022-04-13 14:37:53 [INFO] [TRAIN] epoch: 205, iter: 50800/120000, loss: 0.8917, lr: 0.006093, batch_cost: 0.2171, reader_cost: 0.00082, ips: 13.8208 samples/sec | ETA 04:10:20
- 2022-04-13 14:38:07 [INFO] [TRAIN] epoch: 206, iter: 50850/120000, loss: 0.8760, lr: 0.006089, batch_cost: 0.2860, reader_cost: 0.05432, ips: 10.4889 samples/sec | ETA 05:29:37
- 2022-04-13 14:38:19 [INFO] [TRAIN] epoch: 206, iter: 50900/120000, loss: 0.8856, lr: 0.006085, batch_cost: 0.2340, reader_cost: 0.00105, ips: 12.8192 samples/sec | ETA 04:29:31
- 2022-04-13 14:38:29 [INFO] [TRAIN] epoch: 206, iter: 50950/120000, loss: 0.8792, lr: 0.006081, batch_cost: 0.2068, reader_cost: 0.00103, ips: 14.5044 samples/sec | ETA 03:58:01
- 2022-04-13 14:38:40 [INFO] [TRAIN] epoch: 206, iter: 51000/120000, loss: 0.8947, lr: 0.006077, batch_cost: 0.2146, reader_cost: 0.00078, ips: 13.9776 samples/sec | ETA 04:06:49
- 2022-04-13 14:38:51 [INFO] [TRAIN] epoch: 206, iter: 51050/120000, loss: 0.9134, lr: 0.006073, batch_cost: 0.2236, reader_cost: 0.00065, ips: 13.4158 samples/sec | ETA 04:16:58
- 2022-04-13 14:39:05 [INFO] [TRAIN] epoch: 207, iter: 51100/120000, loss: 0.8837, lr: 0.006069, batch_cost: 0.2720, reader_cost: 0.06060, ips: 11.0308 samples/sec | ETA 05:12:18
- 2022-04-13 14:39:15 [INFO] [TRAIN] epoch: 207, iter: 51150/120000, loss: 0.8980, lr: 0.006065, batch_cost: 0.2068, reader_cost: 0.00100, ips: 14.5074 samples/sec | ETA 03:57:17
- 2022-04-13 14:39:25 [INFO] [TRAIN] epoch: 207, iter: 51200/120000, loss: 0.8794, lr: 0.006061, batch_cost: 0.2000, reader_cost: 0.00154, ips: 14.9970 samples/sec | ETA 03:49:22
- 2022-04-13 14:39:36 [INFO] [TRAIN] epoch: 207, iter: 51250/120000, loss: 0.8880, lr: 0.006057, batch_cost: 0.2170, reader_cost: 0.00069, ips: 13.8218 samples/sec | ETA 04:08:42
- 2022-04-13 14:39:46 [INFO] [TRAIN] epoch: 207, iter: 51300/120000, loss: 0.9087, lr: 0.006053, batch_cost: 0.2108, reader_cost: 0.00065, ips: 14.2291 samples/sec | ETA 04:01:24
- 2022-04-13 14:40:00 [INFO] [TRAIN] epoch: 208, iter: 51350/120000, loss: 0.8722, lr: 0.006049, batch_cost: 0.2671, reader_cost: 0.05554, ips: 11.2325 samples/sec | ETA 05:05:35
- 2022-04-13 14:40:11 [INFO] [TRAIN] epoch: 208, iter: 51400/120000, loss: 0.8826, lr: 0.006046, batch_cost: 0.2203, reader_cost: 0.00121, ips: 13.6202 samples/sec | ETA 04:11:49
- 2022-04-13 14:40:21 [INFO] [TRAIN] epoch: 208, iter: 51450/120000, loss: 0.9194, lr: 0.006042, batch_cost: 0.1972, reader_cost: 0.00068, ips: 15.2137 samples/sec | ETA 03:45:17
- 2022-04-13 14:40:31 [INFO] [TRAIN] epoch: 208, iter: 51500/120000, loss: 0.8720, lr: 0.006038, batch_cost: 0.1974, reader_cost: 0.00115, ips: 15.2010 samples/sec | ETA 03:45:18
- 2022-04-13 14:40:41 [INFO] [TRAIN] epoch: 208, iter: 51550/120000, loss: 0.8835, lr: 0.006034, batch_cost: 0.2032, reader_cost: 0.00150, ips: 14.7620 samples/sec | ETA 03:51:50
- 2022-04-13 14:40:54 [INFO] [TRAIN] epoch: 209, iter: 51600/120000, loss: 0.8830, lr: 0.006030, batch_cost: 0.2687, reader_cost: 0.05977, ips: 11.1669 samples/sec | ETA 05:06:15
- 2022-04-13 14:41:04 [INFO] [TRAIN] epoch: 209, iter: 51650/120000, loss: 0.8922, lr: 0.006026, batch_cost: 0.2062, reader_cost: 0.00165, ips: 14.5463 samples/sec | ETA 03:54:56
- 2022-04-13 14:41:15 [INFO] [TRAIN] epoch: 209, iter: 51700/120000, loss: 0.9602, lr: 0.006022, batch_cost: 0.2104, reader_cost: 0.00081, ips: 14.2590 samples/sec | ETA 03:59:29
- 2022-04-13 14:41:26 [INFO] [TRAIN] epoch: 209, iter: 51750/120000, loss: 0.8952, lr: 0.006018, batch_cost: 0.2235, reader_cost: 0.00438, ips: 13.4242 samples/sec | ETA 04:14:12
- 2022-04-13 14:41:37 [INFO] [TRAIN] epoch: 209, iter: 51800/120000, loss: 0.8679, lr: 0.006014, batch_cost: 0.2211, reader_cost: 0.00080, ips: 13.5656 samples/sec | ETA 04:11:22
- 2022-04-13 14:41:51 [INFO] [TRAIN] epoch: 210, iter: 51850/120000, loss: 0.8892, lr: 0.006010, batch_cost: 0.2661, reader_cost: 0.05272, ips: 11.2728 samples/sec | ETA 05:02:16
- 2022-04-13 14:42:01 [INFO] [TRAIN] epoch: 210, iter: 51900/120000, loss: 0.9120, lr: 0.006006, batch_cost: 0.2075, reader_cost: 0.00066, ips: 14.4551 samples/sec | ETA 03:55:33
- 2022-04-13 14:42:11 [INFO] [TRAIN] epoch: 210, iter: 51950/120000, loss: 0.8978, lr: 0.006002, batch_cost: 0.2040, reader_cost: 0.00066, ips: 14.7052 samples/sec | ETA 03:51:22
- 2022-04-13 14:42:22 [INFO] [TRAIN] epoch: 210, iter: 52000/120000, loss: 0.9044, lr: 0.005998, batch_cost: 0.2126, reader_cost: 0.00059, ips: 14.1105 samples/sec | ETA 04:00:57
- 2022-04-13 14:42:22 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1961 - reader cost: 0.1526
- 2022-04-13 14:42:46 [INFO] [EVAL] #Images: 500 mIoU: 0.7297 Acc: 0.9546 Kappa: 0.9410 Dice: 0.8345
- 2022-04-13 14:42:46 [INFO] [EVAL] Class IoU:
- [0.9779 0.8276 0.9157 0.4583 0.5751 0.6217 0.683 0.7692 0.918 0.6097
- 0.9359 0.7994 0.6066 0.9396 0.6845 0.7366 0.5395 0.5326 0.7337]
- 2022-04-13 14:42:46 [INFO] [EVAL] Class Precision:
- [0.9886 0.9118 0.9503 0.7504 0.7485 0.7602 0.7793 0.9053 0.9565 0.8409
- 0.9569 0.8695 0.7117 0.9587 0.8692 0.9583 0.5738 0.6305 0.9119]
- 2022-04-13 14:42:46 [INFO] [EVAL] Class Recall:
- [0.9891 0.8997 0.9617 0.5407 0.7128 0.7734 0.8469 0.8365 0.958 0.6892
- 0.9771 0.9084 0.8043 0.9793 0.7631 0.761 0.9003 0.7742 0.7897]
- 2022-04-13 14:42:47 [INFO] [EVAL] The model with the best validation mIoU (0.7494) was saved at iter 48000.
- 2022-04-13 14:42:59 [INFO] [TRAIN] epoch: 210, iter: 52050/120000, loss: 0.9130, lr: 0.005994, batch_cost: 0.2310, reader_cost: 0.00130, ips: 12.9879 samples/sec | ETA 04:21:35
- 2022-04-13 14:43:12 [INFO] [TRAIN] epoch: 211, iter: 52100/120000, loss: 0.9373, lr: 0.005990, batch_cost: 0.2681, reader_cost: 0.05699, ips: 11.1899 samples/sec | ETA 05:03:23
- 2022-04-13 14:43:23 [INFO] [TRAIN] epoch: 211, iter: 52150/120000, loss: 0.8993, lr: 0.005986, batch_cost: 0.2178, reader_cost: 0.00174, ips: 13.7715 samples/sec | ETA 04:06:20
- 2022-04-13 14:43:34 [INFO] [TRAIN] epoch: 211, iter: 52200/120000, loss: 0.8810, lr: 0.005982, batch_cost: 0.2160, reader_cost: 0.00056, ips: 13.8914 samples/sec | ETA 04:04:02
- 2022-04-13 14:43:45 [INFO] [TRAIN] epoch: 211, iter: 52250/120000, loss: 0.9240, lr: 0.005978, batch_cost: 0.2260, reader_cost: 0.00156, ips: 13.2760 samples/sec | ETA 04:15:09
- 2022-04-13 14:43:56 [INFO] [TRAIN] epoch: 211, iter: 52300/120000, loss: 0.9131, lr: 0.005974, batch_cost: 0.2160, reader_cost: 0.00057, ips: 13.8907 samples/sec | ETA 04:03:41
- 2022-04-13 14:44:09 [INFO] [TRAIN] epoch: 212, iter: 52350/120000, loss: 0.8650, lr: 0.005970, batch_cost: 0.2718, reader_cost: 0.05407, ips: 11.0391 samples/sec | ETA 05:06:24
- 2022-04-13 14:44:20 [INFO] [TRAIN] epoch: 212, iter: 52400/120000, loss: 0.9619, lr: 0.005966, batch_cost: 0.2117, reader_cost: 0.00091, ips: 14.1697 samples/sec | ETA 03:58:32
- 2022-04-13 14:44:31 [INFO] [TRAIN] epoch: 212, iter: 52450/120000, loss: 0.9067, lr: 0.005962, batch_cost: 0.2271, reader_cost: 0.00121, ips: 13.2111 samples/sec | ETA 04:15:39
- 2022-04-13 14:44:42 [INFO] [TRAIN] epoch: 212, iter: 52500/120000, loss: 0.8969, lr: 0.005958, batch_cost: 0.2100, reader_cost: 0.00080, ips: 14.2873 samples/sec | ETA 03:56:13
- 2022-04-13 14:44:52 [INFO] [TRAIN] epoch: 212, iter: 52550/120000, loss: 0.8791, lr: 0.005954, batch_cost: 0.2054, reader_cost: 0.00092, ips: 14.6058 samples/sec | ETA 03:50:54
- 2022-04-13 14:45:06 [INFO] [TRAIN] epoch: 213, iter: 52600/120000, loss: 0.9110, lr: 0.005950, batch_cost: 0.2832, reader_cost: 0.06096, ips: 10.5915 samples/sec | ETA 05:18:10
- 2022-04-13 14:45:17 [INFO] [TRAIN] epoch: 213, iter: 52650/120000, loss: 0.9417, lr: 0.005946, batch_cost: 0.2103, reader_cost: 0.00169, ips: 14.2641 samples/sec | ETA 03:56:04
- 2022-04-13 14:45:27 [INFO] [TRAIN] epoch: 213, iter: 52700/120000, loss: 0.8955, lr: 0.005942, batch_cost: 0.2089, reader_cost: 0.00084, ips: 14.3612 samples/sec | ETA 03:54:18
- 2022-04-13 14:45:38 [INFO] [TRAIN] epoch: 213, iter: 52750/120000, loss: 0.8696, lr: 0.005938, batch_cost: 0.2144, reader_cost: 0.00128, ips: 13.9924 samples/sec | ETA 04:00:18
- 2022-04-13 14:45:49 [INFO] [TRAIN] epoch: 213, iter: 52800/120000, loss: 0.8777, lr: 0.005934, batch_cost: 0.2250, reader_cost: 0.00142, ips: 13.3343 samples/sec | ETA 04:11:58
- 2022-04-13 14:46:03 [INFO] [TRAIN] epoch: 214, iter: 52850/120000, loss: 0.9290, lr: 0.005930, batch_cost: 0.2722, reader_cost: 0.06124, ips: 11.0194 samples/sec | ETA 05:04:41
- 2022-04-13 14:46:13 [INFO] [TRAIN] epoch: 214, iter: 52900/120000, loss: 0.9119, lr: 0.005926, batch_cost: 0.2044, reader_cost: 0.00199, ips: 14.6794 samples/sec | ETA 03:48:33
- 2022-04-13 14:46:24 [INFO] [TRAIN] epoch: 214, iter: 52950/120000, loss: 0.9353, lr: 0.005922, batch_cost: 0.2165, reader_cost: 0.00054, ips: 13.8583 samples/sec | ETA 04:01:54
- 2022-04-13 14:46:35 [INFO] [TRAIN] epoch: 214, iter: 53000/120000, loss: 0.9004, lr: 0.005918, batch_cost: 0.2258, reader_cost: 0.00101, ips: 13.2888 samples/sec | ETA 04:12:05
- 2022-04-13 14:46:47 [INFO] [TRAIN] epoch: 214, iter: 53050/120000, loss: 0.8905, lr: 0.005915, batch_cost: 0.2375, reader_cost: 0.00140, ips: 12.6317 samples/sec | ETA 04:25:00
- 2022-04-13 14:47:00 [INFO] [TRAIN] epoch: 215, iter: 53100/120000, loss: 0.8515, lr: 0.005911, batch_cost: 0.2674, reader_cost: 0.05846, ips: 11.2212 samples/sec | ETA 04:58:05
- 2022-04-13 14:47:11 [INFO] [TRAIN] epoch: 215, iter: 53150/120000, loss: 0.9125, lr: 0.005907, batch_cost: 0.2139, reader_cost: 0.00132, ips: 14.0253 samples/sec | ETA 03:58:19
- 2022-04-13 14:47:21 [INFO] [TRAIN] epoch: 215, iter: 53200/120000, loss: 0.8549, lr: 0.005903, batch_cost: 0.2043, reader_cost: 0.00146, ips: 14.6833 samples/sec | ETA 03:47:28
- 2022-04-13 14:47:32 [INFO] [TRAIN] epoch: 215, iter: 53250/120000, loss: 0.9219, lr: 0.005899, batch_cost: 0.2167, reader_cost: 0.00121, ips: 13.8433 samples/sec | ETA 04:01:05
- 2022-04-13 14:47:43 [INFO] [TRAIN] epoch: 215, iter: 53300/120000, loss: 0.8652, lr: 0.005895, batch_cost: 0.2241, reader_cost: 0.00082, ips: 13.3840 samples/sec | ETA 04:09:10
- 2022-04-13 14:47:57 [INFO] [TRAIN] epoch: 216, iter: 53350/120000, loss: 0.9033, lr: 0.005891, batch_cost: 0.2798, reader_cost: 0.05788, ips: 10.7214 samples/sec | ETA 05:10:49
- 2022-04-13 14:48:07 [INFO] [TRAIN] epoch: 216, iter: 53400/120000, loss: 0.8646, lr: 0.005887, batch_cost: 0.1999, reader_cost: 0.00127, ips: 15.0062 samples/sec | ETA 03:41:54
- 2022-04-13 14:48:18 [INFO] [TRAIN] epoch: 216, iter: 53450/120000, loss: 0.8580, lr: 0.005883, batch_cost: 0.2129, reader_cost: 0.00124, ips: 14.0923 samples/sec | ETA 03:56:07
- 2022-04-13 14:48:30 [INFO] [TRAIN] epoch: 216, iter: 53500/120000, loss: 0.8892, lr: 0.005879, batch_cost: 0.2433, reader_cost: 0.00062, ips: 12.3283 samples/sec | ETA 04:29:42
- 2022-04-13 14:48:41 [INFO] [TRAIN] epoch: 216, iter: 53550/120000, loss: 0.8996, lr: 0.005875, batch_cost: 0.2247, reader_cost: 0.00073, ips: 13.3483 samples/sec | ETA 04:08:54
- 2022-04-13 14:48:55 [INFO] [TRAIN] epoch: 217, iter: 53600/120000, loss: 0.8865, lr: 0.005871, batch_cost: 0.2671, reader_cost: 0.05196, ips: 11.2320 samples/sec | ETA 04:55:35
- 2022-04-13 14:49:05 [INFO] [TRAIN] epoch: 217, iter: 53650/120000, loss: 0.8586, lr: 0.005867, batch_cost: 0.2086, reader_cost: 0.00090, ips: 14.3782 samples/sec | ETA 03:50:43
- 2022-04-13 14:49:16 [INFO] [TRAIN] epoch: 217, iter: 53700/120000, loss: 0.8655, lr: 0.005863, batch_cost: 0.2198, reader_cost: 0.00222, ips: 13.6495 samples/sec | ETA 04:02:51
- 2022-04-13 14:49:26 [INFO] [TRAIN] epoch: 217, iter: 53750/120000, loss: 0.8568, lr: 0.005859, batch_cost: 0.2029, reader_cost: 0.00074, ips: 14.7866 samples/sec | ETA 03:44:01
- 2022-04-13 14:49:36 [INFO] [TRAIN] epoch: 217, iter: 53800/120000, loss: 0.9213, lr: 0.005855, batch_cost: 0.1993, reader_cost: 0.00090, ips: 15.0545 samples/sec | ETA 03:39:52
- 2022-04-13 14:49:50 [INFO] [TRAIN] epoch: 218, iter: 53850/120000, loss: 0.9020, lr: 0.005851, batch_cost: 0.2832, reader_cost: 0.05116, ips: 10.5920 samples/sec | ETA 05:12:15
- 2022-04-13 14:50:01 [INFO] [TRAIN] epoch: 218, iter: 53900/120000, loss: 0.8786, lr: 0.005847, batch_cost: 0.2048, reader_cost: 0.00091, ips: 14.6482 samples/sec | ETA 03:45:37
- 2022-04-13 14:50:11 [INFO] [TRAIN] epoch: 218, iter: 53950/120000, loss: 0.8555, lr: 0.005843, batch_cost: 0.2109, reader_cost: 0.00079, ips: 14.2256 samples/sec | ETA 03:52:09
- 2022-04-13 14:50:21 [INFO] [TRAIN] epoch: 218, iter: 54000/120000, loss: 0.8786, lr: 0.005839, batch_cost: 0.1977, reader_cost: 0.00106, ips: 15.1729 samples/sec | ETA 03:37:29
- 2022-04-13 14:50:32 [INFO] [TRAIN] epoch: 218, iter: 54050/120000, loss: 0.8646, lr: 0.005835, batch_cost: 0.2095, reader_cost: 0.00098, ips: 14.3230 samples/sec | ETA 03:50:13
- 2022-04-13 14:50:45 [INFO] [TRAIN] epoch: 219, iter: 54100/120000, loss: 0.8899, lr: 0.005831, batch_cost: 0.2740, reader_cost: 0.05407, ips: 10.9479 samples/sec | ETA 05:00:58
- 2022-04-13 14:50:55 [INFO] [TRAIN] epoch: 219, iter: 54150/120000, loss: 0.8718, lr: 0.005827, batch_cost: 0.1956, reader_cost: 0.00118, ips: 15.3394 samples/sec | ETA 03:34:38
- 2022-04-13 14:51:06 [INFO] [TRAIN] epoch: 219, iter: 54200/120000, loss: 0.8803, lr: 0.005823, batch_cost: 0.2131, reader_cost: 0.00090, ips: 14.0806 samples/sec | ETA 03:53:39
- 2022-04-13 14:51:18 [INFO] [TRAIN] epoch: 219, iter: 54250/120000, loss: 0.8944, lr: 0.005819, batch_cost: 0.2429, reader_cost: 0.00042, ips: 12.3526 samples/sec | ETA 04:26:08
- 2022-04-13 14:51:29 [INFO] [TRAIN] epoch: 219, iter: 54300/120000, loss: 0.9034, lr: 0.005815, batch_cost: 0.2190, reader_cost: 0.00143, ips: 13.7010 samples/sec | ETA 03:59:45
- 2022-04-13 14:51:42 [INFO] [TRAIN] epoch: 220, iter: 54350/120000, loss: 0.9167, lr: 0.005811, batch_cost: 0.2691, reader_cost: 0.04921, ips: 11.1490 samples/sec | ETA 04:54:25
- 2022-04-13 14:51:52 [INFO] [TRAIN] epoch: 220, iter: 54400/120000, loss: 0.8934, lr: 0.005807, batch_cost: 0.2007, reader_cost: 0.00119, ips: 14.9474 samples/sec | ETA 03:39:26
- 2022-04-13 14:52:02 [INFO] [TRAIN] epoch: 220, iter: 54450/120000, loss: 0.8730, lr: 0.005803, batch_cost: 0.1972, reader_cost: 0.00161, ips: 15.2150 samples/sec | ETA 03:35:24
- 2022-04-13 14:52:13 [INFO] [TRAIN] epoch: 220, iter: 54500/120000, loss: 0.8811, lr: 0.005799, batch_cost: 0.2159, reader_cost: 0.00074, ips: 13.8945 samples/sec | ETA 03:55:42
- 2022-04-13 14:52:24 [INFO] [TRAIN] epoch: 220, iter: 54550/120000, loss: 0.8702, lr: 0.005795, batch_cost: 0.2166, reader_cost: 0.00058, ips: 13.8508 samples/sec | ETA 03:56:16
- 2022-04-13 14:52:38 [INFO] [TRAIN] epoch: 221, iter: 54600/120000, loss: 0.8826, lr: 0.005791, batch_cost: 0.2760, reader_cost: 0.06074, ips: 10.8694 samples/sec | ETA 05:00:50
- 2022-04-13 14:52:49 [INFO] [TRAIN] epoch: 221, iter: 54650/120000, loss: 0.9119, lr: 0.005787, batch_cost: 0.2305, reader_cost: 0.00110, ips: 13.0144 samples/sec | ETA 04:11:04
- 2022-04-13 14:52:59 [INFO] [TRAIN] epoch: 221, iter: 54700/120000, loss: 0.8846, lr: 0.005783, batch_cost: 0.2018, reader_cost: 0.00129, ips: 14.8628 samples/sec | ETA 03:39:40
- 2022-04-13 14:53:10 [INFO] [TRAIN] epoch: 221, iter: 54750/120000, loss: 0.9051, lr: 0.005779, batch_cost: 0.2071, reader_cost: 0.00115, ips: 14.4881 samples/sec | ETA 03:45:11
- 2022-04-13 14:53:21 [INFO] [TRAIN] epoch: 221, iter: 54800/120000, loss: 0.8711, lr: 0.005775, batch_cost: 0.2227, reader_cost: 0.00069, ips: 13.4716 samples/sec | ETA 04:01:59
- 2022-04-13 14:53:34 [INFO] [TRAIN] epoch: 222, iter: 54850/120000, loss: 0.8426, lr: 0.005771, batch_cost: 0.2651, reader_cost: 0.05748, ips: 11.3177 samples/sec | ETA 04:47:49
- 2022-04-13 14:53:44 [INFO] [TRAIN] epoch: 222, iter: 54900/120000, loss: 0.8598, lr: 0.005767, batch_cost: 0.1993, reader_cost: 0.00058, ips: 15.0541 samples/sec | ETA 03:36:13
- 2022-04-13 14:53:55 [INFO] [TRAIN] epoch: 222, iter: 54950/120000, loss: 0.8963, lr: 0.005763, batch_cost: 0.2279, reader_cost: 0.00133, ips: 13.1661 samples/sec | ETA 04:07:02
- 2022-04-13 14:54:06 [INFO] [TRAIN] epoch: 222, iter: 55000/120000, loss: 0.9249, lr: 0.005759, batch_cost: 0.2095, reader_cost: 0.00082, ips: 14.3183 samples/sec | ETA 03:46:58
- 2022-04-13 14:54:16 [INFO] [TRAIN] epoch: 222, iter: 55050/120000, loss: 0.8864, lr: 0.005755, batch_cost: 0.1972, reader_cost: 0.00098, ips: 15.2157 samples/sec | ETA 03:33:25
- 2022-04-13 14:54:29 [INFO] [TRAIN] epoch: 223, iter: 55100/120000, loss: 0.8953, lr: 0.005751, batch_cost: 0.2768, reader_cost: 0.05288, ips: 10.8372 samples/sec | ETA 04:59:25
- 2022-04-13 14:54:40 [INFO] [TRAIN] epoch: 223, iter: 55150/120000, loss: 0.8721, lr: 0.005747, batch_cost: 0.2138, reader_cost: 0.00109, ips: 14.0328 samples/sec | ETA 03:51:03
- 2022-04-13 14:54:50 [INFO] [TRAIN] epoch: 223, iter: 55200/120000, loss: 0.9055, lr: 0.005743, batch_cost: 0.2033, reader_cost: 0.00281, ips: 14.7590 samples/sec | ETA 03:39:31
- 2022-04-13 14:55:02 [INFO] [TRAIN] epoch: 223, iter: 55250/120000, loss: 0.8892, lr: 0.005739, batch_cost: 0.2339, reader_cost: 0.00066, ips: 12.8284 samples/sec | ETA 04:12:22
- 2022-04-13 14:55:13 [INFO] [TRAIN] epoch: 223, iter: 55300/120000, loss: 0.8813, lr: 0.005735, batch_cost: 0.2098, reader_cost: 0.00050, ips: 14.2991 samples/sec | ETA 03:46:14
- 2022-04-13 14:55:27 [INFO] [TRAIN] epoch: 224, iter: 55350/120000, loss: 0.9060, lr: 0.005731, batch_cost: 0.2902, reader_cost: 0.05710, ips: 10.3393 samples/sec | ETA 05:12:38
- 2022-04-13 14:55:37 [INFO] [TRAIN] epoch: 224, iter: 55400/120000, loss: 0.8732, lr: 0.005727, batch_cost: 0.1993, reader_cost: 0.00138, ips: 15.0559 samples/sec | ETA 03:34:32
- 2022-04-13 14:55:48 [INFO] [TRAIN] epoch: 224, iter: 55450/120000, loss: 0.8931, lr: 0.005723, batch_cost: 0.2165, reader_cost: 0.00074, ips: 13.8566 samples/sec | ETA 03:52:55
- 2022-04-13 14:55:59 [INFO] [TRAIN] epoch: 224, iter: 55500/120000, loss: 0.9163, lr: 0.005719, batch_cost: 0.2258, reader_cost: 0.00083, ips: 13.2851 samples/sec | ETA 04:02:45
- 2022-04-13 14:56:10 [INFO] [TRAIN] epoch: 224, iter: 55550/120000, loss: 0.8848, lr: 0.005715, batch_cost: 0.2096, reader_cost: 0.00124, ips: 14.3133 samples/sec | ETA 03:45:08
- 2022-04-13 14:56:24 [INFO] [TRAIN] epoch: 225, iter: 55600/120000, loss: 0.9084, lr: 0.005711, batch_cost: 0.2839, reader_cost: 0.05395, ips: 10.5652 samples/sec | ETA 05:04:46
- 2022-04-13 14:56:34 [INFO] [TRAIN] epoch: 225, iter: 55650/120000, loss: 0.9062, lr: 0.005707, batch_cost: 0.2108, reader_cost: 0.00090, ips: 14.2338 samples/sec | ETA 03:46:02
- 2022-04-13 14:56:45 [INFO] [TRAIN] epoch: 225, iter: 55700/120000, loss: 0.8584, lr: 0.005703, batch_cost: 0.2166, reader_cost: 0.00128, ips: 13.8502 samples/sec | ETA 03:52:07
- 2022-04-13 14:56:56 [INFO] [TRAIN] epoch: 225, iter: 55750/120000, loss: 0.8578, lr: 0.005699, batch_cost: 0.2184, reader_cost: 0.00073, ips: 13.7364 samples/sec | ETA 03:53:52
- 2022-04-13 14:57:07 [INFO] [TRAIN] epoch: 225, iter: 55800/120000, loss: 0.9229, lr: 0.005695, batch_cost: 0.2087, reader_cost: 0.00049, ips: 14.3756 samples/sec | ETA 03:43:17
- 2022-04-13 14:57:20 [INFO] [TRAIN] epoch: 226, iter: 55850/120000, loss: 0.8948, lr: 0.005691, batch_cost: 0.2737, reader_cost: 0.06214, ips: 10.9597 samples/sec | ETA 04:52:39
- 2022-04-13 14:57:32 [INFO] [TRAIN] epoch: 226, iter: 55900/120000, loss: 0.9147, lr: 0.005687, batch_cost: 0.2289, reader_cost: 0.00048, ips: 13.1041 samples/sec | ETA 04:04:34
- 2022-04-13 14:57:42 [INFO] [TRAIN] epoch: 226, iter: 55950/120000, loss: 0.9059, lr: 0.005683, batch_cost: 0.2073, reader_cost: 0.00057, ips: 14.4721 samples/sec | ETA 03:41:17
- 2022-04-13 14:57:53 [INFO] [TRAIN] epoch: 226, iter: 56000/120000, loss: 0.8814, lr: 0.005679, batch_cost: 0.2245, reader_cost: 0.00091, ips: 13.3619 samples/sec | ETA 03:59:29
- 2022-04-13 14:57:53 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1966 - reader cost: 0.1520
- 2022-04-13 14:58:18 [INFO] [EVAL] #Images: 500 mIoU: 0.7123 Acc: 0.9530 Kappa: 0.9389 Dice: 0.8094
- 2022-04-13 14:58:18 [INFO] [EVAL] Class IoU:
- [0.9761 0.8161 0.9147 0.4105 0.5799 0.6125 0.6794 0.7672 0.9127 0.6351
- 0.9455 0.7043 0.0693 0.9428 0.7914 0.7649 0.6912 0.5851 0.7349]
- 2022-04-13 14:58:18 [INFO] [EVAL] Class Precision:
- [0.9847 0.9245 0.9481 0.851 0.8518 0.8336 0.8606 0.8928 0.9409 0.8109
- 0.9636 0.7461 0.7293 0.972 0.8521 0.9705 0.7789 0.7753 0.8454]
- 2022-04-13 14:58:18 [INFO] [EVAL] Class Recall:
- [0.9912 0.8744 0.963 0.4423 0.645 0.6979 0.7634 0.8451 0.9682 0.7454
- 0.9806 0.9263 0.0712 0.9691 0.9175 0.7831 0.8599 0.7045 0.849 ]
- 2022-04-13 14:58:19 [INFO] [EVAL] The model with the best validation mIoU (0.7494) was saved at iter 48000.
- 2022-04-13 14:58:32 [INFO] [TRAIN] epoch: 227, iter: 56050/120000, loss: 0.9070, lr: 0.005675, batch_cost: 0.2594, reader_cost: 0.05054, ips: 11.5669 samples/sec | ETA 04:36:26
- 2022-04-13 14:58:42 [INFO] [TRAIN] epoch: 227, iter: 56100/120000, loss: 0.9208, lr: 0.005671, batch_cost: 0.2086, reader_cost: 0.00119, ips: 14.3793 samples/sec | ETA 03:42:11
- 2022-04-13 14:58:53 [INFO] [TRAIN] epoch: 227, iter: 56150/120000, loss: 0.8665, lr: 0.005667, batch_cost: 0.2224, reader_cost: 0.00098, ips: 13.4912 samples/sec | ETA 03:56:38
- 2022-04-13 14:59:04 [INFO] [TRAIN] epoch: 227, iter: 56200/120000, loss: 0.8704, lr: 0.005663, batch_cost: 0.2125, reader_cost: 0.00116, ips: 14.1150 samples/sec | ETA 03:46:00
- 2022-04-13 14:59:14 [INFO] [TRAIN] epoch: 227, iter: 56250/120000, loss: 0.9261, lr: 0.005659, batch_cost: 0.2000, reader_cost: 0.00125, ips: 14.9981 samples/sec | ETA 03:32:31
- 2022-04-13 14:59:27 [INFO] [TRAIN] epoch: 228, iter: 56300/120000, loss: 0.8931, lr: 0.005655, batch_cost: 0.2642, reader_cost: 0.06131, ips: 11.3531 samples/sec | ETA 04:40:32
- 2022-04-13 14:59:37 [INFO] [TRAIN] epoch: 228, iter: 56350/120000, loss: 0.9542, lr: 0.005651, batch_cost: 0.2026, reader_cost: 0.00144, ips: 14.8085 samples/sec | ETA 03:34:54
- 2022-04-13 14:59:47 [INFO] [TRAIN] epoch: 228, iter: 56400/120000, loss: 0.9140, lr: 0.005647, batch_cost: 0.1973, reader_cost: 0.00113, ips: 15.2052 samples/sec | ETA 03:29:08
- 2022-04-13 14:59:58 [INFO] [TRAIN] epoch: 228, iter: 56450/120000, loss: 0.8723, lr: 0.005643, batch_cost: 0.2122, reader_cost: 0.00166, ips: 14.1401 samples/sec | ETA 03:44:42
- 2022-04-13 15:00:08 [INFO] [TRAIN] epoch: 228, iter: 56500/120000, loss: 0.8842, lr: 0.005639, batch_cost: 0.2158, reader_cost: 0.00084, ips: 13.9032 samples/sec | ETA 03:48:21
- 2022-04-13 15:00:21 [INFO] [TRAIN] epoch: 229, iter: 56550/120000, loss: 0.8467, lr: 0.005635, batch_cost: 0.2599, reader_cost: 0.06093, ips: 11.5415 samples/sec | ETA 04:34:52
- 2022-04-13 15:00:32 [INFO] [TRAIN] epoch: 229, iter: 56600/120000, loss: 0.9269, lr: 0.005631, batch_cost: 0.2121, reader_cost: 0.00065, ips: 14.1432 samples/sec | ETA 03:44:08
- 2022-04-13 15:00:42 [INFO] [TRAIN] epoch: 229, iter: 56650/120000, loss: 0.9303, lr: 0.005627, batch_cost: 0.2078, reader_cost: 0.00073, ips: 14.4364 samples/sec | ETA 03:39:24
- 2022-04-13 15:00:53 [INFO] [TRAIN] epoch: 229, iter: 56700/120000, loss: 0.8771, lr: 0.005623, batch_cost: 0.2220, reader_cost: 0.00081, ips: 13.5145 samples/sec | ETA 03:54:11
- 2022-04-13 15:01:04 [INFO] [TRAIN] epoch: 229, iter: 56750/120000, loss: 0.8952, lr: 0.005619, batch_cost: 0.2059, reader_cost: 0.00107, ips: 14.5667 samples/sec | ETA 03:37:06
- 2022-04-13 15:01:18 [INFO] [TRAIN] epoch: 230, iter: 56800/120000, loss: 0.8963, lr: 0.005616, batch_cost: 0.2791, reader_cost: 0.05251, ips: 10.7487 samples/sec | ETA 04:53:59
- 2022-04-13 15:01:28 [INFO] [TRAIN] epoch: 230, iter: 56850/120000, loss: 0.8841, lr: 0.005612, batch_cost: 0.2020, reader_cost: 0.00095, ips: 14.8493 samples/sec | ETA 03:32:38
- 2022-04-13 15:01:38 [INFO] [TRAIN] epoch: 230, iter: 56900/120000, loss: 0.9008, lr: 0.005608, batch_cost: 0.2030, reader_cost: 0.00179, ips: 14.7807 samples/sec | ETA 03:33:27
- 2022-04-13 15:01:48 [INFO] [TRAIN] epoch: 230, iter: 56950/120000, loss: 0.9320, lr: 0.005604, batch_cost: 0.2049, reader_cost: 0.00067, ips: 14.6419 samples/sec | ETA 03:35:18
- 2022-04-13 15:01:59 [INFO] [TRAIN] epoch: 230, iter: 57000/120000, loss: 0.8985, lr: 0.005600, batch_cost: 0.2089, reader_cost: 0.00065, ips: 14.3643 samples/sec | ETA 03:39:17
- 2022-04-13 15:02:12 [INFO] [TRAIN] epoch: 231, iter: 57050/120000, loss: 0.9188, lr: 0.005596, batch_cost: 0.2666, reader_cost: 0.06127, ips: 11.2513 samples/sec | ETA 04:39:44
- 2022-04-13 15:02:22 [INFO] [TRAIN] epoch: 231, iter: 57100/120000, loss: 0.9093, lr: 0.005592, batch_cost: 0.2031, reader_cost: 0.00078, ips: 14.7744 samples/sec | ETA 03:32:52
- 2022-04-13 15:02:32 [INFO] [TRAIN] epoch: 231, iter: 57150/120000, loss: 0.8820, lr: 0.005588, batch_cost: 0.2020, reader_cost: 0.00046, ips: 14.8528 samples/sec | ETA 03:31:34
- 2022-04-13 15:02:43 [INFO] [TRAIN] epoch: 231, iter: 57200/120000, loss: 0.9140, lr: 0.005584, batch_cost: 0.2058, reader_cost: 0.00099, ips: 14.5738 samples/sec | ETA 03:35:27
- 2022-04-13 15:02:53 [INFO] [TRAIN] epoch: 231, iter: 57250/120000, loss: 0.9032, lr: 0.005580, batch_cost: 0.2034, reader_cost: 0.00074, ips: 14.7499 samples/sec | ETA 03:32:42
- 2022-04-13 15:03:06 [INFO] [TRAIN] epoch: 232, iter: 57300/120000, loss: 0.8975, lr: 0.005576, batch_cost: 0.2662, reader_cost: 0.05616, ips: 11.2690 samples/sec | ETA 04:38:11
- 2022-04-13 15:03:16 [INFO] [TRAIN] epoch: 232, iter: 57350/120000, loss: 0.8924, lr: 0.005571, batch_cost: 0.2014, reader_cost: 0.00080, ips: 14.8968 samples/sec | ETA 03:30:16
- 2022-04-13 15:03:27 [INFO] [TRAIN] epoch: 232, iter: 57400/120000, loss: 0.8547, lr: 0.005567, batch_cost: 0.2083, reader_cost: 0.00153, ips: 14.4003 samples/sec | ETA 03:37:21
- 2022-04-13 15:03:38 [INFO] [TRAIN] epoch: 232, iter: 57450/120000, loss: 0.8856, lr: 0.005563, batch_cost: 0.2242, reader_cost: 0.00046, ips: 13.3801 samples/sec | ETA 03:53:44
- 2022-04-13 15:03:49 [INFO] [TRAIN] epoch: 232, iter: 57500/120000, loss: 0.8838, lr: 0.005559, batch_cost: 0.2164, reader_cost: 0.00063, ips: 13.8655 samples/sec | ETA 03:45:22
- 2022-04-13 15:04:02 [INFO] [TRAIN] epoch: 233, iter: 57550/120000, loss: 0.9059, lr: 0.005555, batch_cost: 0.2653, reader_cost: 0.05640, ips: 11.3072 samples/sec | ETA 04:36:09
- 2022-04-13 15:04:12 [INFO] [TRAIN] epoch: 233, iter: 57600/120000, loss: 0.8599, lr: 0.005551, batch_cost: 0.1948, reader_cost: 0.00084, ips: 15.3994 samples/sec | ETA 03:22:36
- 2022-04-13 15:04:22 [INFO] [TRAIN] epoch: 233, iter: 57650/120000, loss: 0.8902, lr: 0.005547, batch_cost: 0.2186, reader_cost: 0.00055, ips: 13.7210 samples/sec | ETA 03:47:12
- 2022-04-13 15:04:33 [INFO] [TRAIN] epoch: 233, iter: 57700/120000, loss: 0.8733, lr: 0.005543, batch_cost: 0.2046, reader_cost: 0.00156, ips: 14.6645 samples/sec | ETA 03:32:25
- 2022-04-13 15:04:43 [INFO] [TRAIN] epoch: 233, iter: 57750/120000, loss: 0.9169, lr: 0.005539, batch_cost: 0.2039, reader_cost: 0.00106, ips: 14.7128 samples/sec | ETA 03:31:33
- 2022-04-13 15:04:56 [INFO] [TRAIN] epoch: 234, iter: 57800/120000, loss: 0.8969, lr: 0.005535, batch_cost: 0.2687, reader_cost: 0.05455, ips: 11.1663 samples/sec | ETA 04:38:30
- 2022-04-13 15:05:07 [INFO] [TRAIN] epoch: 234, iter: 57850/120000, loss: 0.9081, lr: 0.005531, batch_cost: 0.2180, reader_cost: 0.00072, ips: 13.7584 samples/sec | ETA 03:45:51
- 2022-04-13 15:05:18 [INFO] [TRAIN] epoch: 234, iter: 57900/120000, loss: 0.8971, lr: 0.005527, batch_cost: 0.2097, reader_cost: 0.00484, ips: 14.3033 samples/sec | ETA 03:37:05
- 2022-04-13 15:05:29 [INFO] [TRAIN] epoch: 234, iter: 57950/120000, loss: 0.8723, lr: 0.005523, batch_cost: 0.2262, reader_cost: 0.00120, ips: 13.2624 samples/sec | ETA 03:53:55
- 2022-04-13 15:05:40 [INFO] [TRAIN] epoch: 234, iter: 58000/120000, loss: 0.8936, lr: 0.005519, batch_cost: 0.2260, reader_cost: 0.00077, ips: 13.2751 samples/sec | ETA 03:53:31
- 2022-04-13 15:05:54 [INFO] [TRAIN] epoch: 235, iter: 58050/120000, loss: 0.8872, lr: 0.005515, batch_cost: 0.2639, reader_cost: 0.05091, ips: 11.3664 samples/sec | ETA 04:32:30
- 2022-04-13 15:06:04 [INFO] [TRAIN] epoch: 235, iter: 58100/120000, loss: 0.8801, lr: 0.005511, batch_cost: 0.2046, reader_cost: 0.00119, ips: 14.6599 samples/sec | ETA 03:31:07
- 2022-04-13 15:06:14 [INFO] [TRAIN] epoch: 235, iter: 58150/120000, loss: 0.8855, lr: 0.005507, batch_cost: 0.2125, reader_cost: 0.00064, ips: 14.1171 samples/sec | ETA 03:39:03
- 2022-04-13 15:06:25 [INFO] [TRAIN] epoch: 235, iter: 58200/120000, loss: 0.9453, lr: 0.005503, batch_cost: 0.2122, reader_cost: 0.00042, ips: 14.1402 samples/sec | ETA 03:38:31
- 2022-04-13 15:06:36 [INFO] [TRAIN] epoch: 235, iter: 58250/120000, loss: 0.8927, lr: 0.005499, batch_cost: 0.2106, reader_cost: 0.00115, ips: 14.2431 samples/sec | ETA 03:36:46
- 2022-04-13 15:06:49 [INFO] [TRAIN] epoch: 236, iter: 58300/120000, loss: 0.8900, lr: 0.005495, batch_cost: 0.2713, reader_cost: 0.05414, ips: 11.0572 samples/sec | ETA 04:39:00
- 2022-04-13 15:06:59 [INFO] [TRAIN] epoch: 236, iter: 58350/120000, loss: 0.9156, lr: 0.005491, batch_cost: 0.2073, reader_cost: 0.00101, ips: 14.4717 samples/sec | ETA 03:33:00
- 2022-04-13 15:07:10 [INFO] [TRAIN] epoch: 236, iter: 58400/120000, loss: 0.8937, lr: 0.005487, batch_cost: 0.2103, reader_cost: 0.00116, ips: 14.2648 samples/sec | ETA 03:35:54
- 2022-04-13 15:07:20 [INFO] [TRAIN] epoch: 236, iter: 58450/120000, loss: 0.9025, lr: 0.005483, batch_cost: 0.1980, reader_cost: 0.00151, ips: 15.1534 samples/sec | ETA 03:23:05
- 2022-04-13 15:07:31 [INFO] [TRAIN] epoch: 236, iter: 58500/120000, loss: 0.8548, lr: 0.005479, batch_cost: 0.2200, reader_cost: 0.00114, ips: 13.6388 samples/sec | ETA 03:45:27
- 2022-04-13 15:07:44 [INFO] [TRAIN] epoch: 237, iter: 58550/120000, loss: 0.8848, lr: 0.005475, batch_cost: 0.2655, reader_cost: 0.05440, ips: 11.2994 samples/sec | ETA 04:31:55
- 2022-04-13 15:07:54 [INFO] [TRAIN] epoch: 237, iter: 58600/120000, loss: 0.8884, lr: 0.005471, batch_cost: 0.1980, reader_cost: 0.00164, ips: 15.1519 samples/sec | ETA 03:22:36
- 2022-04-13 15:08:05 [INFO] [TRAIN] epoch: 237, iter: 58650/120000, loss: 0.8989, lr: 0.005467, batch_cost: 0.2104, reader_cost: 0.00092, ips: 14.2615 samples/sec | ETA 03:35:05
- 2022-04-13 15:08:15 [INFO] [TRAIN] epoch: 237, iter: 58700/120000, loss: 0.9137, lr: 0.005463, batch_cost: 0.2106, reader_cost: 0.00112, ips: 14.2435 samples/sec | ETA 03:35:11
- 2022-04-13 15:08:25 [INFO] [TRAIN] epoch: 237, iter: 58750/120000, loss: 0.8755, lr: 0.005459, batch_cost: 0.2070, reader_cost: 0.00171, ips: 14.4938 samples/sec | ETA 03:31:17
- 2022-04-13 15:08:39 [INFO] [TRAIN] epoch: 238, iter: 58800/120000, loss: 0.8686, lr: 0.005455, batch_cost: 0.2649, reader_cost: 0.04938, ips: 11.3245 samples/sec | ETA 04:30:12
- 2022-04-13 15:08:49 [INFO] [TRAIN] epoch: 238, iter: 58850/120000, loss: 0.8694, lr: 0.005451, batch_cost: 0.2029, reader_cost: 0.00104, ips: 14.7862 samples/sec | ETA 03:26:46
- 2022-04-13 15:09:00 [INFO] [TRAIN] epoch: 238, iter: 58900/120000, loss: 0.9318, lr: 0.005447, batch_cost: 0.2163, reader_cost: 0.00043, ips: 13.8697 samples/sec | ETA 03:40:15
- 2022-04-13 15:09:10 [INFO] [TRAIN] epoch: 238, iter: 58950/120000, loss: 0.8897, lr: 0.005443, batch_cost: 0.2114, reader_cost: 0.00057, ips: 14.1914 samples/sec | ETA 03:35:05
- 2022-04-13 15:09:21 [INFO] [TRAIN] epoch: 238, iter: 59000/120000, loss: 0.9158, lr: 0.005439, batch_cost: 0.2116, reader_cost: 0.00069, ips: 14.1808 samples/sec | ETA 03:35:04
- 2022-04-13 15:09:34 [INFO] [TRAIN] epoch: 239, iter: 59050/120000, loss: 0.8895, lr: 0.005435, batch_cost: 0.2720, reader_cost: 0.04951, ips: 11.0295 samples/sec | ETA 04:36:18
- 2022-04-13 15:09:45 [INFO] [TRAIN] epoch: 239, iter: 59100/120000, loss: 0.9009, lr: 0.005431, batch_cost: 0.2132, reader_cost: 0.00114, ips: 14.0714 samples/sec | ETA 03:36:23
- 2022-04-13 15:09:55 [INFO] [TRAIN] epoch: 239, iter: 59150/120000, loss: 0.8781, lr: 0.005427, batch_cost: 0.1994, reader_cost: 0.00088, ips: 15.0482 samples/sec | ETA 03:22:10
- 2022-04-13 15:10:06 [INFO] [TRAIN] epoch: 239, iter: 59200/120000, loss: 0.8786, lr: 0.005423, batch_cost: 0.2253, reader_cost: 0.00109, ips: 13.3144 samples/sec | ETA 03:48:19
- 2022-04-13 15:10:17 [INFO] [TRAIN] epoch: 239, iter: 59250/120000, loss: 0.8696, lr: 0.005419, batch_cost: 0.2104, reader_cost: 0.00143, ips: 14.2587 samples/sec | ETA 03:33:01
- 2022-04-13 15:10:31 [INFO] [TRAIN] epoch: 240, iter: 59300/120000, loss: 0.8790, lr: 0.005415, batch_cost: 0.2755, reader_cost: 0.05638, ips: 10.8890 samples/sec | ETA 04:38:43
- 2022-04-13 15:10:41 [INFO] [TRAIN] epoch: 240, iter: 59350/120000, loss: 0.8665, lr: 0.005411, batch_cost: 0.2155, reader_cost: 0.00053, ips: 13.9212 samples/sec | ETA 03:37:50
- 2022-04-13 15:10:51 [INFO] [TRAIN] epoch: 240, iter: 59400/120000, loss: 0.8880, lr: 0.005407, batch_cost: 0.1964, reader_cost: 0.00128, ips: 15.2742 samples/sec | ETA 03:18:22
- 2022-04-13 15:11:02 [INFO] [TRAIN] epoch: 240, iter: 59450/120000, loss: 0.8861, lr: 0.005403, batch_cost: 0.2127, reader_cost: 0.00090, ips: 14.1065 samples/sec | ETA 03:34:37
- 2022-04-13 15:11:12 [INFO] [TRAIN] epoch: 240, iter: 59500/120000, loss: 0.8656, lr: 0.005399, batch_cost: 0.2100, reader_cost: 0.00085, ips: 14.2838 samples/sec | ETA 03:31:46
- 2022-04-13 15:11:26 [INFO] [TRAIN] epoch: 241, iter: 59550/120000, loss: 0.9134, lr: 0.005395, batch_cost: 0.2750, reader_cost: 0.06620, ips: 10.9077 samples/sec | ETA 04:37:05
- 2022-04-13 15:11:37 [INFO] [TRAIN] epoch: 241, iter: 59600/120000, loss: 0.8823, lr: 0.005391, batch_cost: 0.2109, reader_cost: 0.00112, ips: 14.2237 samples/sec | ETA 03:32:19
- 2022-04-13 15:11:47 [INFO] [TRAIN] epoch: 241, iter: 59650/120000, loss: 0.8917, lr: 0.005387, batch_cost: 0.2063, reader_cost: 0.00086, ips: 14.5398 samples/sec | ETA 03:27:32
- 2022-04-13 15:11:57 [INFO] [TRAIN] epoch: 241, iter: 59700/120000, loss: 0.8818, lr: 0.005383, batch_cost: 0.2088, reader_cost: 0.00057, ips: 14.3648 samples/sec | ETA 03:29:53
- 2022-04-13 15:12:08 [INFO] [TRAIN] epoch: 241, iter: 59750/120000, loss: 0.8846, lr: 0.005379, batch_cost: 0.2187, reader_cost: 0.00119, ips: 13.7182 samples/sec | ETA 03:39:35
- 2022-04-13 15:12:22 [INFO] [TRAIN] epoch: 242, iter: 59800/120000, loss: 0.8687, lr: 0.005375, batch_cost: 0.2686, reader_cost: 0.05778, ips: 11.1683 samples/sec | ETA 04:29:30
- 2022-04-13 15:12:32 [INFO] [TRAIN] epoch: 242, iter: 59850/120000, loss: 0.8723, lr: 0.005371, batch_cost: 0.2037, reader_cost: 0.00148, ips: 14.7265 samples/sec | ETA 03:24:13
- 2022-04-13 15:12:44 [INFO] [TRAIN] epoch: 242, iter: 59900/120000, loss: 0.9174, lr: 0.005367, batch_cost: 0.2356, reader_cost: 0.00044, ips: 12.7343 samples/sec | ETA 03:55:58
- 2022-04-13 15:12:54 [INFO] [TRAIN] epoch: 242, iter: 59950/120000, loss: 0.8858, lr: 0.005363, batch_cost: 0.1964, reader_cost: 0.00070, ips: 15.2780 samples/sec | ETA 03:16:31
- 2022-04-13 15:13:04 [INFO] [TRAIN] epoch: 242, iter: 60000/120000, loss: 0.8613, lr: 0.005359, batch_cost: 0.2169, reader_cost: 0.00105, ips: 13.8343 samples/sec | ETA 03:36:51
- 2022-04-13 15:13:04 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1957 - reader cost: 0.1504
- 2022-04-13 15:13:29 [INFO] [EVAL] #Images: 500 mIoU: 0.7036 Acc: 0.9453 Kappa: 0.9290 Dice: 0.8138
- 2022-04-13 15:13:29 [INFO] [EVAL] Class IoU:
- [0.9726 0.795 0.8935 0.3802 0.5594 0.5564 0.6127 0.7578 0.8965 0.5504
- 0.9263 0.7569 0.5532 0.932 0.7457 0.7552 0.5955 0.4292 0.7007]
- 2022-04-13 15:13:29 [INFO] [EVAL] Class Precision:
- [0.9892 0.8572 0.9309 0.6826 0.7502 0.7913 0.8524 0.8987 0.9488 0.7648
- 0.9413 0.8289 0.7742 0.9633 0.8235 0.8881 0.8498 0.5272 0.8945]
- 2022-04-13 15:13:29 [INFO] [EVAL] Class Recall:
- [0.983 0.9163 0.957 0.4618 0.6875 0.6521 0.6855 0.8286 0.9421 0.6626
- 0.9831 0.897 0.6596 0.9663 0.8876 0.8346 0.6655 0.6978 0.7638]
- 2022-04-13 15:13:30 [INFO] [EVAL] The model with the best validation mIoU (0.7494) was saved at iter 48000.
- 2022-04-13 15:13:43 [INFO] [TRAIN] epoch: 243, iter: 60050/120000, loss: 0.8582, lr: 0.005355, batch_cost: 0.2727, reader_cost: 0.05015, ips: 10.9992 samples/sec | ETA 04:32:31
- 2022-04-13 15:13:53 [INFO] [TRAIN] epoch: 243, iter: 60100/120000, loss: 0.8733, lr: 0.005351, batch_cost: 0.1955, reader_cost: 0.00099, ips: 15.3465 samples/sec | ETA 03:15:09
- 2022-04-13 15:14:03 [INFO] [TRAIN] epoch: 243, iter: 60150/120000, loss: 0.8552, lr: 0.005347, batch_cost: 0.1987, reader_cost: 0.00105, ips: 15.0996 samples/sec | ETA 03:18:11
- 2022-04-13 15:14:13 [INFO] [TRAIN] epoch: 243, iter: 60200/120000, loss: 0.8901, lr: 0.005343, batch_cost: 0.2091, reader_cost: 0.00098, ips: 14.3500 samples/sec | ETA 03:28:21
- 2022-04-13 15:14:23 [INFO] [TRAIN] epoch: 243, iter: 60250/120000, loss: 0.9224, lr: 0.005339, batch_cost: 0.2017, reader_cost: 0.00133, ips: 14.8748 samples/sec | ETA 03:20:50
- 2022-04-13 15:14:37 [INFO] [TRAIN] epoch: 244, iter: 60300/120000, loss: 0.8848, lr: 0.005335, batch_cost: 0.2614, reader_cost: 0.05111, ips: 11.4785 samples/sec | ETA 04:20:03
- 2022-04-13 15:14:47 [INFO] [TRAIN] epoch: 244, iter: 60350/120000, loss: 0.8907, lr: 0.005331, batch_cost: 0.2065, reader_cost: 0.00101, ips: 14.5307 samples/sec | ETA 03:25:15
- 2022-04-13 15:14:58 [INFO] [TRAIN] epoch: 244, iter: 60400/120000, loss: 0.8700, lr: 0.005327, batch_cost: 0.2199, reader_cost: 0.00087, ips: 13.6409 samples/sec | ETA 03:38:27
- 2022-04-13 15:15:09 [INFO] [TRAIN] epoch: 244, iter: 60450/120000, loss: 0.8786, lr: 0.005323, batch_cost: 0.2153, reader_cost: 0.00075, ips: 13.9331 samples/sec | ETA 03:33:42
- 2022-04-13 15:15:19 [INFO] [TRAIN] epoch: 244, iter: 60500/120000, loss: 0.8825, lr: 0.005319, batch_cost: 0.2121, reader_cost: 0.00085, ips: 14.1414 samples/sec | ETA 03:30:22
- 2022-04-13 15:15:33 [INFO] [TRAIN] epoch: 245, iter: 60550/120000, loss: 0.8751, lr: 0.005315, batch_cost: 0.2759, reader_cost: 0.06308, ips: 10.8748 samples/sec | ETA 04:33:20
- 2022-04-13 15:15:43 [INFO] [TRAIN] epoch: 245, iter: 60600/120000, loss: 0.9161, lr: 0.005311, batch_cost: 0.2012, reader_cost: 0.00060, ips: 14.9075 samples/sec | ETA 03:19:13
- 2022-04-13 15:15:53 [INFO] [TRAIN] epoch: 245, iter: 60650/120000, loss: 0.9221, lr: 0.005307, batch_cost: 0.2064, reader_cost: 0.00061, ips: 14.5324 samples/sec | ETA 03:24:11
- 2022-04-13 15:16:05 [INFO] [TRAIN] epoch: 245, iter: 60700/120000, loss: 0.9262, lr: 0.005303, batch_cost: 0.2212, reader_cost: 0.00042, ips: 13.5597 samples/sec | ETA 03:38:39
- 2022-04-13 15:16:15 [INFO] [TRAIN] epoch: 245, iter: 60750/120000, loss: 0.9150, lr: 0.005299, batch_cost: 0.2037, reader_cost: 0.00066, ips: 14.7293 samples/sec | ETA 03:21:07
- 2022-04-13 15:16:29 [INFO] [TRAIN] epoch: 246, iter: 60800/120000, loss: 0.8876, lr: 0.005295, batch_cost: 0.2769, reader_cost: 0.06089, ips: 10.8346 samples/sec | ETA 04:33:11
- 2022-04-13 15:16:39 [INFO] [TRAIN] epoch: 246, iter: 60850/120000, loss: 0.8790, lr: 0.005291, batch_cost: 0.2102, reader_cost: 0.00228, ips: 14.2718 samples/sec | ETA 03:27:13
- 2022-04-13 15:16:50 [INFO] [TRAIN] epoch: 246, iter: 60900/120000, loss: 0.8735, lr: 0.005287, batch_cost: 0.2098, reader_cost: 0.00127, ips: 14.3005 samples/sec | ETA 03:26:38
- 2022-04-13 15:17:00 [INFO] [TRAIN] epoch: 246, iter: 60950/120000, loss: 0.8896, lr: 0.005283, batch_cost: 0.2148, reader_cost: 0.00092, ips: 13.9661 samples/sec | ETA 03:31:24
- 2022-04-13 15:17:11 [INFO] [TRAIN] epoch: 246, iter: 61000/120000, loss: 0.8590, lr: 0.005278, batch_cost: 0.2092, reader_cost: 0.00102, ips: 14.3394 samples/sec | ETA 03:25:43
- 2022-04-13 15:17:24 [INFO] [TRAIN] epoch: 247, iter: 61050/120000, loss: 0.8703, lr: 0.005274, batch_cost: 0.2669, reader_cost: 0.05319, ips: 11.2405 samples/sec | ETA 04:22:13
- 2022-04-13 15:17:34 [INFO] [TRAIN] epoch: 247, iter: 61100/120000, loss: 0.8771, lr: 0.005270, batch_cost: 0.2051, reader_cost: 0.00149, ips: 14.6240 samples/sec | ETA 03:21:22
- 2022-04-13 15:17:44 [INFO] [TRAIN] epoch: 247, iter: 61150/120000, loss: 0.8797, lr: 0.005266, batch_cost: 0.2020, reader_cost: 0.00080, ips: 14.8543 samples/sec | ETA 03:18:05
- 2022-04-13 15:17:55 [INFO] [TRAIN] epoch: 247, iter: 61200/120000, loss: 0.8871, lr: 0.005262, batch_cost: 0.2099, reader_cost: 0.00105, ips: 14.2912 samples/sec | ETA 03:25:43
- 2022-04-13 15:18:05 [INFO] [TRAIN] epoch: 247, iter: 61250/120000, loss: 0.8612, lr: 0.005258, batch_cost: 0.1977, reader_cost: 0.00078, ips: 15.1742 samples/sec | ETA 03:13:35
- 2022-04-13 15:18:19 [INFO] [TRAIN] epoch: 248, iter: 61300/120000, loss: 0.8530, lr: 0.005254, batch_cost: 0.2774, reader_cost: 0.05715, ips: 10.8133 samples/sec | ETA 04:31:25
- 2022-04-13 15:18:29 [INFO] [TRAIN] epoch: 248, iter: 61350/120000, loss: 0.8773, lr: 0.005250, batch_cost: 0.2086, reader_cost: 0.00074, ips: 14.3850 samples/sec | ETA 03:23:51
- 2022-04-13 15:18:40 [INFO] [TRAIN] epoch: 248, iter: 61400/120000, loss: 0.9048, lr: 0.005246, batch_cost: 0.2102, reader_cost: 0.00120, ips: 14.2746 samples/sec | ETA 03:25:15
- 2022-04-13 15:18:51 [INFO] [TRAIN] epoch: 248, iter: 61450/120000, loss: 0.8815, lr: 0.005242, batch_cost: 0.2286, reader_cost: 0.00085, ips: 13.1211 samples/sec | ETA 03:43:06
- 2022-04-13 15:19:02 [INFO] [TRAIN] epoch: 248, iter: 61500/120000, loss: 0.8556, lr: 0.005238, batch_cost: 0.2116, reader_cost: 0.00107, ips: 14.1754 samples/sec | ETA 03:26:20
- 2022-04-13 15:19:15 [INFO] [TRAIN] epoch: 249, iter: 61550/120000, loss: 0.8786, lr: 0.005234, batch_cost: 0.2688, reader_cost: 0.06059, ips: 11.1605 samples/sec | ETA 04:21:51
- 2022-04-13 15:19:25 [INFO] [TRAIN] epoch: 249, iter: 61600/120000, loss: 0.8886, lr: 0.005230, batch_cost: 0.2041, reader_cost: 0.00140, ips: 14.6959 samples/sec | ETA 03:18:41
- 2022-04-13 15:19:36 [INFO] [TRAIN] epoch: 249, iter: 61650/120000, loss: 0.9244, lr: 0.005226, batch_cost: 0.2070, reader_cost: 0.00147, ips: 14.4913 samples/sec | ETA 03:21:19
- 2022-04-13 15:19:46 [INFO] [TRAIN] epoch: 249, iter: 61700/120000, loss: 0.8889, lr: 0.005222, batch_cost: 0.2096, reader_cost: 0.00059, ips: 14.3146 samples/sec | ETA 03:23:38
- 2022-04-13 15:19:57 [INFO] [TRAIN] epoch: 249, iter: 61750/120000, loss: 0.9093, lr: 0.005218, batch_cost: 0.2139, reader_cost: 0.00058, ips: 14.0258 samples/sec | ETA 03:27:39
- 2022-04-13 15:20:10 [INFO] [TRAIN] epoch: 250, iter: 61800/120000, loss: 0.8944, lr: 0.005214, batch_cost: 0.2692, reader_cost: 0.05203, ips: 11.1453 samples/sec | ETA 04:21:05
- 2022-04-13 15:20:21 [INFO] [TRAIN] epoch: 250, iter: 61850/120000, loss: 0.8996, lr: 0.005210, batch_cost: 0.2070, reader_cost: 0.00100, ips: 14.4939 samples/sec | ETA 03:20:36
- 2022-04-13 15:20:31 [INFO] [TRAIN] epoch: 250, iter: 61900/120000, loss: 0.8779, lr: 0.005206, batch_cost: 0.2045, reader_cost: 0.00106, ips: 14.6701 samples/sec | ETA 03:18:01
- 2022-04-13 15:20:41 [INFO] [TRAIN] epoch: 250, iter: 61950/120000, loss: 0.8712, lr: 0.005202, batch_cost: 0.2039, reader_cost: 0.00045, ips: 14.7107 samples/sec | ETA 03:17:18
- 2022-04-13 15:20:51 [INFO] [TRAIN] epoch: 250, iter: 62000/120000, loss: 0.8899, lr: 0.005198, batch_cost: 0.1907, reader_cost: 0.00040, ips: 15.7292 samples/sec | ETA 03:04:22
- 2022-04-13 15:21:06 [INFO] [TRAIN] epoch: 251, iter: 62050/120000, loss: 0.8908, lr: 0.005194, batch_cost: 0.3109, reader_cost: 0.05539, ips: 9.6509 samples/sec | ETA 05:00:13
- 2022-04-13 15:21:16 [INFO] [TRAIN] epoch: 251, iter: 62100/120000, loss: 0.8808, lr: 0.005190, batch_cost: 0.2017, reader_cost: 0.00164, ips: 14.8755 samples/sec | ETA 03:14:36
- 2022-04-13 15:21:27 [INFO] [TRAIN] epoch: 251, iter: 62150/120000, loss: 0.9128, lr: 0.005186, batch_cost: 0.2080, reader_cost: 0.00117, ips: 14.4254 samples/sec | ETA 03:20:30
- 2022-04-13 15:21:37 [INFO] [TRAIN] epoch: 251, iter: 62200/120000, loss: 0.9133, lr: 0.005182, batch_cost: 0.2142, reader_cost: 0.00119, ips: 14.0069 samples/sec | ETA 03:26:19
- 2022-04-13 15:21:50 [INFO] [TRAIN] epoch: 252, iter: 62250/120000, loss: 0.8579, lr: 0.005178, batch_cost: 0.2498, reader_cost: 0.05783, ips: 12.0098 samples/sec | ETA 04:00:25
- 2022-04-13 15:22:01 [INFO] [TRAIN] epoch: 252, iter: 62300/120000, loss: 0.8989, lr: 0.005174, batch_cost: 0.2219, reader_cost: 0.00113, ips: 13.5208 samples/sec | ETA 03:33:22
- 2022-04-13 15:22:14 [INFO] [TRAIN] epoch: 252, iter: 62350/120000, loss: 0.8873, lr: 0.005170, batch_cost: 0.2563, reader_cost: 0.00042, ips: 11.7072 samples/sec | ETA 04:06:12
- 2022-04-13 15:22:24 [INFO] [TRAIN] epoch: 252, iter: 62400/120000, loss: 0.9035, lr: 0.005166, batch_cost: 0.2037, reader_cost: 0.00127, ips: 14.7295 samples/sec | ETA 03:15:31
- 2022-04-13 15:22:35 [INFO] [TRAIN] epoch: 252, iter: 62450/120000, loss: 0.9017, lr: 0.005162, batch_cost: 0.2209, reader_cost: 0.00080, ips: 13.5823 samples/sec | ETA 03:31:51
- 2022-04-13 15:22:48 [INFO] [TRAIN] epoch: 253, iter: 62500/120000, loss: 0.8706, lr: 0.005158, batch_cost: 0.2583, reader_cost: 0.04983, ips: 11.6165 samples/sec | ETA 04:07:29
- 2022-04-13 15:22:59 [INFO] [TRAIN] epoch: 253, iter: 62550/120000, loss: 0.8801, lr: 0.005154, batch_cost: 0.2154, reader_cost: 0.00084, ips: 13.9271 samples/sec | ETA 03:26:15
- 2022-04-13 15:23:09 [INFO] [TRAIN] epoch: 253, iter: 62600/120000, loss: 0.9124, lr: 0.005149, batch_cost: 0.2111, reader_cost: 0.00151, ips: 14.2106 samples/sec | ETA 03:21:57
- 2022-04-13 15:23:19 [INFO] [TRAIN] epoch: 253, iter: 62650/120000, loss: 0.8970, lr: 0.005145, batch_cost: 0.2022, reader_cost: 0.00145, ips: 14.8394 samples/sec | ETA 03:13:14
- 2022-04-13 15:23:29 [INFO] [TRAIN] epoch: 253, iter: 62700/120000, loss: 0.9157, lr: 0.005141, batch_cost: 0.2021, reader_cost: 0.00075, ips: 14.8463 samples/sec | ETA 03:12:58
- 2022-04-13 15:23:43 [INFO] [TRAIN] epoch: 254, iter: 62750/120000, loss: 0.8782, lr: 0.005137, batch_cost: 0.2630, reader_cost: 0.05339, ips: 11.4069 samples/sec | ETA 04:10:56
- 2022-04-13 15:23:53 [INFO] [TRAIN] epoch: 254, iter: 62800/120000, loss: 0.8980, lr: 0.005133, batch_cost: 0.2176, reader_cost: 0.00187, ips: 13.7882 samples/sec | ETA 03:27:25
- 2022-04-13 15:24:05 [INFO] [TRAIN] epoch: 254, iter: 62850/120000, loss: 0.8886, lr: 0.005129, batch_cost: 0.2341, reader_cost: 0.00139, ips: 12.8154 samples/sec | ETA 03:42:58
- 2022-04-13 15:24:15 [INFO] [TRAIN] epoch: 254, iter: 62900/120000, loss: 0.8817, lr: 0.005125, batch_cost: 0.2058, reader_cost: 0.00088, ips: 14.5762 samples/sec | ETA 03:15:52
- 2022-04-13 15:24:26 [INFO] [TRAIN] epoch: 254, iter: 62950/120000, loss: 0.9094, lr: 0.005121, batch_cost: 0.2004, reader_cost: 0.00054, ips: 14.9736 samples/sec | ETA 03:10:30
- 2022-04-13 15:24:39 [INFO] [TRAIN] epoch: 255, iter: 63000/120000, loss: 0.8948, lr: 0.005117, batch_cost: 0.2629, reader_cost: 0.06178, ips: 11.4110 samples/sec | ETA 04:09:45
- 2022-04-13 15:24:49 [INFO] [TRAIN] epoch: 255, iter: 63050/120000, loss: 0.8658, lr: 0.005113, batch_cost: 0.2141, reader_cost: 0.00120, ips: 14.0120 samples/sec | ETA 03:23:13
- 2022-04-13 15:25:01 [INFO] [TRAIN] epoch: 255, iter: 63100/120000, loss: 0.8664, lr: 0.005109, batch_cost: 0.2244, reader_cost: 0.00204, ips: 13.3678 samples/sec | ETA 03:32:49
- 2022-04-13 15:25:11 [INFO] [TRAIN] epoch: 255, iter: 63150/120000, loss: 0.8960, lr: 0.005105, batch_cost: 0.2013, reader_cost: 0.00082, ips: 14.9033 samples/sec | ETA 03:10:43
- 2022-04-13 15:25:22 [INFO] [TRAIN] epoch: 255, iter: 63200/120000, loss: 0.8805, lr: 0.005101, batch_cost: 0.2222, reader_cost: 0.00134, ips: 13.5016 samples/sec | ETA 03:30:20
- 2022-04-13 15:25:35 [INFO] [TRAIN] epoch: 256, iter: 63250/120000, loss: 0.9112, lr: 0.005097, batch_cost: 0.2735, reader_cost: 0.04842, ips: 10.9691 samples/sec | ETA 04:18:40
- 2022-04-13 15:25:48 [INFO] [TRAIN] epoch: 256, iter: 63300/120000, loss: 0.9139, lr: 0.005093, batch_cost: 0.2422, reader_cost: 0.00101, ips: 12.3844 samples/sec | ETA 03:48:54
- 2022-04-13 15:25:57 [INFO] [TRAIN] epoch: 256, iter: 63350/120000, loss: 0.9040, lr: 0.005089, batch_cost: 0.1957, reader_cost: 0.00161, ips: 15.3286 samples/sec | ETA 03:04:47
- 2022-04-13 15:26:08 [INFO] [TRAIN] epoch: 256, iter: 63400/120000, loss: 0.9042, lr: 0.005085, batch_cost: 0.2055, reader_cost: 0.00094, ips: 14.5971 samples/sec | ETA 03:13:52
- 2022-04-13 15:26:18 [INFO] [TRAIN] epoch: 256, iter: 63450/120000, loss: 0.8814, lr: 0.005081, batch_cost: 0.2053, reader_cost: 0.00337, ips: 14.6148 samples/sec | ETA 03:13:28
- 2022-04-13 15:26:32 [INFO] [TRAIN] epoch: 257, iter: 63500/120000, loss: 0.8712, lr: 0.005077, batch_cost: 0.2760, reader_cost: 0.05784, ips: 10.8677 samples/sec | ETA 04:19:56
- 2022-04-13 15:26:43 [INFO] [TRAIN] epoch: 257, iter: 63550/120000, loss: 0.8634, lr: 0.005073, batch_cost: 0.2167, reader_cost: 0.00125, ips: 13.8472 samples/sec | ETA 03:23:49
- 2022-04-13 15:26:53 [INFO] [TRAIN] epoch: 257, iter: 63600/120000, loss: 0.8716, lr: 0.005069, batch_cost: 0.2066, reader_cost: 0.00097, ips: 14.5181 samples/sec | ETA 03:14:14
- 2022-04-13 15:27:03 [INFO] [TRAIN] epoch: 257, iter: 63650/120000, loss: 0.9015, lr: 0.005065, batch_cost: 0.2016, reader_cost: 0.00106, ips: 14.8788 samples/sec | ETA 03:09:21
- 2022-04-13 15:27:13 [INFO] [TRAIN] epoch: 257, iter: 63700/120000, loss: 0.8726, lr: 0.005061, batch_cost: 0.2060, reader_cost: 0.00122, ips: 14.5655 samples/sec | ETA 03:13:15
- 2022-04-13 15:27:27 [INFO] [TRAIN] epoch: 258, iter: 63750/120000, loss: 0.8645, lr: 0.005057, batch_cost: 0.2776, reader_cost: 0.06565, ips: 10.8054 samples/sec | ETA 04:20:17
- 2022-04-13 15:27:37 [INFO] [TRAIN] epoch: 258, iter: 63800/120000, loss: 0.8791, lr: 0.005053, batch_cost: 0.1976, reader_cost: 0.00065, ips: 15.1796 samples/sec | ETA 03:05:07
- 2022-04-13 15:27:47 [INFO] [TRAIN] epoch: 258, iter: 63850/120000, loss: 0.8576, lr: 0.005048, batch_cost: 0.2049, reader_cost: 0.00105, ips: 14.6443 samples/sec | ETA 03:11:42
- 2022-04-13 15:27:58 [INFO] [TRAIN] epoch: 258, iter: 63900/120000, loss: 0.8538, lr: 0.005044, batch_cost: 0.2198, reader_cost: 0.00126, ips: 13.6493 samples/sec | ETA 03:25:30
- 2022-04-13 15:28:08 [INFO] [TRAIN] epoch: 258, iter: 63950/120000, loss: 0.8925, lr: 0.005040, batch_cost: 0.2014, reader_cost: 0.00128, ips: 14.8978 samples/sec | ETA 03:08:06
- 2022-04-13 15:28:22 [INFO] [TRAIN] epoch: 259, iter: 64000/120000, loss: 0.8802, lr: 0.005036, batch_cost: 0.2674, reader_cost: 0.05344, ips: 11.2191 samples/sec | ETA 04:09:34
- 2022-04-13 15:28:22 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1931 - reader cost: 0.1502
- 2022-04-13 15:28:46 [INFO] [EVAL] #Images: 500 mIoU: 0.7509 Acc: 0.9554 Kappa: 0.9420 Dice: 0.8480
- 2022-04-13 15:28:46 [INFO] [EVAL] Class IoU:
- [0.9793 0.8377 0.9155 0.4493 0.4921 0.622 0.699 0.7677 0.9156 0.588
- 0.9452 0.7776 0.5619 0.9462 0.8177 0.857 0.757 0.5813 0.7564]
- 2022-04-13 15:28:46 [INFO] [EVAL] Class Precision:
- [0.9875 0.9241 0.9563 0.6759 0.8679 0.7877 0.8426 0.8542 0.9487 0.7332
- 0.9664 0.8304 0.7173 0.9701 0.8908 0.9175 0.8981 0.7971 0.8549]
- 2022-04-13 15:28:46 [INFO] [EVAL] Class Recall:
- [0.9915 0.8996 0.9555 0.5726 0.5319 0.7473 0.8039 0.8834 0.9634 0.7481
- 0.9774 0.9244 0.7217 0.9746 0.9089 0.9285 0.8282 0.6822 0.8678]
- 2022-04-13 15:28:47 [INFO] [EVAL] The model with the best validation mIoU (0.7509) was saved at iter 64000.
- 2022-04-13 15:28:58 [INFO] [TRAIN] epoch: 259, iter: 64050/120000, loss: 0.8680, lr: 0.005032, batch_cost: 0.2114, reader_cost: 0.00082, ips: 14.1928 samples/sec | ETA 03:17:06
- 2022-04-13 15:29:09 [INFO] [TRAIN] epoch: 259, iter: 64100/120000, loss: 0.8865, lr: 0.005028, batch_cost: 0.2272, reader_cost: 0.00111, ips: 13.2051 samples/sec | ETA 03:31:39
- 2022-04-13 15:29:19 [INFO] [TRAIN] epoch: 259, iter: 64150/120000, loss: 0.8500, lr: 0.005024, batch_cost: 0.2037, reader_cost: 0.00108, ips: 14.7268 samples/sec | ETA 03:09:37
- 2022-04-13 15:29:30 [INFO] [TRAIN] epoch: 259, iter: 64200/120000, loss: 0.9172, lr: 0.005020, batch_cost: 0.2103, reader_cost: 0.00092, ips: 14.2660 samples/sec | ETA 03:15:34
- 2022-04-13 15:29:44 [INFO] [TRAIN] epoch: 260, iter: 64250/120000, loss: 0.8980, lr: 0.005016, batch_cost: 0.2810, reader_cost: 0.05437, ips: 10.6763 samples/sec | ETA 04:21:05
- 2022-04-13 15:29:54 [INFO] [TRAIN] epoch: 260, iter: 64300/120000, loss: 0.8808, lr: 0.005012, batch_cost: 0.2095, reader_cost: 0.00077, ips: 14.3190 samples/sec | ETA 03:14:29
- 2022-04-13 15:30:06 [INFO] [TRAIN] epoch: 260, iter: 64350/120000, loss: 0.8819, lr: 0.005008, batch_cost: 0.2330, reader_cost: 0.00139, ips: 12.8730 samples/sec | ETA 03:36:09
- 2022-04-13 15:30:16 [INFO] [TRAIN] epoch: 260, iter: 64400/120000, loss: 0.8838, lr: 0.005004, batch_cost: 0.1997, reader_cost: 0.00132, ips: 15.0218 samples/sec | ETA 03:05:03
- 2022-04-13 15:30:27 [INFO] [TRAIN] epoch: 260, iter: 64450/120000, loss: 0.8723, lr: 0.005000, batch_cost: 0.2082, reader_cost: 0.00078, ips: 14.4091 samples/sec | ETA 03:12:45
- 2022-04-13 15:30:40 [INFO] [TRAIN] epoch: 261, iter: 64500/120000, loss: 0.8425, lr: 0.004996, batch_cost: 0.2671, reader_cost: 0.05924, ips: 11.2306 samples/sec | ETA 04:07:05
- 2022-04-13 15:30:52 [INFO] [TRAIN] epoch: 261, iter: 64550/120000, loss: 0.8477, lr: 0.004992, batch_cost: 0.2392, reader_cost: 0.00100, ips: 12.5431 samples/sec | ETA 03:41:02
- 2022-04-13 15:31:02 [INFO] [TRAIN] epoch: 261, iter: 64600/120000, loss: 0.8825, lr: 0.004988, batch_cost: 0.2030, reader_cost: 0.00071, ips: 14.7816 samples/sec | ETA 03:07:23
- 2022-04-13 15:31:13 [INFO] [TRAIN] epoch: 261, iter: 64650/120000, loss: 0.8549, lr: 0.004984, batch_cost: 0.2116, reader_cost: 0.00082, ips: 14.1803 samples/sec | ETA 03:15:09
- 2022-04-13 15:31:23 [INFO] [TRAIN] epoch: 261, iter: 64700/120000, loss: 0.9162, lr: 0.004980, batch_cost: 0.2133, reader_cost: 0.00101, ips: 14.0623 samples/sec | ETA 03:16:37
- 2022-04-13 15:31:36 [INFO] [TRAIN] epoch: 262, iter: 64750/120000, loss: 0.8798, lr: 0.004976, batch_cost: 0.2641, reader_cost: 0.05558, ips: 11.3582 samples/sec | ETA 04:03:12
- 2022-04-13 15:31:46 [INFO] [TRAIN] epoch: 262, iter: 64800/120000, loss: 0.8809, lr: 0.004972, batch_cost: 0.1959, reader_cost: 0.00106, ips: 15.3169 samples/sec | ETA 03:00:11
- 2022-04-13 15:31:58 [INFO] [TRAIN] epoch: 262, iter: 64850/120000, loss: 0.8778, lr: 0.004967, batch_cost: 0.2296, reader_cost: 0.00188, ips: 13.0654 samples/sec | ETA 03:31:03
- 2022-04-13 15:32:09 [INFO] [TRAIN] epoch: 262, iter: 64900/120000, loss: 0.8984, lr: 0.004963, batch_cost: 0.2283, reader_cost: 0.00118, ips: 13.1425 samples/sec | ETA 03:29:37
- 2022-04-13 15:32:20 [INFO] [TRAIN] epoch: 262, iter: 64950/120000, loss: 0.8876, lr: 0.004959, batch_cost: 0.2090, reader_cost: 0.00107, ips: 14.3539 samples/sec | ETA 03:11:45
- 2022-04-13 15:32:33 [INFO] [TRAIN] epoch: 263, iter: 65000/120000, loss: 0.8894, lr: 0.004955, batch_cost: 0.2700, reader_cost: 0.05309, ips: 11.1122 samples/sec | ETA 04:07:28
- 2022-04-13 15:32:44 [INFO] [TRAIN] epoch: 263, iter: 65050/120000, loss: 0.8916, lr: 0.004951, batch_cost: 0.2215, reader_cost: 0.00095, ips: 13.5453 samples/sec | ETA 03:22:50
- 2022-04-13 15:32:55 [INFO] [TRAIN] epoch: 263, iter: 65100/120000, loss: 0.8515, lr: 0.004947, batch_cost: 0.2140, reader_cost: 0.00074, ips: 14.0180 samples/sec | ETA 03:15:49
- 2022-04-13 15:33:06 [INFO] [TRAIN] epoch: 263, iter: 65150/120000, loss: 0.8813, lr: 0.004943, batch_cost: 0.2151, reader_cost: 0.00146, ips: 13.9497 samples/sec | ETA 03:16:35
- 2022-04-13 15:33:16 [INFO] [TRAIN] epoch: 263, iter: 65200/120000, loss: 0.8787, lr: 0.004939, batch_cost: 0.2042, reader_cost: 0.00062, ips: 14.6919 samples/sec | ETA 03:06:29
- 2022-04-13 15:33:31 [INFO] [TRAIN] epoch: 264, iter: 65250/120000, loss: 0.8667, lr: 0.004935, batch_cost: 0.3011, reader_cost: 0.05994, ips: 9.9620 samples/sec | ETA 04:34:47
- 2022-04-13 15:33:41 [INFO] [TRAIN] epoch: 264, iter: 65300/120000, loss: 0.8833, lr: 0.004931, batch_cost: 0.2115, reader_cost: 0.00082, ips: 14.1817 samples/sec | ETA 03:12:51
- 2022-04-13 15:33:52 [INFO] [TRAIN] epoch: 264, iter: 65350/120000, loss: 0.8964, lr: 0.004927, batch_cost: 0.2048, reader_cost: 0.00136, ips: 14.6472 samples/sec | ETA 03:06:33
- 2022-04-13 15:34:02 [INFO] [TRAIN] epoch: 264, iter: 65400/120000, loss: 0.8878, lr: 0.004923, batch_cost: 0.2143, reader_cost: 0.00147, ips: 13.9985 samples/sec | ETA 03:15:01
- 2022-04-13 15:34:13 [INFO] [TRAIN] epoch: 264, iter: 65450/120000, loss: 0.8770, lr: 0.004919, batch_cost: 0.2181, reader_cost: 0.00117, ips: 13.7566 samples/sec | ETA 03:18:16
- 2022-04-13 15:34:27 [INFO] [TRAIN] epoch: 265, iter: 65500/120000, loss: 0.8623, lr: 0.004915, batch_cost: 0.2631, reader_cost: 0.06011, ips: 11.4038 samples/sec | ETA 03:58:57
- 2022-04-13 15:34:38 [INFO] [TRAIN] epoch: 265, iter: 65550/120000, loss: 0.8595, lr: 0.004911, batch_cost: 0.2234, reader_cost: 0.00079, ips: 13.4282 samples/sec | ETA 03:22:44
- 2022-04-13 15:34:50 [INFO] [TRAIN] epoch: 265, iter: 65600/120000, loss: 0.8813, lr: 0.004907, batch_cost: 0.2375, reader_cost: 0.00082, ips: 12.6338 samples/sec | ETA 03:35:17
- 2022-04-13 15:35:00 [INFO] [TRAIN] epoch: 265, iter: 65650/120000, loss: 0.8650, lr: 0.004903, batch_cost: 0.2078, reader_cost: 0.00109, ips: 14.4371 samples/sec | ETA 03:08:13
- 2022-04-13 15:35:11 [INFO] [TRAIN] epoch: 265, iter: 65700/120000, loss: 0.8715, lr: 0.004899, batch_cost: 0.2263, reader_cost: 0.00086, ips: 13.2596 samples/sec | ETA 03:24:45
- 2022-04-13 15:35:25 [INFO] [TRAIN] epoch: 266, iter: 65750/120000, loss: 0.8745, lr: 0.004894, batch_cost: 0.2684, reader_cost: 0.05968, ips: 11.1773 samples/sec | ETA 04:02:40
- 2022-04-13 15:35:34 [INFO] [TRAIN] epoch: 266, iter: 65800/120000, loss: 0.8675, lr: 0.004890, batch_cost: 0.1934, reader_cost: 0.00108, ips: 15.5150 samples/sec | ETA 02:54:40
- 2022-04-13 15:35:47 [INFO] [TRAIN] epoch: 266, iter: 65850/120000, loss: 0.8811, lr: 0.004886, batch_cost: 0.2446, reader_cost: 0.00079, ips: 12.2625 samples/sec | ETA 03:40:47
- 2022-04-13 15:35:58 [INFO] [TRAIN] epoch: 266, iter: 65900/120000, loss: 0.8863, lr: 0.004882, batch_cost: 0.2263, reader_cost: 0.00132, ips: 13.2568 samples/sec | ETA 03:24:02
- 2022-04-13 15:36:09 [INFO] [TRAIN] epoch: 266, iter: 65950/120000, loss: 0.8900, lr: 0.004878, batch_cost: 0.2140, reader_cost: 0.00078, ips: 14.0203 samples/sec | ETA 03:12:45
- 2022-04-13 15:36:22 [INFO] [TRAIN] epoch: 267, iter: 66000/120000, loss: 0.8648, lr: 0.004874, batch_cost: 0.2775, reader_cost: 0.06139, ips: 10.8115 samples/sec | ETA 04:09:44
- 2022-04-13 15:36:34 [INFO] [TRAIN] epoch: 267, iter: 66050/120000, loss: 0.8656, lr: 0.004870, batch_cost: 0.2322, reader_cost: 0.00098, ips: 12.9225 samples/sec | ETA 03:28:44
- 2022-04-13 15:36:47 [INFO] [TRAIN] epoch: 267, iter: 66100/120000, loss: 0.8782, lr: 0.004866, batch_cost: 0.2646, reader_cost: 0.00091, ips: 11.3372 samples/sec | ETA 03:57:42
- 2022-04-13 15:36:58 [INFO] [TRAIN] epoch: 267, iter: 66150/120000, loss: 0.8388, lr: 0.004862, batch_cost: 0.2082, reader_cost: 0.00147, ips: 14.4083 samples/sec | ETA 03:06:52
- 2022-04-13 15:37:09 [INFO] [TRAIN] epoch: 267, iter: 66200/120000, loss: 0.8917, lr: 0.004858, batch_cost: 0.2342, reader_cost: 0.00082, ips: 12.8093 samples/sec | ETA 03:30:00
- 2022-04-13 15:37:23 [INFO] [TRAIN] epoch: 268, iter: 66250/120000, loss: 0.8497, lr: 0.004854, batch_cost: 0.2648, reader_cost: 0.04795, ips: 11.3290 samples/sec | ETA 03:57:13
- 2022-04-13 15:37:33 [INFO] [TRAIN] epoch: 268, iter: 66300/120000, loss: 0.9063, lr: 0.004850, batch_cost: 0.1983, reader_cost: 0.00113, ips: 15.1323 samples/sec | ETA 02:57:26
- 2022-04-13 15:37:43 [INFO] [TRAIN] epoch: 268, iter: 66350/120000, loss: 0.8424, lr: 0.004846, batch_cost: 0.2010, reader_cost: 0.00111, ips: 14.9246 samples/sec | ETA 02:59:44
- 2022-04-13 15:37:53 [INFO] [TRAIN] epoch: 268, iter: 66400/120000, loss: 0.8429, lr: 0.004842, batch_cost: 0.1998, reader_cost: 0.00109, ips: 15.0153 samples/sec | ETA 02:58:29
- 2022-04-13 15:38:03 [INFO] [TRAIN] epoch: 268, iter: 66450/120000, loss: 0.8594, lr: 0.004838, batch_cost: 0.2015, reader_cost: 0.00065, ips: 14.8919 samples/sec | ETA 02:59:47
- 2022-04-13 15:38:17 [INFO] [TRAIN] epoch: 269, iter: 66500/120000, loss: 0.8799, lr: 0.004834, batch_cost: 0.2788, reader_cost: 0.05785, ips: 10.7616 samples/sec | ETA 04:08:34
- 2022-04-13 15:38:27 [INFO] [TRAIN] epoch: 269, iter: 66550/120000, loss: 0.8603, lr: 0.004829, batch_cost: 0.2024, reader_cost: 0.00093, ips: 14.8246 samples/sec | ETA 03:00:16
- 2022-04-13 15:38:37 [INFO] [TRAIN] epoch: 269, iter: 66600/120000, loss: 0.8576, lr: 0.004825, batch_cost: 0.2093, reader_cost: 0.00172, ips: 14.3303 samples/sec | ETA 03:06:19
- 2022-04-13 15:38:47 [INFO] [TRAIN] epoch: 269, iter: 66650/120000, loss: 0.8388, lr: 0.004821, batch_cost: 0.2046, reader_cost: 0.00106, ips: 14.6635 samples/sec | ETA 03:01:54
- 2022-04-13 15:38:58 [INFO] [TRAIN] epoch: 269, iter: 66700/120000, loss: 0.8765, lr: 0.004817, batch_cost: 0.2110, reader_cost: 0.00088, ips: 14.2171 samples/sec | ETA 03:07:27
- 2022-04-13 15:39:12 [INFO] [TRAIN] epoch: 270, iter: 66750/120000, loss: 0.8911, lr: 0.004813, batch_cost: 0.2822, reader_cost: 0.05903, ips: 10.6299 samples/sec | ETA 04:10:28
- 2022-04-13 15:39:22 [INFO] [TRAIN] epoch: 270, iter: 66800/120000, loss: 0.8496, lr: 0.004809, batch_cost: 0.2046, reader_cost: 0.00126, ips: 14.6626 samples/sec | ETA 03:01:24
- 2022-04-13 15:39:33 [INFO] [TRAIN] epoch: 270, iter: 66850/120000, loss: 0.8686, lr: 0.004805, batch_cost: 0.2211, reader_cost: 0.00095, ips: 13.5687 samples/sec | ETA 03:15:51
- 2022-04-13 15:39:44 [INFO] [TRAIN] epoch: 270, iter: 66900/120000, loss: 0.8605, lr: 0.004801, batch_cost: 0.2129, reader_cost: 0.00092, ips: 14.0928 samples/sec | ETA 03:08:23
- 2022-04-13 15:39:54 [INFO] [TRAIN] epoch: 270, iter: 66950/120000, loss: 0.8809, lr: 0.004797, batch_cost: 0.1976, reader_cost: 0.00090, ips: 15.1794 samples/sec | ETA 02:54:44
- 2022-04-13 15:40:08 [INFO] [TRAIN] epoch: 271, iter: 67000/120000, loss: 0.8644, lr: 0.004793, batch_cost: 0.2728, reader_cost: 0.05223, ips: 10.9987 samples/sec | ETA 04:00:56
- 2022-04-13 15:40:18 [INFO] [TRAIN] epoch: 271, iter: 67050/120000, loss: 0.8643, lr: 0.004789, batch_cost: 0.2051, reader_cost: 0.00104, ips: 14.6295 samples/sec | ETA 03:00:58
- 2022-04-13 15:40:29 [INFO] [TRAIN] epoch: 271, iter: 67100/120000, loss: 0.8423, lr: 0.004785, batch_cost: 0.2235, reader_cost: 0.00129, ips: 13.4241 samples/sec | ETA 03:17:02
- 2022-04-13 15:40:40 [INFO] [TRAIN] epoch: 271, iter: 67150/120000, loss: 0.8950, lr: 0.004781, batch_cost: 0.2124, reader_cost: 0.00068, ips: 14.1245 samples/sec | ETA 03:07:05
- 2022-04-13 15:40:50 [INFO] [TRAIN] epoch: 271, iter: 67200/120000, loss: 0.8696, lr: 0.004777, batch_cost: 0.2135, reader_cost: 0.00125, ips: 14.0487 samples/sec | ETA 03:07:55
- 2022-04-13 15:41:05 [INFO] [TRAIN] epoch: 272, iter: 67250/120000, loss: 0.8590, lr: 0.004772, batch_cost: 0.2833, reader_cost: 0.05287, ips: 10.5885 samples/sec | ETA 04:09:05
- 2022-04-13 15:41:15 [INFO] [TRAIN] epoch: 272, iter: 67300/120000, loss: 0.8667, lr: 0.004768, batch_cost: 0.2021, reader_cost: 0.00063, ips: 14.8464 samples/sec | ETA 02:57:29
- 2022-04-13 15:41:25 [INFO] [TRAIN] epoch: 272, iter: 67350/120000, loss: 0.8641, lr: 0.004764, batch_cost: 0.2140, reader_cost: 0.00141, ips: 14.0212 samples/sec | ETA 03:07:45
- 2022-04-13 15:41:37 [INFO] [TRAIN] epoch: 272, iter: 67400/120000, loss: 0.8738, lr: 0.004760, batch_cost: 0.2309, reader_cost: 0.00131, ips: 12.9925 samples/sec | ETA 03:22:25
- 2022-04-13 15:41:47 [INFO] [TRAIN] epoch: 272, iter: 67450/120000, loss: 0.8696, lr: 0.004756, batch_cost: 0.2077, reader_cost: 0.00136, ips: 14.4429 samples/sec | ETA 03:01:55
- 2022-04-13 15:42:02 [INFO] [TRAIN] epoch: 273, iter: 67500/120000, loss: 0.9163, lr: 0.004752, batch_cost: 0.2884, reader_cost: 0.06342, ips: 10.4040 samples/sec | ETA 04:12:18
- 2022-04-13 15:42:13 [INFO] [TRAIN] epoch: 273, iter: 67550/120000, loss: 0.8784, lr: 0.004748, batch_cost: 0.2323, reader_cost: 0.00104, ips: 12.9153 samples/sec | ETA 03:23:03
- 2022-04-13 15:42:24 [INFO] [TRAIN] epoch: 273, iter: 67600/120000, loss: 0.8500, lr: 0.004744, batch_cost: 0.2148, reader_cost: 0.00123, ips: 13.9649 samples/sec | ETA 03:07:36
- 2022-04-13 15:42:34 [INFO] [TRAIN] epoch: 273, iter: 67650/120000, loss: 0.8555, lr: 0.004740, batch_cost: 0.2047, reader_cost: 0.00069, ips: 14.6557 samples/sec | ETA 02:58:35
- 2022-04-13 15:42:44 [INFO] [TRAIN] epoch: 273, iter: 67700/120000, loss: 0.8759, lr: 0.004736, batch_cost: 0.1903, reader_cost: 0.00070, ips: 15.7682 samples/sec | ETA 02:45:50
- 2022-04-13 15:42:58 [INFO] [TRAIN] epoch: 274, iter: 67750/120000, loss: 0.8825, lr: 0.004732, batch_cost: 0.2941, reader_cost: 0.07270, ips: 10.1998 samples/sec | ETA 04:16:07
- 2022-04-13 15:43:09 [INFO] [TRAIN] epoch: 274, iter: 67800/120000, loss: 0.8859, lr: 0.004728, batch_cost: 0.2140, reader_cost: 0.00114, ips: 14.0212 samples/sec | ETA 03:06:08
- 2022-04-13 15:43:20 [INFO] [TRAIN] epoch: 274, iter: 67850/120000, loss: 0.8602, lr: 0.004724, batch_cost: 0.2142, reader_cost: 0.00072, ips: 14.0079 samples/sec | ETA 03:06:08
- 2022-04-13 15:43:31 [INFO] [TRAIN] epoch: 274, iter: 67900/120000, loss: 0.8778, lr: 0.004720, batch_cost: 0.2274, reader_cost: 0.00081, ips: 13.1901 samples/sec | ETA 03:17:29
- 2022-04-13 15:43:41 [INFO] [TRAIN] epoch: 274, iter: 67950/120000, loss: 0.8758, lr: 0.004715, batch_cost: 0.1926, reader_cost: 0.00087, ips: 15.5730 samples/sec | ETA 02:47:06
- 2022-04-13 15:43:55 [INFO] [TRAIN] epoch: 275, iter: 68000/120000, loss: 0.8695, lr: 0.004711, batch_cost: 0.2912, reader_cost: 0.04756, ips: 10.3039 samples/sec | ETA 04:12:19
- 2022-04-13 15:43:55 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1945 - reader cost: 0.1498
- 2022-04-13 15:44:20 [INFO] [EVAL] #Images: 500 mIoU: 0.7523 Acc: 0.9566 Kappa: 0.9436 Dice: 0.8484
- 2022-04-13 15:44:20 [INFO] [EVAL] Class IoU:
- [0.9791 0.8339 0.9148 0.4463 0.5742 0.6276 0.6922 0.761 0.9192 0.5913
- 0.9446 0.8025 0.613 0.9451 0.7675 0.8812 0.8032 0.4405 0.7573]
- 2022-04-13 15:44:20 [INFO] [EVAL] Class Precision:
- [0.9909 0.8892 0.9378 0.8523 0.8093 0.8298 0.8642 0.9072 0.9583 0.8598
- 0.9706 0.8677 0.7778 0.9739 0.8919 0.9496 0.9276 0.6658 0.8778]
- 2022-04-13 15:44:20 [INFO] [EVAL] Class Recall:
- [0.988 0.9306 0.9739 0.4838 0.664 0.7203 0.7767 0.8253 0.9575 0.6544
- 0.9723 0.9144 0.7432 0.9697 0.8463 0.9245 0.8569 0.5655 0.8466]
- 2022-04-13 15:44:21 [INFO] [EVAL] The model with the best validation mIoU (0.7523) was saved at iter 68000.
- 2022-04-13 15:44:32 [INFO] [TRAIN] epoch: 275, iter: 68050/120000, loss: 0.8805, lr: 0.004707, batch_cost: 0.2143, reader_cost: 0.00086, ips: 14.0015 samples/sec | ETA 03:05:30
- 2022-04-13 15:44:42 [INFO] [TRAIN] epoch: 275, iter: 68100/120000, loss: 0.8535, lr: 0.004703, batch_cost: 0.2029, reader_cost: 0.00110, ips: 14.7832 samples/sec | ETA 02:55:32
- 2022-04-13 15:44:52 [INFO] [TRAIN] epoch: 275, iter: 68150/120000, loss: 0.8690, lr: 0.004699, batch_cost: 0.2029, reader_cost: 0.00105, ips: 14.7879 samples/sec | ETA 02:55:18
- 2022-04-13 15:45:03 [INFO] [TRAIN] epoch: 275, iter: 68200/120000, loss: 0.8700, lr: 0.004695, batch_cost: 0.2245, reader_cost: 0.00154, ips: 13.3613 samples/sec | ETA 03:13:50
- 2022-04-13 15:45:17 [INFO] [TRAIN] epoch: 276, iter: 68250/120000, loss: 0.8498, lr: 0.004691, batch_cost: 0.2852, reader_cost: 0.05632, ips: 10.5192 samples/sec | ETA 04:05:58
- 2022-04-13 15:45:27 [INFO] [TRAIN] epoch: 276, iter: 68300/120000, loss: 0.8755, lr: 0.004687, batch_cost: 0.2009, reader_cost: 0.00065, ips: 14.9350 samples/sec | ETA 02:53:05
- 2022-04-13 15:45:38 [INFO] [TRAIN] epoch: 276, iter: 68350/120000, loss: 0.8484, lr: 0.004683, batch_cost: 0.2090, reader_cost: 0.00056, ips: 14.3528 samples/sec | ETA 02:59:55
- 2022-04-13 15:45:49 [INFO] [TRAIN] epoch: 276, iter: 68400/120000, loss: 0.8679, lr: 0.004679, batch_cost: 0.2288, reader_cost: 0.00116, ips: 13.1099 samples/sec | ETA 03:16:47
- 2022-04-13 15:46:02 [INFO] [TRAIN] epoch: 277, iter: 68450/120000, loss: 0.8740, lr: 0.004675, batch_cost: 0.2600, reader_cost: 0.06009, ips: 11.5382 samples/sec | ETA 03:43:23
- 2022-04-13 15:46:13 [INFO] [TRAIN] epoch: 277, iter: 68500/120000, loss: 0.8505, lr: 0.004671, batch_cost: 0.2139, reader_cost: 0.00072, ips: 14.0230 samples/sec | ETA 03:03:37
- 2022-04-13 15:46:24 [INFO] [TRAIN] epoch: 277, iter: 68550/120000, loss: 0.8681, lr: 0.004667, batch_cost: 0.2135, reader_cost: 0.00071, ips: 14.0507 samples/sec | ETA 03:03:05
- 2022-04-13 15:46:35 [INFO] [TRAIN] epoch: 277, iter: 68600/120000, loss: 0.8432, lr: 0.004662, batch_cost: 0.2205, reader_cost: 0.00079, ips: 13.6066 samples/sec | ETA 03:08:52
- 2022-04-13 15:46:45 [INFO] [TRAIN] epoch: 277, iter: 68650/120000, loss: 0.8609, lr: 0.004658, batch_cost: 0.2062, reader_cost: 0.00221, ips: 14.5494 samples/sec | ETA 02:56:28
- 2022-04-13 15:46:58 [INFO] [TRAIN] epoch: 278, iter: 68700/120000, loss: 0.8619, lr: 0.004654, batch_cost: 0.2618, reader_cost: 0.05144, ips: 11.4599 samples/sec | ETA 03:43:49
- 2022-04-13 15:47:09 [INFO] [TRAIN] epoch: 278, iter: 68750/120000, loss: 0.8840, lr: 0.004650, batch_cost: 0.2088, reader_cost: 0.00131, ips: 14.3686 samples/sec | ETA 02:58:20
- 2022-04-13 15:47:19 [INFO] [TRAIN] epoch: 278, iter: 68800/120000, loss: 0.8755, lr: 0.004646, batch_cost: 0.2006, reader_cost: 0.00091, ips: 14.9558 samples/sec | ETA 02:51:10
- 2022-04-13 15:47:29 [INFO] [TRAIN] epoch: 278, iter: 68850/120000, loss: 0.8813, lr: 0.004642, batch_cost: 0.2035, reader_cost: 0.00078, ips: 14.7423 samples/sec | ETA 02:53:28
- 2022-04-13 15:47:39 [INFO] [TRAIN] epoch: 278, iter: 68900/120000, loss: 0.8651, lr: 0.004638, batch_cost: 0.2111, reader_cost: 0.00053, ips: 14.2114 samples/sec | ETA 02:59:47
- 2022-04-13 15:47:53 [INFO] [TRAIN] epoch: 279, iter: 68950/120000, loss: 0.8799, lr: 0.004634, batch_cost: 0.2712, reader_cost: 0.05546, ips: 11.0618 samples/sec | ETA 03:50:44
- 2022-04-13 15:48:04 [INFO] [TRAIN] epoch: 279, iter: 69000/120000, loss: 0.8705, lr: 0.004630, batch_cost: 0.2140, reader_cost: 0.00109, ips: 14.0164 samples/sec | ETA 03:01:55
- 2022-04-13 15:48:14 [INFO] [TRAIN] epoch: 279, iter: 69050/120000, loss: 0.8581, lr: 0.004626, batch_cost: 0.2088, reader_cost: 0.00099, ips: 14.3659 samples/sec | ETA 02:57:19
- 2022-04-13 15:48:24 [INFO] [TRAIN] epoch: 279, iter: 69100/120000, loss: 0.8469, lr: 0.004622, batch_cost: 0.2059, reader_cost: 0.00067, ips: 14.5674 samples/sec | ETA 02:54:42
- 2022-04-13 15:48:35 [INFO] [TRAIN] epoch: 279, iter: 69150/120000, loss: 0.8626, lr: 0.004617, batch_cost: 0.2160, reader_cost: 0.00106, ips: 13.8902 samples/sec | ETA 03:03:02
- 2022-04-13 15:48:49 [INFO] [TRAIN] epoch: 280, iter: 69200/120000, loss: 0.8688, lr: 0.004613, batch_cost: 0.2680, reader_cost: 0.06516, ips: 11.1938 samples/sec | ETA 03:46:54
- 2022-04-13 15:48:59 [INFO] [TRAIN] epoch: 280, iter: 69250/120000, loss: 0.8804, lr: 0.004609, batch_cost: 0.2050, reader_cost: 0.00095, ips: 14.6320 samples/sec | ETA 02:53:25
- 2022-04-13 15:49:10 [INFO] [TRAIN] epoch: 280, iter: 69300/120000, loss: 0.8724, lr: 0.004605, batch_cost: 0.2142, reader_cost: 0.00094, ips: 14.0030 samples/sec | ETA 03:01:01
- 2022-04-13 15:49:20 [INFO] [TRAIN] epoch: 280, iter: 69350/120000, loss: 0.8583, lr: 0.004601, batch_cost: 0.2077, reader_cost: 0.00083, ips: 14.4463 samples/sec | ETA 02:55:18
- 2022-04-13 15:49:30 [INFO] [TRAIN] epoch: 280, iter: 69400/120000, loss: 0.8568, lr: 0.004597, batch_cost: 0.2075, reader_cost: 0.00179, ips: 14.4556 samples/sec | ETA 02:55:01
- 2022-04-13 15:49:44 [INFO] [TRAIN] epoch: 281, iter: 69450/120000, loss: 0.8689, lr: 0.004593, batch_cost: 0.2684, reader_cost: 0.05286, ips: 11.1762 samples/sec | ETA 03:46:09
- 2022-04-13 15:49:54 [INFO] [TRAIN] epoch: 281, iter: 69500/120000, loss: 0.8860, lr: 0.004589, batch_cost: 0.2105, reader_cost: 0.00049, ips: 14.2551 samples/sec | ETA 02:57:07
- 2022-04-13 15:50:04 [INFO] [TRAIN] epoch: 281, iter: 69550/120000, loss: 0.8678, lr: 0.004585, batch_cost: 0.2025, reader_cost: 0.00127, ips: 14.8169 samples/sec | ETA 02:50:14
- 2022-04-13 15:50:15 [INFO] [TRAIN] epoch: 281, iter: 69600/120000, loss: 0.8726, lr: 0.004581, batch_cost: 0.2219, reader_cost: 0.00110, ips: 13.5206 samples/sec | ETA 03:06:22
- 2022-04-13 15:50:26 [INFO] [TRAIN] epoch: 281, iter: 69650/120000, loss: 0.8676, lr: 0.004577, batch_cost: 0.2139, reader_cost: 0.00081, ips: 14.0258 samples/sec | ETA 02:59:29
- 2022-04-13 15:50:39 [INFO] [TRAIN] epoch: 282, iter: 69700/120000, loss: 0.8616, lr: 0.004573, batch_cost: 0.2651, reader_cost: 0.05942, ips: 11.3164 samples/sec | ETA 03:42:14
- 2022-04-13 15:50:50 [INFO] [TRAIN] epoch: 282, iter: 69750/120000, loss: 0.9074, lr: 0.004568, batch_cost: 0.2063, reader_cost: 0.00104, ips: 14.5452 samples/sec | ETA 02:52:44
- 2022-04-13 15:51:00 [INFO] [TRAIN] epoch: 282, iter: 69800/120000, loss: 0.8852, lr: 0.004564, batch_cost: 0.2107, reader_cost: 0.00095, ips: 14.2352 samples/sec | ETA 02:56:19
- 2022-04-13 15:51:11 [INFO] [TRAIN] epoch: 282, iter: 69850/120000, loss: 0.8635, lr: 0.004560, batch_cost: 0.2125, reader_cost: 0.00103, ips: 14.1158 samples/sec | ETA 02:57:38
- 2022-04-13 15:51:22 [INFO] [TRAIN] epoch: 282, iter: 69900/120000, loss: 0.8607, lr: 0.004556, batch_cost: 0.2273, reader_cost: 0.00062, ips: 13.1956 samples/sec | ETA 03:09:50
- 2022-04-13 15:51:36 [INFO] [TRAIN] epoch: 283, iter: 69950/120000, loss: 0.8917, lr: 0.004552, batch_cost: 0.2734, reader_cost: 0.06247, ips: 10.9723 samples/sec | ETA 03:48:04
- 2022-04-13 15:51:46 [INFO] [TRAIN] epoch: 283, iter: 70000/120000, loss: 0.8859, lr: 0.004548, batch_cost: 0.2036, reader_cost: 0.00068, ips: 14.7351 samples/sec | ETA 02:49:39
- 2022-04-13 15:51:57 [INFO] [TRAIN] epoch: 283, iter: 70050/120000, loss: 0.9102, lr: 0.004544, batch_cost: 0.2187, reader_cost: 0.00189, ips: 13.7176 samples/sec | ETA 03:02:03
- 2022-04-13 15:52:09 [INFO] [TRAIN] epoch: 283, iter: 70100/120000, loss: 0.8993, lr: 0.004540, batch_cost: 0.2437, reader_cost: 0.00185, ips: 12.3082 samples/sec | ETA 03:22:42
- 2022-04-13 15:52:21 [INFO] [TRAIN] epoch: 283, iter: 70150/120000, loss: 0.8630, lr: 0.004536, batch_cost: 0.2382, reader_cost: 0.00102, ips: 12.5949 samples/sec | ETA 03:17:53
- 2022-04-13 15:52:35 [INFO] [TRAIN] epoch: 284, iter: 70200/120000, loss: 0.8524, lr: 0.004532, batch_cost: 0.2841, reader_cost: 0.05084, ips: 10.5581 samples/sec | ETA 03:55:50
- 2022-04-13 15:52:46 [INFO] [TRAIN] epoch: 284, iter: 70250/120000, loss: 0.8410, lr: 0.004528, batch_cost: 0.2037, reader_cost: 0.00103, ips: 14.7259 samples/sec | ETA 02:48:55
- 2022-04-13 15:52:56 [INFO] [TRAIN] epoch: 284, iter: 70300/120000, loss: 0.8688, lr: 0.004523, batch_cost: 0.2179, reader_cost: 0.00132, ips: 13.7648 samples/sec | ETA 03:00:32
- 2022-04-13 15:53:07 [INFO] [TRAIN] epoch: 284, iter: 70350/120000, loss: 0.8740, lr: 0.004519, batch_cost: 0.2118, reader_cost: 0.00059, ips: 14.1631 samples/sec | ETA 02:55:16
- 2022-04-13 15:53:17 [INFO] [TRAIN] epoch: 284, iter: 70400/120000, loss: 0.8634, lr: 0.004515, batch_cost: 0.2070, reader_cost: 0.00064, ips: 14.4902 samples/sec | ETA 02:51:09
- 2022-04-13 15:53:31 [INFO] [TRAIN] epoch: 285, iter: 70450/120000, loss: 0.8951, lr: 0.004511, batch_cost: 0.2672, reader_cost: 0.05393, ips: 11.2296 samples/sec | ETA 03:40:37
- 2022-04-13 15:53:41 [INFO] [TRAIN] epoch: 285, iter: 70500/120000, loss: 0.8827, lr: 0.004507, batch_cost: 0.2106, reader_cost: 0.00078, ips: 14.2424 samples/sec | ETA 02:53:46
- 2022-04-13 15:53:52 [INFO] [TRAIN] epoch: 285, iter: 70550/120000, loss: 0.8642, lr: 0.004503, batch_cost: 0.2050, reader_cost: 0.00117, ips: 14.6368 samples/sec | ETA 02:48:55
- 2022-04-13 15:54:03 [INFO] [TRAIN] epoch: 285, iter: 70600/120000, loss: 0.8691, lr: 0.004499, batch_cost: 0.2280, reader_cost: 0.00148, ips: 13.1557 samples/sec | ETA 03:07:45
- 2022-04-13 15:54:13 [INFO] [TRAIN] epoch: 285, iter: 70650/120000, loss: 0.8225, lr: 0.004495, batch_cost: 0.2063, reader_cost: 0.00058, ips: 14.5415 samples/sec | ETA 02:49:41
- 2022-04-13 15:54:27 [INFO] [TRAIN] epoch: 286, iter: 70700/120000, loss: 0.8659, lr: 0.004491, batch_cost: 0.2686, reader_cost: 0.05957, ips: 11.1697 samples/sec | ETA 03:40:41
- 2022-04-13 15:54:37 [INFO] [TRAIN] epoch: 286, iter: 70750/120000, loss: 0.8899, lr: 0.004487, batch_cost: 0.2019, reader_cost: 0.00169, ips: 14.8553 samples/sec | ETA 02:45:45
- 2022-04-13 15:54:47 [INFO] [TRAIN] epoch: 286, iter: 70800/120000, loss: 0.9061, lr: 0.004482, batch_cost: 0.2113, reader_cost: 0.00116, ips: 14.1990 samples/sec | ETA 02:53:15
- 2022-04-13 15:54:58 [INFO] [TRAIN] epoch: 286, iter: 70850/120000, loss: 0.8809, lr: 0.004478, batch_cost: 0.2117, reader_cost: 0.00109, ips: 14.1704 samples/sec | ETA 02:53:25
- 2022-04-13 15:55:09 [INFO] [TRAIN] epoch: 286, iter: 70900/120000, loss: 0.8741, lr: 0.004474, batch_cost: 0.2113, reader_cost: 0.00079, ips: 14.1959 samples/sec | ETA 02:52:56
- 2022-04-13 15:55:22 [INFO] [TRAIN] epoch: 287, iter: 70950/120000, loss: 0.8973, lr: 0.004470, batch_cost: 0.2676, reader_cost: 0.06049, ips: 11.2103 samples/sec | ETA 03:38:46
- 2022-04-13 15:55:33 [INFO] [TRAIN] epoch: 287, iter: 71000/120000, loss: 0.8629, lr: 0.004466, batch_cost: 0.2143, reader_cost: 0.00070, ips: 13.9971 samples/sec | ETA 02:55:02
- 2022-04-13 15:55:43 [INFO] [TRAIN] epoch: 287, iter: 71050/120000, loss: 0.8676, lr: 0.004462, batch_cost: 0.2109, reader_cost: 0.00068, ips: 14.2267 samples/sec | ETA 02:52:02
- 2022-04-13 15:55:53 [INFO] [TRAIN] epoch: 287, iter: 71100/120000, loss: 0.8477, lr: 0.004458, batch_cost: 0.2062, reader_cost: 0.00110, ips: 14.5475 samples/sec | ETA 02:48:04
- 2022-04-13 15:56:04 [INFO] [TRAIN] epoch: 287, iter: 71150/120000, loss: 0.8806, lr: 0.004454, batch_cost: 0.2068, reader_cost: 0.00083, ips: 14.5061 samples/sec | ETA 02:48:22
- 2022-04-13 15:56:17 [INFO] [TRAIN] epoch: 288, iter: 71200/120000, loss: 0.8802, lr: 0.004450, batch_cost: 0.2668, reader_cost: 0.05944, ips: 11.2428 samples/sec | ETA 03:37:01
- 2022-04-13 15:56:28 [INFO] [TRAIN] epoch: 288, iter: 71250/120000, loss: 0.8502, lr: 0.004446, batch_cost: 0.2267, reader_cost: 0.00139, ips: 13.2306 samples/sec | ETA 03:04:13
- 2022-04-13 15:56:39 [INFO] [TRAIN] epoch: 288, iter: 71300/120000, loss: 0.8547, lr: 0.004441, batch_cost: 0.2018, reader_cost: 0.00045, ips: 14.8642 samples/sec | ETA 02:43:49
- 2022-04-13 15:56:48 [INFO] [TRAIN] epoch: 288, iter: 71350/120000, loss: 0.8735, lr: 0.004437, batch_cost: 0.1967, reader_cost: 0.00063, ips: 15.2547 samples/sec | ETA 02:39:27
- 2022-04-13 15:57:00 [INFO] [TRAIN] epoch: 288, iter: 71400/120000, loss: 0.8778, lr: 0.004433, batch_cost: 0.2278, reader_cost: 0.00116, ips: 13.1684 samples/sec | ETA 03:04:31
- 2022-04-13 15:57:13 [INFO] [TRAIN] epoch: 289, iter: 71450/120000, loss: 0.9033, lr: 0.004429, batch_cost: 0.2714, reader_cost: 0.05603, ips: 11.0549 samples/sec | ETA 03:39:35
- 2022-04-13 15:57:24 [INFO] [TRAIN] epoch: 289, iter: 71500/120000, loss: 0.8501, lr: 0.004425, batch_cost: 0.2073, reader_cost: 0.00095, ips: 14.4696 samples/sec | ETA 02:47:35
- 2022-04-13 15:57:34 [INFO] [TRAIN] epoch: 289, iter: 71550/120000, loss: 0.8724, lr: 0.004421, batch_cost: 0.2000, reader_cost: 0.00062, ips: 15.0019 samples/sec | ETA 02:41:28
- 2022-04-13 15:57:44 [INFO] [TRAIN] epoch: 289, iter: 71600/120000, loss: 0.8650, lr: 0.004417, batch_cost: 0.2040, reader_cost: 0.00079, ips: 14.7092 samples/sec | ETA 02:44:31
- 2022-04-13 15:57:54 [INFO] [TRAIN] epoch: 289, iter: 71650/120000, loss: 0.8620, lr: 0.004413, batch_cost: 0.2051, reader_cost: 0.00045, ips: 14.6294 samples/sec | ETA 02:45:14
- 2022-04-13 15:58:07 [INFO] [TRAIN] epoch: 290, iter: 71700/120000, loss: 0.8581, lr: 0.004409, batch_cost: 0.2615, reader_cost: 0.05328, ips: 11.4704 samples/sec | ETA 03:30:32
- 2022-04-13 15:58:18 [INFO] [TRAIN] epoch: 290, iter: 71750/120000, loss: 0.8734, lr: 0.004404, batch_cost: 0.2109, reader_cost: 0.00134, ips: 14.2268 samples/sec | ETA 02:49:34
- 2022-04-13 15:58:28 [INFO] [TRAIN] epoch: 290, iter: 71800/120000, loss: 0.9305, lr: 0.004400, batch_cost: 0.1996, reader_cost: 0.00149, ips: 15.0287 samples/sec | ETA 02:40:21
- 2022-04-13 15:58:38 [INFO] [TRAIN] epoch: 290, iter: 71850/120000, loss: 0.9049, lr: 0.004396, batch_cost: 0.2039, reader_cost: 0.00096, ips: 14.7113 samples/sec | ETA 02:43:38
- 2022-04-13 15:58:49 [INFO] [TRAIN] epoch: 290, iter: 71900/120000, loss: 0.8645, lr: 0.004392, batch_cost: 0.2105, reader_cost: 0.00068, ips: 14.2545 samples/sec | ETA 02:48:43
- 2022-04-13 15:59:02 [INFO] [TRAIN] epoch: 291, iter: 71950/120000, loss: 0.8915, lr: 0.004388, batch_cost: 0.2692, reader_cost: 0.05924, ips: 11.1431 samples/sec | ETA 03:35:36
- 2022-04-13 15:59:12 [INFO] [TRAIN] epoch: 291, iter: 72000/120000, loss: 0.8549, lr: 0.004384, batch_cost: 0.2066, reader_cost: 0.00123, ips: 14.5192 samples/sec | ETA 02:45:17
- 2022-04-13 15:59:12 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1956 - reader cost: 0.1525
- 2022-04-13 15:59:37 [INFO] [EVAL] #Images: 500 mIoU: 0.7501 Acc: 0.9586 Kappa: 0.9461 Dice: 0.8492
- 2022-04-13 15:59:37 [INFO] [EVAL] Class IoU:
- [0.9807 0.8423 0.9205 0.5434 0.6102 0.6198 0.709 0.7863 0.9222 0.6212
- 0.9476 0.8053 0.5742 0.9442 0.6649 0.8466 0.5971 0.5603 0.7565]
- 2022-04-13 15:59:37 [INFO] [EVAL] Class Precision:
- [0.9885 0.9259 0.951 0.8036 0.8179 0.8185 0.8688 0.9113 0.9506 0.8519
- 0.9721 0.8891 0.8024 0.9654 0.8792 0.8947 0.9313 0.6457 0.8778]
- 2022-04-13 15:59:37 [INFO] [EVAL] Class Recall:
- [0.992 0.9032 0.9662 0.6266 0.7062 0.7186 0.794 0.8515 0.9686 0.6963
- 0.9742 0.8953 0.6687 0.9773 0.7318 0.9404 0.6246 0.809 0.8455]
- 2022-04-13 15:59:38 [INFO] [EVAL] The model with the best validation mIoU (0.7523) was saved at iter 68000.
- 2022-04-13 15:59:47 [INFO] [TRAIN] epoch: 291, iter: 72050/120000, loss: 0.8323, lr: 0.004380, batch_cost: 0.1945, reader_cost: 0.00088, ips: 15.4216 samples/sec | ETA 02:35:27
- 2022-04-13 15:59:58 [INFO] [TRAIN] epoch: 291, iter: 72100/120000, loss: 0.8585, lr: 0.004376, batch_cost: 0.2060, reader_cost: 0.00093, ips: 14.5646 samples/sec | ETA 02:44:26
- 2022-04-13 16:00:08 [INFO] [TRAIN] epoch: 291, iter: 72150/120000, loss: 0.8672, lr: 0.004372, batch_cost: 0.2110, reader_cost: 0.00085, ips: 14.2206 samples/sec | ETA 02:48:14
- 2022-04-13 16:00:21 [INFO] [TRAIN] epoch: 292, iter: 72200/120000, loss: 0.8790, lr: 0.004367, batch_cost: 0.2604, reader_cost: 0.04858, ips: 11.5192 samples/sec | ETA 03:27:28
- 2022-04-13 16:00:31 [INFO] [TRAIN] epoch: 292, iter: 72250/120000, loss: 0.8591, lr: 0.004363, batch_cost: 0.2015, reader_cost: 0.00145, ips: 14.8909 samples/sec | ETA 02:40:19
- 2022-04-13 16:00:42 [INFO] [TRAIN] epoch: 292, iter: 72300/120000, loss: 0.8780, lr: 0.004359, batch_cost: 0.2176, reader_cost: 0.00089, ips: 13.7889 samples/sec | ETA 02:52:57
- 2022-04-13 16:00:53 [INFO] [TRAIN] epoch: 292, iter: 72350/120000, loss: 0.8689, lr: 0.004355, batch_cost: 0.2179, reader_cost: 0.00057, ips: 13.7682 samples/sec | ETA 02:53:02
- 2022-04-13 16:01:04 [INFO] [TRAIN] epoch: 292, iter: 72400/120000, loss: 0.8776, lr: 0.004351, batch_cost: 0.2171, reader_cost: 0.00084, ips: 13.8203 samples/sec | ETA 02:52:12
- 2022-04-13 16:01:17 [INFO] [TRAIN] epoch: 293, iter: 72450/120000, loss: 0.9036, lr: 0.004347, batch_cost: 0.2613, reader_cost: 0.04908, ips: 11.4802 samples/sec | ETA 03:27:05
- 2022-04-13 16:01:29 [INFO] [TRAIN] epoch: 293, iter: 72500/120000, loss: 0.8680, lr: 0.004343, batch_cost: 0.2327, reader_cost: 0.00158, ips: 12.8933 samples/sec | ETA 03:04:12
- 2022-04-13 16:01:39 [INFO] [TRAIN] epoch: 293, iter: 72550/120000, loss: 0.8586, lr: 0.004339, batch_cost: 0.2010, reader_cost: 0.00084, ips: 14.9289 samples/sec | ETA 02:38:55
- 2022-04-13 16:01:49 [INFO] [TRAIN] epoch: 293, iter: 72600/120000, loss: 0.8605, lr: 0.004335, batch_cost: 0.2022, reader_cost: 0.00107, ips: 14.8332 samples/sec | ETA 02:39:46
- 2022-04-13 16:01:59 [INFO] [TRAIN] epoch: 293, iter: 72650/120000, loss: 0.8867, lr: 0.004330, batch_cost: 0.2008, reader_cost: 0.00033, ips: 14.9426 samples/sec | ETA 02:38:26
- 2022-04-13 16:02:13 [INFO] [TRAIN] epoch: 294, iter: 72700/120000, loss: 0.8798, lr: 0.004326, batch_cost: 0.2881, reader_cost: 0.05713, ips: 10.4136 samples/sec | ETA 03:47:06
- 2022-04-13 16:02:23 [INFO] [TRAIN] epoch: 294, iter: 72750/120000, loss: 0.8460, lr: 0.004322, batch_cost: 0.1985, reader_cost: 0.00102, ips: 15.1111 samples/sec | ETA 02:36:20
- 2022-04-13 16:02:33 [INFO] [TRAIN] epoch: 294, iter: 72800/120000, loss: 0.8845, lr: 0.004318, batch_cost: 0.2024, reader_cost: 0.00078, ips: 14.8188 samples/sec | ETA 02:39:15
- 2022-04-13 16:02:44 [INFO] [TRAIN] epoch: 294, iter: 72850/120000, loss: 0.8618, lr: 0.004314, batch_cost: 0.2076, reader_cost: 0.00112, ips: 14.4480 samples/sec | ETA 02:43:10
- 2022-04-13 16:02:54 [INFO] [TRAIN] epoch: 294, iter: 72900/120000, loss: 0.8742, lr: 0.004310, batch_cost: 0.2073, reader_cost: 0.00065, ips: 14.4746 samples/sec | ETA 02:42:41
- 2022-04-13 16:03:07 [INFO] [TRAIN] epoch: 295, iter: 72950/120000, loss: 0.8691, lr: 0.004306, batch_cost: 0.2672, reader_cost: 0.06150, ips: 11.2260 samples/sec | ETA 03:29:33
- 2022-04-13 16:03:18 [INFO] [TRAIN] epoch: 295, iter: 73000/120000, loss: 0.8866, lr: 0.004302, batch_cost: 0.2137, reader_cost: 0.00078, ips: 14.0361 samples/sec | ETA 02:47:25
- 2022-04-13 16:03:28 [INFO] [TRAIN] epoch: 295, iter: 73050/120000, loss: 0.8827, lr: 0.004298, batch_cost: 0.2040, reader_cost: 0.00068, ips: 14.7072 samples/sec | ETA 02:39:36
- 2022-04-13 16:03:39 [INFO] [TRAIN] epoch: 295, iter: 73100/120000, loss: 0.8828, lr: 0.004293, batch_cost: 0.2188, reader_cost: 0.00099, ips: 13.7096 samples/sec | ETA 02:51:02
- 2022-04-13 16:03:50 [INFO] [TRAIN] epoch: 295, iter: 73150/120000, loss: 0.8655, lr: 0.004289, batch_cost: 0.2069, reader_cost: 0.00102, ips: 14.4986 samples/sec | ETA 02:41:34
- 2022-04-13 16:04:04 [INFO] [TRAIN] epoch: 296, iter: 73200/120000, loss: 0.8630, lr: 0.004285, batch_cost: 0.2845, reader_cost: 0.05789, ips: 10.5467 samples/sec | ETA 03:41:52
- 2022-04-13 16:04:15 [INFO] [TRAIN] epoch: 296, iter: 73250/120000, loss: 0.8837, lr: 0.004281, batch_cost: 0.2188, reader_cost: 0.00062, ips: 13.7119 samples/sec | ETA 02:50:28
- 2022-04-13 16:04:25 [INFO] [TRAIN] epoch: 296, iter: 73300/120000, loss: 0.8609, lr: 0.004277, batch_cost: 0.2068, reader_cost: 0.00097, ips: 14.5043 samples/sec | ETA 02:40:59
- 2022-04-13 16:04:37 [INFO] [TRAIN] epoch: 296, iter: 73350/120000, loss: 0.9153, lr: 0.004273, batch_cost: 0.2342, reader_cost: 0.00084, ips: 12.8091 samples/sec | ETA 03:02:05
- 2022-04-13 16:04:47 [INFO] [TRAIN] epoch: 296, iter: 73400/120000, loss: 0.8777, lr: 0.004269, batch_cost: 0.2000, reader_cost: 0.00099, ips: 15.0037 samples/sec | ETA 02:35:17
- 2022-04-13 16:05:01 [INFO] [TRAIN] epoch: 297, iter: 73450/120000, loss: 0.8843, lr: 0.004265, batch_cost: 0.2850, reader_cost: 0.05546, ips: 10.5262 samples/sec | ETA 03:41:06
- 2022-04-13 16:05:11 [INFO] [TRAIN] epoch: 297, iter: 73500/120000, loss: 0.8748, lr: 0.004260, batch_cost: 0.2081, reader_cost: 0.00067, ips: 14.4178 samples/sec | ETA 02:41:15
- 2022-04-13 16:05:23 [INFO] [TRAIN] epoch: 297, iter: 73550/120000, loss: 0.8957, lr: 0.004256, batch_cost: 0.2235, reader_cost: 0.00092, ips: 13.4252 samples/sec | ETA 02:52:59
- 2022-04-13 16:05:33 [INFO] [TRAIN] epoch: 297, iter: 73600/120000, loss: 0.8686, lr: 0.004252, batch_cost: 0.2127, reader_cost: 0.00081, ips: 14.1035 samples/sec | ETA 02:44:29
- 2022-04-13 16:05:44 [INFO] [TRAIN] epoch: 297, iter: 73650/120000, loss: 0.8917, lr: 0.004248, batch_cost: 0.2229, reader_cost: 0.00041, ips: 13.4562 samples/sec | ETA 02:52:13
- 2022-04-13 16:05:58 [INFO] [TRAIN] epoch: 298, iter: 73700/120000, loss: 0.8816, lr: 0.004244, batch_cost: 0.2714, reader_cost: 0.05660, ips: 11.0539 samples/sec | ETA 03:29:25
- 2022-04-13 16:06:09 [INFO] [TRAIN] epoch: 298, iter: 73750/120000, loss: 0.8929, lr: 0.004240, batch_cost: 0.2149, reader_cost: 0.00114, ips: 13.9625 samples/sec | ETA 02:45:37
- 2022-04-13 16:06:20 [INFO] [TRAIN] epoch: 298, iter: 73800/120000, loss: 0.8981, lr: 0.004236, batch_cost: 0.2173, reader_cost: 0.00091, ips: 13.8043 samples/sec | ETA 02:47:20
- 2022-04-13 16:06:30 [INFO] [TRAIN] epoch: 298, iter: 73850/120000, loss: 0.8972, lr: 0.004232, batch_cost: 0.2152, reader_cost: 0.00113, ips: 13.9400 samples/sec | ETA 02:45:31
- 2022-04-13 16:06:42 [INFO] [TRAIN] epoch: 298, iter: 73900/120000, loss: 0.8864, lr: 0.004227, batch_cost: 0.2284, reader_cost: 0.00064, ips: 13.1354 samples/sec | ETA 02:55:28
- 2022-04-13 16:06:57 [INFO] [TRAIN] epoch: 299, iter: 73950/120000, loss: 0.8598, lr: 0.004223, batch_cost: 0.3029, reader_cost: 0.05500, ips: 9.9040 samples/sec | ETA 03:52:28
- 2022-04-13 16:07:08 [INFO] [TRAIN] epoch: 299, iter: 74000/120000, loss: 0.8656, lr: 0.004219, batch_cost: 0.2259, reader_cost: 0.00113, ips: 13.2781 samples/sec | ETA 02:53:13
- 2022-04-13 16:07:19 [INFO] [TRAIN] epoch: 299, iter: 74050/120000, loss: 0.8771, lr: 0.004215, batch_cost: 0.2261, reader_cost: 0.00111, ips: 13.2692 samples/sec | ETA 02:53:08
- 2022-04-13 16:07:32 [INFO] [TRAIN] epoch: 299, iter: 74100/120000, loss: 0.8658, lr: 0.004211, batch_cost: 0.2419, reader_cost: 0.00089, ips: 12.4034 samples/sec | ETA 03:05:01
- 2022-04-13 16:07:42 [INFO] [TRAIN] epoch: 299, iter: 74150/120000, loss: 0.8805, lr: 0.004207, batch_cost: 0.2007, reader_cost: 0.00060, ips: 14.9508 samples/sec | ETA 02:33:20
- 2022-04-13 16:07:56 [INFO] [TRAIN] epoch: 300, iter: 74200/120000, loss: 0.8441, lr: 0.004203, batch_cost: 0.2790, reader_cost: 0.05789, ips: 10.7516 samples/sec | ETA 03:32:59
- 2022-04-13 16:08:06 [INFO] [TRAIN] epoch: 300, iter: 74250/120000, loss: 0.9045, lr: 0.004199, batch_cost: 0.2170, reader_cost: 0.00069, ips: 13.8274 samples/sec | ETA 02:45:25
- 2022-04-13 16:08:17 [INFO] [TRAIN] epoch: 300, iter: 74300/120000, loss: 0.8735, lr: 0.004194, batch_cost: 0.2182, reader_cost: 0.00171, ips: 13.7514 samples/sec | ETA 02:46:09
- 2022-04-13 16:08:28 [INFO] [TRAIN] epoch: 300, iter: 74350/120000, loss: 0.8744, lr: 0.004190, batch_cost: 0.2102, reader_cost: 0.00091, ips: 14.2751 samples/sec | ETA 02:39:53
- 2022-04-13 16:08:38 [INFO] [TRAIN] epoch: 300, iter: 74400/120000, loss: 0.8434, lr: 0.004186, batch_cost: 0.1953, reader_cost: 0.00044, ips: 15.3603 samples/sec | ETA 02:28:26
- 2022-04-13 16:08:53 [INFO] [TRAIN] epoch: 301, iter: 74450/120000, loss: 0.8557, lr: 0.004182, batch_cost: 0.3028, reader_cost: 0.05816, ips: 9.9082 samples/sec | ETA 03:49:51
- 2022-04-13 16:09:03 [INFO] [TRAIN] epoch: 301, iter: 74500/120000, loss: 0.8498, lr: 0.004178, batch_cost: 0.2133, reader_cost: 0.00135, ips: 14.0637 samples/sec | ETA 02:41:45
- 2022-04-13 16:09:14 [INFO] [TRAIN] epoch: 301, iter: 74550/120000, loss: 0.8537, lr: 0.004174, batch_cost: 0.2151, reader_cost: 0.00113, ips: 13.9485 samples/sec | ETA 02:42:55
- 2022-04-13 16:09:24 [INFO] [TRAIN] epoch: 301, iter: 74600/120000, loss: 0.8492, lr: 0.004170, batch_cost: 0.2022, reader_cost: 0.00087, ips: 14.8368 samples/sec | ETA 02:32:59
- 2022-04-13 16:09:37 [INFO] [TRAIN] epoch: 302, iter: 74650/120000, loss: 0.8992, lr: 0.004165, batch_cost: 0.2631, reader_cost: 0.05172, ips: 11.4019 samples/sec | ETA 03:18:52
- 2022-04-13 16:09:48 [INFO] [TRAIN] epoch: 302, iter: 74700/120000, loss: 0.8769, lr: 0.004161, batch_cost: 0.2168, reader_cost: 0.00062, ips: 13.8369 samples/sec | ETA 02:43:41
- 2022-04-13 16:09:59 [INFO] [TRAIN] epoch: 302, iter: 74750/120000, loss: 0.8520, lr: 0.004157, batch_cost: 0.2060, reader_cost: 0.00073, ips: 14.5646 samples/sec | ETA 02:35:20
- 2022-04-13 16:10:10 [INFO] [TRAIN] epoch: 302, iter: 74800/120000, loss: 0.8712, lr: 0.004153, batch_cost: 0.2240, reader_cost: 0.00090, ips: 13.3955 samples/sec | ETA 02:48:42
- 2022-04-13 16:10:21 [INFO] [TRAIN] epoch: 302, iter: 74850/120000, loss: 0.8786, lr: 0.004149, batch_cost: 0.2160, reader_cost: 0.00077, ips: 13.8910 samples/sec | ETA 02:42:30
- 2022-04-13 16:10:34 [INFO] [TRAIN] epoch: 303, iter: 74900/120000, loss: 0.8291, lr: 0.004145, batch_cost: 0.2597, reader_cost: 0.05167, ips: 11.5505 samples/sec | ETA 03:15:13
- 2022-04-13 16:10:44 [INFO] [TRAIN] epoch: 303, iter: 74950/120000, loss: 0.8753, lr: 0.004141, batch_cost: 0.2090, reader_cost: 0.00083, ips: 14.3512 samples/sec | ETA 02:36:57
- 2022-04-13 16:10:55 [INFO] [TRAIN] epoch: 303, iter: 75000/120000, loss: 0.8514, lr: 0.004137, batch_cost: 0.2218, reader_cost: 0.00127, ips: 13.5258 samples/sec | ETA 02:46:20
- 2022-04-13 16:11:06 [INFO] [TRAIN] epoch: 303, iter: 75050/120000, loss: 0.8870, lr: 0.004132, batch_cost: 0.2219, reader_cost: 0.00210, ips: 13.5216 samples/sec | ETA 02:46:12
- 2022-04-13 16:11:17 [INFO] [TRAIN] epoch: 303, iter: 75100/120000, loss: 0.8735, lr: 0.004128, batch_cost: 0.2094, reader_cost: 0.00104, ips: 14.3285 samples/sec | ETA 02:36:40
- 2022-04-13 16:11:30 [INFO] [TRAIN] epoch: 304, iter: 75150/120000, loss: 0.8613, lr: 0.004124, batch_cost: 0.2720, reader_cost: 0.05252, ips: 11.0304 samples/sec | ETA 03:23:18
- 2022-04-13 16:11:40 [INFO] [TRAIN] epoch: 304, iter: 75200/120000, loss: 0.9002, lr: 0.004120, batch_cost: 0.2026, reader_cost: 0.00123, ips: 14.8058 samples/sec | ETA 02:31:17
- 2022-04-13 16:11:51 [INFO] [TRAIN] epoch: 304, iter: 75250/120000, loss: 0.8592, lr: 0.004116, batch_cost: 0.2114, reader_cost: 0.00073, ips: 14.1897 samples/sec | ETA 02:37:41
- 2022-04-13 16:12:01 [INFO] [TRAIN] epoch: 304, iter: 75300/120000, loss: 0.8769, lr: 0.004112, batch_cost: 0.2056, reader_cost: 0.00128, ips: 14.5915 samples/sec | ETA 02:33:10
- 2022-04-13 16:12:11 [INFO] [TRAIN] epoch: 304, iter: 75350/120000, loss: 0.8905, lr: 0.004108, batch_cost: 0.2042, reader_cost: 0.00114, ips: 14.6914 samples/sec | ETA 02:31:57
- 2022-04-13 16:12:25 [INFO] [TRAIN] epoch: 305, iter: 75400/120000, loss: 0.8571, lr: 0.004103, batch_cost: 0.2618, reader_cost: 0.04941, ips: 11.4582 samples/sec | ETA 03:14:37
- 2022-04-13 16:12:35 [INFO] [TRAIN] epoch: 305, iter: 75450/120000, loss: 0.8663, lr: 0.004099, batch_cost: 0.2114, reader_cost: 0.00124, ips: 14.1903 samples/sec | ETA 02:36:58
- 2022-04-13 16:12:45 [INFO] [TRAIN] epoch: 305, iter: 75500/120000, loss: 0.8403, lr: 0.004095, batch_cost: 0.2017, reader_cost: 0.00162, ips: 14.8710 samples/sec | ETA 02:29:37
- 2022-04-13 16:12:56 [INFO] [TRAIN] epoch: 305, iter: 75550/120000, loss: 0.9118, lr: 0.004091, batch_cost: 0.2240, reader_cost: 0.00073, ips: 13.3917 samples/sec | ETA 02:45:57
- 2022-04-13 16:13:07 [INFO] [TRAIN] epoch: 305, iter: 75600/120000, loss: 0.8827, lr: 0.004087, batch_cost: 0.2165, reader_cost: 0.00112, ips: 13.8566 samples/sec | ETA 02:40:12
- 2022-04-13 16:13:22 [INFO] [TRAIN] epoch: 306, iter: 75650/120000, loss: 0.8666, lr: 0.004083, batch_cost: 0.2874, reader_cost: 0.05694, ips: 10.4367 samples/sec | ETA 03:32:28
- 2022-04-13 16:13:32 [INFO] [TRAIN] epoch: 306, iter: 75700/120000, loss: 0.8702, lr: 0.004079, batch_cost: 0.1987, reader_cost: 0.00057, ips: 15.0964 samples/sec | ETA 02:26:43
- 2022-04-13 16:13:43 [INFO] [TRAIN] epoch: 306, iter: 75750/120000, loss: 0.8839, lr: 0.004074, batch_cost: 0.2193, reader_cost: 0.00110, ips: 13.6812 samples/sec | ETA 02:41:43
- 2022-04-13 16:13:53 [INFO] [TRAIN] epoch: 306, iter: 75800/120000, loss: 0.8884, lr: 0.004070, batch_cost: 0.2048, reader_cost: 0.00330, ips: 14.6501 samples/sec | ETA 02:30:51
- 2022-04-13 16:14:03 [INFO] [TRAIN] epoch: 306, iter: 75850/120000, loss: 0.8620, lr: 0.004066, batch_cost: 0.2080, reader_cost: 0.00080, ips: 14.4256 samples/sec | ETA 02:33:01
- 2022-04-13 16:14:17 [INFO] [TRAIN] epoch: 307, iter: 75900/120000, loss: 0.8500, lr: 0.004062, batch_cost: 0.2732, reader_cost: 0.07103, ips: 10.9801 samples/sec | ETA 03:20:49
- 2022-04-13 16:14:28 [INFO] [TRAIN] epoch: 307, iter: 75950/120000, loss: 0.8643, lr: 0.004058, batch_cost: 0.2158, reader_cost: 0.00106, ips: 13.9049 samples/sec | ETA 02:38:23
- 2022-04-13 16:14:38 [INFO] [TRAIN] epoch: 307, iter: 76000/120000, loss: 0.8870, lr: 0.004054, batch_cost: 0.1988, reader_cost: 0.00088, ips: 15.0905 samples/sec | ETA 02:25:47
- 2022-04-13 16:14:38 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1946 - reader cost: 0.1515
- 2022-04-13 16:15:02 [INFO] [EVAL] #Images: 500 mIoU: 0.6552 Acc: 0.9500 Kappa: 0.9350 Dice: 0.7618
- 2022-04-13 16:15:02 [INFO] [EVAL] Class IoU:
- [0.9777 0.8257 0.9066 0.4986 0.4637 0.6253 0.6964 0.7472 0.9171 0.641
- 0.9346 0.7841 0.5631 0.9341 0.0858 0.4151 0.4166 0.2749 0.7407]
- 2022-04-13 16:15:02 [INFO] [EVAL] Class Precision:
- [0.9875 0.9234 0.9554 0.7353 0.7687 0.7939 0.8107 0.9157 0.9411 0.8579
- 0.9555 0.8593 0.8181 0.9575 0.8179 0.4219 0.6381 0.8947 0.823 ]
- 2022-04-13 16:15:02 [INFO] [EVAL] Class Recall:
- [0.99 0.8864 0.9466 0.6077 0.5388 0.7465 0.8317 0.8024 0.9729 0.7171
- 0.9771 0.8995 0.6436 0.9745 0.0875 0.9629 0.5455 0.2841 0.881 ]
- 2022-04-13 16:15:03 [INFO] [EVAL] The model with the best validation mIoU (0.7523) was saved at iter 68000.
- 2022-04-13 16:15:14 [INFO] [TRAIN] epoch: 307, iter: 76050/120000, loss: 0.8571, lr: 0.004050, batch_cost: 0.2321, reader_cost: 0.00134, ips: 12.9255 samples/sec | ETA 02:50:00
- 2022-04-13 16:15:25 [INFO] [TRAIN] epoch: 307, iter: 76100/120000, loss: 0.8570, lr: 0.004045, batch_cost: 0.2066, reader_cost: 0.00132, ips: 14.5239 samples/sec | ETA 02:31:07
- 2022-04-13 16:15:38 [INFO] [TRAIN] epoch: 308, iter: 76150/120000, loss: 0.8657, lr: 0.004041, batch_cost: 0.2686, reader_cost: 0.06237, ips: 11.1678 samples/sec | ETA 03:16:19
- 2022-04-13 16:15:48 [INFO] [TRAIN] epoch: 308, iter: 76200/120000, loss: 0.8604, lr: 0.004037, batch_cost: 0.1981, reader_cost: 0.00117, ips: 15.1455 samples/sec | ETA 02:24:35
- 2022-04-13 16:15:58 [INFO] [TRAIN] epoch: 308, iter: 76250/120000, loss: 0.8657, lr: 0.004033, batch_cost: 0.2048, reader_cost: 0.00127, ips: 14.6502 samples/sec | ETA 02:29:18
- 2022-04-13 16:16:09 [INFO] [TRAIN] epoch: 308, iter: 76300/120000, loss: 0.8917, lr: 0.004029, batch_cost: 0.2089, reader_cost: 0.00063, ips: 14.3641 samples/sec | ETA 02:32:06
- 2022-04-13 16:16:19 [INFO] [TRAIN] epoch: 308, iter: 76350/120000, loss: 0.8870, lr: 0.004025, batch_cost: 0.2059, reader_cost: 0.00147, ips: 14.5717 samples/sec | ETA 02:29:46
- 2022-04-13 16:16:33 [INFO] [TRAIN] epoch: 309, iter: 76400/120000, loss: 0.8717, lr: 0.004021, batch_cost: 0.2829, reader_cost: 0.06112, ips: 10.6039 samples/sec | ETA 03:25:35
- 2022-04-13 16:16:43 [INFO] [TRAIN] epoch: 309, iter: 76450/120000, loss: 0.8514, lr: 0.004016, batch_cost: 0.2079, reader_cost: 0.00129, ips: 14.4328 samples/sec | ETA 02:30:52
- 2022-04-13 16:16:55 [INFO] [TRAIN] epoch: 309, iter: 76500/120000, loss: 0.8671, lr: 0.004012, batch_cost: 0.2325, reader_cost: 0.00073, ips: 12.9056 samples/sec | ETA 02:48:31
- 2022-04-13 16:17:06 [INFO] [TRAIN] epoch: 309, iter: 76550/120000, loss: 0.8448, lr: 0.004008, batch_cost: 0.2206, reader_cost: 0.00110, ips: 13.6005 samples/sec | ETA 02:39:44
- 2022-04-13 16:17:18 [INFO] [TRAIN] epoch: 309, iter: 76600/120000, loss: 0.8448, lr: 0.004004, batch_cost: 0.2300, reader_cost: 0.00089, ips: 13.0458 samples/sec | ETA 02:46:20
- 2022-04-13 16:17:31 [INFO] [TRAIN] epoch: 310, iter: 76650/120000, loss: 0.8438, lr: 0.004000, batch_cost: 0.2768, reader_cost: 0.05502, ips: 10.8396 samples/sec | ETA 03:19:57
- 2022-04-13 16:17:42 [INFO] [TRAIN] epoch: 310, iter: 76700/120000, loss: 0.8635, lr: 0.003996, batch_cost: 0.2211, reader_cost: 0.00069, ips: 13.5711 samples/sec | ETA 02:39:31
- 2022-04-13 16:17:53 [INFO] [TRAIN] epoch: 310, iter: 76750/120000, loss: 0.8554, lr: 0.003991, batch_cost: 0.2113, reader_cost: 0.00067, ips: 14.1963 samples/sec | ETA 02:32:19
- 2022-04-13 16:18:04 [INFO] [TRAIN] epoch: 310, iter: 76800/120000, loss: 0.8540, lr: 0.003987, batch_cost: 0.2171, reader_cost: 0.00105, ips: 13.8201 samples/sec | ETA 02:36:17
- 2022-04-13 16:18:14 [INFO] [TRAIN] epoch: 310, iter: 76850/120000, loss: 0.8336, lr: 0.003983, batch_cost: 0.2068, reader_cost: 0.00122, ips: 14.5038 samples/sec | ETA 02:28:45
- 2022-04-13 16:18:28 [INFO] [TRAIN] epoch: 311, iter: 76900/120000, loss: 0.8655, lr: 0.003979, batch_cost: 0.2700, reader_cost: 0.05485, ips: 11.1092 samples/sec | ETA 03:13:58
- 2022-04-13 16:18:38 [INFO] [TRAIN] epoch: 311, iter: 76950/120000, loss: 0.8636, lr: 0.003975, batch_cost: 0.1985, reader_cost: 0.00084, ips: 15.1112 samples/sec | ETA 02:22:26
- 2022-04-13 16:18:48 [INFO] [TRAIN] epoch: 311, iter: 77000/120000, loss: 0.8706, lr: 0.003971, batch_cost: 0.2034, reader_cost: 0.00082, ips: 14.7512 samples/sec | ETA 02:25:45
- 2022-04-13 16:18:58 [INFO] [TRAIN] epoch: 311, iter: 77050/120000, loss: 0.8523, lr: 0.003967, batch_cost: 0.2017, reader_cost: 0.00101, ips: 14.8762 samples/sec | ETA 02:24:21
- 2022-04-13 16:19:09 [INFO] [TRAIN] epoch: 311, iter: 77100/120000, loss: 0.8456, lr: 0.003962, batch_cost: 0.2144, reader_cost: 0.00136, ips: 13.9918 samples/sec | ETA 02:33:18
- 2022-04-13 16:19:23 [INFO] [TRAIN] epoch: 312, iter: 77150/120000, loss: 0.8761, lr: 0.003958, batch_cost: 0.2875, reader_cost: 0.05766, ips: 10.4340 samples/sec | ETA 03:25:20
- 2022-04-13 16:19:33 [INFO] [TRAIN] epoch: 312, iter: 77200/120000, loss: 0.8683, lr: 0.003954, batch_cost: 0.1974, reader_cost: 0.00149, ips: 15.1944 samples/sec | ETA 02:20:50
- 2022-04-13 16:19:44 [INFO] [TRAIN] epoch: 312, iter: 77250/120000, loss: 0.8301, lr: 0.003950, batch_cost: 0.2119, reader_cost: 0.00143, ips: 14.1582 samples/sec | ETA 02:30:58
- 2022-04-13 16:19:54 [INFO] [TRAIN] epoch: 312, iter: 77300/120000, loss: 0.8541, lr: 0.003946, batch_cost: 0.2008, reader_cost: 0.00149, ips: 14.9407 samples/sec | ETA 02:22:53
- 2022-04-13 16:20:05 [INFO] [TRAIN] epoch: 312, iter: 77350/120000, loss: 0.8550, lr: 0.003942, batch_cost: 0.2210, reader_cost: 0.00075, ips: 13.5764 samples/sec | ETA 02:37:04
- 2022-04-13 16:20:18 [INFO] [TRAIN] epoch: 313, iter: 77400/120000, loss: 0.8401, lr: 0.003937, batch_cost: 0.2650, reader_cost: 0.06390, ips: 11.3223 samples/sec | ETA 03:08:07
- 2022-04-13 16:20:29 [INFO] [TRAIN] epoch: 313, iter: 77450/120000, loss: 0.8339, lr: 0.003933, batch_cost: 0.2233, reader_cost: 0.00117, ips: 13.4354 samples/sec | ETA 02:38:21
- 2022-04-13 16:20:39 [INFO] [TRAIN] epoch: 313, iter: 77500/120000, loss: 0.8344, lr: 0.003929, batch_cost: 0.2086, reader_cost: 0.00163, ips: 14.3797 samples/sec | ETA 02:27:46
- 2022-04-13 16:20:50 [INFO] [TRAIN] epoch: 313, iter: 77550/120000, loss: 0.8967, lr: 0.003925, batch_cost: 0.2086, reader_cost: 0.00124, ips: 14.3807 samples/sec | ETA 02:27:35
- 2022-04-13 16:21:01 [INFO] [TRAIN] epoch: 313, iter: 77600/120000, loss: 0.9019, lr: 0.003921, batch_cost: 0.2174, reader_cost: 0.00079, ips: 13.8021 samples/sec | ETA 02:33:36
- 2022-04-13 16:21:14 [INFO] [TRAIN] epoch: 314, iter: 77650/120000, loss: 0.8684, lr: 0.003917, batch_cost: 0.2719, reader_cost: 0.05903, ips: 11.0328 samples/sec | ETA 03:11:55
- 2022-04-13 16:21:25 [INFO] [TRAIN] epoch: 314, iter: 77700/120000, loss: 0.8487, lr: 0.003912, batch_cost: 0.2128, reader_cost: 0.00381, ips: 14.0973 samples/sec | ETA 02:30:01
- 2022-04-13 16:21:35 [INFO] [TRAIN] epoch: 314, iter: 77750/120000, loss: 0.8682, lr: 0.003908, batch_cost: 0.2005, reader_cost: 0.00120, ips: 14.9622 samples/sec | ETA 02:21:11
- 2022-04-13 16:21:46 [INFO] [TRAIN] epoch: 314, iter: 77800/120000, loss: 0.8678, lr: 0.003904, batch_cost: 0.2286, reader_cost: 0.00098, ips: 13.1221 samples/sec | ETA 02:40:47
- 2022-04-13 16:21:57 [INFO] [TRAIN] epoch: 314, iter: 77850/120000, loss: 0.8379, lr: 0.003900, batch_cost: 0.2139, reader_cost: 0.00153, ips: 14.0227 samples/sec | ETA 02:30:17
- 2022-04-13 16:22:11 [INFO] [TRAIN] epoch: 315, iter: 77900/120000, loss: 0.8545, lr: 0.003896, batch_cost: 0.2698, reader_cost: 0.05895, ips: 11.1190 samples/sec | ETA 03:09:18
- 2022-04-13 16:22:21 [INFO] [TRAIN] epoch: 315, iter: 77950/120000, loss: 0.8562, lr: 0.003892, batch_cost: 0.2061, reader_cost: 0.00141, ips: 14.5562 samples/sec | ETA 02:24:26
- 2022-04-13 16:22:32 [INFO] [TRAIN] epoch: 315, iter: 78000/120000, loss: 0.8674, lr: 0.003888, batch_cost: 0.2231, reader_cost: 0.00128, ips: 13.4471 samples/sec | ETA 02:36:10
- 2022-04-13 16:22:43 [INFO] [TRAIN] epoch: 315, iter: 78050/120000, loss: 0.8505, lr: 0.003883, batch_cost: 0.2088, reader_cost: 0.00086, ips: 14.3656 samples/sec | ETA 02:26:00
- 2022-04-13 16:22:53 [INFO] [TRAIN] epoch: 315, iter: 78100/120000, loss: 0.8625, lr: 0.003879, batch_cost: 0.2091, reader_cost: 0.00062, ips: 14.3457 samples/sec | ETA 02:26:02
- 2022-04-13 16:23:06 [INFO] [TRAIN] epoch: 316, iter: 78150/120000, loss: 0.9115, lr: 0.003875, batch_cost: 0.2648, reader_cost: 0.05504, ips: 11.3272 samples/sec | ETA 03:04:43
- 2022-04-13 16:23:17 [INFO] [TRAIN] epoch: 316, iter: 78200/120000, loss: 0.8826, lr: 0.003871, batch_cost: 0.2106, reader_cost: 0.00090, ips: 14.2449 samples/sec | ETA 02:26:43
- 2022-04-13 16:23:29 [INFO] [TRAIN] epoch: 316, iter: 78250/120000, loss: 0.9448, lr: 0.003867, batch_cost: 0.2350, reader_cost: 0.00089, ips: 12.7650 samples/sec | ETA 02:43:32
- 2022-04-13 16:23:38 [INFO] [TRAIN] epoch: 316, iter: 78300/120000, loss: 0.8570, lr: 0.003863, batch_cost: 0.1974, reader_cost: 0.00124, ips: 15.1990 samples/sec | ETA 02:17:10
- 2022-04-13 16:23:49 [INFO] [TRAIN] epoch: 316, iter: 78350/120000, loss: 0.8668, lr: 0.003858, batch_cost: 0.2048, reader_cost: 0.00139, ips: 14.6482 samples/sec | ETA 02:22:10
- 2022-04-13 16:24:02 [INFO] [TRAIN] epoch: 317, iter: 78400/120000, loss: 0.8604, lr: 0.003854, batch_cost: 0.2726, reader_cost: 0.05270, ips: 11.0038 samples/sec | ETA 03:09:01
- 2022-04-13 16:24:12 [INFO] [TRAIN] epoch: 317, iter: 78450/120000, loss: 0.8673, lr: 0.003850, batch_cost: 0.2034, reader_cost: 0.00115, ips: 14.7483 samples/sec | ETA 02:20:51
- 2022-04-13 16:24:24 [INFO] [TRAIN] epoch: 317, iter: 78500/120000, loss: 0.8525, lr: 0.003846, batch_cost: 0.2299, reader_cost: 0.00057, ips: 13.0489 samples/sec | ETA 02:39:01
- 2022-04-13 16:24:37 [INFO] [TRAIN] epoch: 317, iter: 78550/120000, loss: 0.8385, lr: 0.003842, batch_cost: 0.2599, reader_cost: 0.00067, ips: 11.5421 samples/sec | ETA 02:59:33
- 2022-04-13 16:24:48 [INFO] [TRAIN] epoch: 317, iter: 78600/120000, loss: 0.8348, lr: 0.003837, batch_cost: 0.2221, reader_cost: 0.00088, ips: 13.5047 samples/sec | ETA 02:33:16
- 2022-04-13 16:25:01 [INFO] [TRAIN] epoch: 318, iter: 78650/120000, loss: 0.8714, lr: 0.003833, batch_cost: 0.2646, reader_cost: 0.05747, ips: 11.3394 samples/sec | ETA 03:02:19
- 2022-04-13 16:25:12 [INFO] [TRAIN] epoch: 318, iter: 78700/120000, loss: 0.8566, lr: 0.003829, batch_cost: 0.2059, reader_cost: 0.00092, ips: 14.5729 samples/sec | ETA 02:21:42
- 2022-04-13 16:25:22 [INFO] [TRAIN] epoch: 318, iter: 78750/120000, loss: 0.8462, lr: 0.003825, batch_cost: 0.2150, reader_cost: 0.00086, ips: 13.9515 samples/sec | ETA 02:27:50
- 2022-04-13 16:25:34 [INFO] [TRAIN] epoch: 318, iter: 78800/120000, loss: 0.8305, lr: 0.003821, batch_cost: 0.2316, reader_cost: 0.00100, ips: 12.9536 samples/sec | ETA 02:39:01
- 2022-04-13 16:25:45 [INFO] [TRAIN] epoch: 318, iter: 78850/120000, loss: 0.8514, lr: 0.003817, batch_cost: 0.2183, reader_cost: 0.00114, ips: 13.7411 samples/sec | ETA 02:29:44
- 2022-04-13 16:25:58 [INFO] [TRAIN] epoch: 319, iter: 78900/120000, loss: 0.8745, lr: 0.003812, batch_cost: 0.2710, reader_cost: 0.05733, ips: 11.0713 samples/sec | ETA 03:05:36
- 2022-04-13 16:26:10 [INFO] [TRAIN] epoch: 319, iter: 78950/120000, loss: 0.8671, lr: 0.003808, batch_cost: 0.2273, reader_cost: 0.00096, ips: 13.1992 samples/sec | ETA 02:35:30
- 2022-04-13 16:26:20 [INFO] [TRAIN] epoch: 319, iter: 79000/120000, loss: 0.8929, lr: 0.003804, batch_cost: 0.2031, reader_cost: 0.00074, ips: 14.7727 samples/sec | ETA 02:18:46
- 2022-04-13 16:26:30 [INFO] [TRAIN] epoch: 319, iter: 79050/120000, loss: 0.8522, lr: 0.003800, batch_cost: 0.2067, reader_cost: 0.00100, ips: 14.5161 samples/sec | ETA 02:21:03
- 2022-04-13 16:26:40 [INFO] [TRAIN] epoch: 319, iter: 79100/120000, loss: 0.8474, lr: 0.003796, batch_cost: 0.1990, reader_cost: 0.00079, ips: 15.0731 samples/sec | ETA 02:15:40
- 2022-04-13 16:26:54 [INFO] [TRAIN] epoch: 320, iter: 79150/120000, loss: 0.8637, lr: 0.003792, batch_cost: 0.2677, reader_cost: 0.05524, ips: 11.2046 samples/sec | ETA 03:02:17
- 2022-04-13 16:27:05 [INFO] [TRAIN] epoch: 320, iter: 79200/120000, loss: 0.8769, lr: 0.003787, batch_cost: 0.2379, reader_cost: 0.00150, ips: 12.6105 samples/sec | ETA 02:41:46
- 2022-04-13 16:27:16 [INFO] [TRAIN] epoch: 320, iter: 79250/120000, loss: 0.8844, lr: 0.003783, batch_cost: 0.2160, reader_cost: 0.00096, ips: 13.8910 samples/sec | ETA 02:26:40
- 2022-04-13 16:27:27 [INFO] [TRAIN] epoch: 320, iter: 79300/120000, loss: 0.8643, lr: 0.003779, batch_cost: 0.2175, reader_cost: 0.00103, ips: 13.7913 samples/sec | ETA 02:27:33
- 2022-04-13 16:27:37 [INFO] [TRAIN] epoch: 320, iter: 79350/120000, loss: 0.8620, lr: 0.003775, batch_cost: 0.1999, reader_cost: 0.00054, ips: 15.0054 samples/sec | ETA 02:15:27
- 2022-04-13 16:27:52 [INFO] [TRAIN] epoch: 321, iter: 79400/120000, loss: 0.8638, lr: 0.003771, batch_cost: 0.3059, reader_cost: 0.05328, ips: 9.8057 samples/sec | ETA 03:27:01
- 2022-04-13 16:28:03 [INFO] [TRAIN] epoch: 321, iter: 79450/120000, loss: 0.8525, lr: 0.003767, batch_cost: 0.2138, reader_cost: 0.00151, ips: 14.0317 samples/sec | ETA 02:24:29
- 2022-04-13 16:28:13 [INFO] [TRAIN] epoch: 321, iter: 79500/120000, loss: 0.8459, lr: 0.003762, batch_cost: 0.2028, reader_cost: 0.00094, ips: 14.7894 samples/sec | ETA 02:16:55
- 2022-04-13 16:28:25 [INFO] [TRAIN] epoch: 321, iter: 79550/120000, loss: 0.8622, lr: 0.003758, batch_cost: 0.2282, reader_cost: 0.00047, ips: 13.1455 samples/sec | ETA 02:33:51
- 2022-04-13 16:28:35 [INFO] [TRAIN] epoch: 321, iter: 79600/120000, loss: 0.8556, lr: 0.003754, batch_cost: 0.1985, reader_cost: 0.00104, ips: 15.1115 samples/sec | ETA 02:13:40
- 2022-04-13 16:28:48 [INFO] [TRAIN] epoch: 322, iter: 79650/120000, loss: 0.8871, lr: 0.003750, batch_cost: 0.2743, reader_cost: 0.05486, ips: 10.9372 samples/sec | ETA 03:04:27
- 2022-04-13 16:28:59 [INFO] [TRAIN] epoch: 322, iter: 79700/120000, loss: 0.8648, lr: 0.003746, batch_cost: 0.2079, reader_cost: 0.00136, ips: 14.4281 samples/sec | ETA 02:19:39
- 2022-04-13 16:29:11 [INFO] [TRAIN] epoch: 322, iter: 79750/120000, loss: 0.8653, lr: 0.003741, batch_cost: 0.2456, reader_cost: 0.00054, ips: 12.2126 samples/sec | ETA 02:44:47
- 2022-04-13 16:29:22 [INFO] [TRAIN] epoch: 322, iter: 79800/120000, loss: 0.8652, lr: 0.003737, batch_cost: 0.2097, reader_cost: 0.00084, ips: 14.3093 samples/sec | ETA 02:20:28
- 2022-04-13 16:29:31 [INFO] [TRAIN] epoch: 322, iter: 79850/120000, loss: 0.8423, lr: 0.003733, batch_cost: 0.1993, reader_cost: 0.00046, ips: 15.0517 samples/sec | ETA 02:13:22
- 2022-04-13 16:29:45 [INFO] [TRAIN] epoch: 323, iter: 79900/120000, loss: 0.8272, lr: 0.003729, batch_cost: 0.2711, reader_cost: 0.05089, ips: 11.0645 samples/sec | ETA 03:01:12
- 2022-04-13 16:29:56 [INFO] [TRAIN] epoch: 323, iter: 79950/120000, loss: 0.8612, lr: 0.003725, batch_cost: 0.2215, reader_cost: 0.00094, ips: 13.5452 samples/sec | ETA 02:27:50
- 2022-04-13 16:30:07 [INFO] [TRAIN] epoch: 323, iter: 80000/120000, loss: 0.8478, lr: 0.003720, batch_cost: 0.2132, reader_cost: 0.00093, ips: 14.0700 samples/sec | ETA 02:22:08
- 2022-04-13 16:30:07 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1958 - reader cost: 0.1516
- 2022-04-13 16:30:31 [INFO] [EVAL] #Images: 500 mIoU: 0.7638 Acc: 0.9578 Kappa: 0.9453 Dice: 0.8573
- 2022-04-13 16:30:31 [INFO] [EVAL] Class IoU:
- [0.9802 0.8441 0.9204 0.4067 0.5691 0.6309 0.7069 0.7941 0.919 0.6308
- 0.9483 0.8117 0.6323 0.9446 0.7749 0.8448 0.7885 0.5979 0.7667]
- 2022-04-13 16:30:31 [INFO] [EVAL] Class Precision:
- [0.9901 0.9153 0.9488 0.8187 0.7474 0.8471 0.8531 0.9082 0.9573 0.7459
- 0.9695 0.89 0.7543 0.9709 0.8476 0.9023 0.8928 0.7012 0.851 ]
- 2022-04-13 16:30:31 [INFO] [EVAL] Class Recall:
- [0.9899 0.9156 0.9684 0.447 0.7046 0.712 0.805 0.8635 0.9583 0.8036
- 0.9775 0.9022 0.7963 0.9721 0.9004 0.9298 0.8709 0.8023 0.8856]
- 2022-04-13 16:30:32 [INFO] [EVAL] The model with the best validation mIoU (0.7638) was saved at iter 80000.
- 2022-04-13 16:30:43 [INFO] [TRAIN] epoch: 323, iter: 80050/120000, loss: 0.8987, lr: 0.003716, batch_cost: 0.2092, reader_cost: 0.00154, ips: 14.3388 samples/sec | ETA 02:19:18
- 2022-04-13 16:30:53 [INFO] [TRAIN] epoch: 323, iter: 80100/120000, loss: 0.8712, lr: 0.003712, batch_cost: 0.1955, reader_cost: 0.00066, ips: 15.3467 samples/sec | ETA 02:09:59
- 2022-04-13 16:31:07 [INFO] [TRAIN] epoch: 324, iter: 80150/120000, loss: 0.8553, lr: 0.003708, batch_cost: 0.2771, reader_cost: 0.05244, ips: 10.8269 samples/sec | ETA 03:04:01
- 2022-04-13 16:31:18 [INFO] [TRAIN] epoch: 324, iter: 80200/120000, loss: 0.8391, lr: 0.003704, batch_cost: 0.2215, reader_cost: 0.00102, ips: 13.5471 samples/sec | ETA 02:26:53
- 2022-04-13 16:31:28 [INFO] [TRAIN] epoch: 324, iter: 80250/120000, loss: 0.9064, lr: 0.003700, batch_cost: 0.2033, reader_cost: 0.00092, ips: 14.7596 samples/sec | ETA 02:14:39
- 2022-04-13 16:31:39 [INFO] [TRAIN] epoch: 324, iter: 80300/120000, loss: 0.8362, lr: 0.003695, batch_cost: 0.2179, reader_cost: 0.00129, ips: 13.7658 samples/sec | ETA 02:24:11
- 2022-04-13 16:31:50 [INFO] [TRAIN] epoch: 324, iter: 80350/120000, loss: 0.8384, lr: 0.003691, batch_cost: 0.2093, reader_cost: 0.00041, ips: 14.3359 samples/sec | ETA 02:18:17
- 2022-04-13 16:32:04 [INFO] [TRAIN] epoch: 325, iter: 80400/120000, loss: 0.8414, lr: 0.003687, batch_cost: 0.2923, reader_cost: 0.05475, ips: 10.2620 samples/sec | ETA 03:12:56
- 2022-04-13 16:32:15 [INFO] [TRAIN] epoch: 325, iter: 80450/120000, loss: 0.8508, lr: 0.003683, batch_cost: 0.2095, reader_cost: 0.00077, ips: 14.3171 samples/sec | ETA 02:18:07
- 2022-04-13 16:32:26 [INFO] [TRAIN] epoch: 325, iter: 80500/120000, loss: 0.8604, lr: 0.003679, batch_cost: 0.2207, reader_cost: 0.00095, ips: 13.5915 samples/sec | ETA 02:25:18
- 2022-04-13 16:32:37 [INFO] [TRAIN] epoch: 325, iter: 80550/120000, loss: 0.8576, lr: 0.003674, batch_cost: 0.2308, reader_cost: 0.00120, ips: 13.0002 samples/sec | ETA 02:31:43
- 2022-04-13 16:32:47 [INFO] [TRAIN] epoch: 325, iter: 80600/120000, loss: 0.8810, lr: 0.003670, batch_cost: 0.1861, reader_cost: 0.00055, ips: 16.1176 samples/sec | ETA 02:02:13
- 2022-04-13 16:33:01 [INFO] [TRAIN] epoch: 326, iter: 80650/120000, loss: 0.8697, lr: 0.003666, batch_cost: 0.2921, reader_cost: 0.05740, ips: 10.2694 samples/sec | ETA 03:11:35
- 2022-04-13 16:33:12 [INFO] [TRAIN] epoch: 326, iter: 80700/120000, loss: 0.8475, lr: 0.003662, batch_cost: 0.2238, reader_cost: 0.00090, ips: 13.4057 samples/sec | ETA 02:26:34
- 2022-04-13 16:33:23 [INFO] [TRAIN] epoch: 326, iter: 80750/120000, loss: 0.8630, lr: 0.003658, batch_cost: 0.2178, reader_cost: 0.00120, ips: 13.7747 samples/sec | ETA 02:22:28
- 2022-04-13 16:33:34 [INFO] [TRAIN] epoch: 326, iter: 80800/120000, loss: 0.8726, lr: 0.003653, batch_cost: 0.2204, reader_cost: 0.00080, ips: 13.6114 samples/sec | ETA 02:23:59
- 2022-04-13 16:33:48 [INFO] [TRAIN] epoch: 327, iter: 80850/120000, loss: 0.8837, lr: 0.003649, batch_cost: 0.2709, reader_cost: 0.05673, ips: 11.0744 samples/sec | ETA 02:56:45
- 2022-04-13 16:33:59 [INFO] [TRAIN] epoch: 327, iter: 80900/120000, loss: 0.8468, lr: 0.003645, batch_cost: 0.2165, reader_cost: 0.00085, ips: 13.8577 samples/sec | ETA 02:21:04
- 2022-04-13 16:34:09 [INFO] [TRAIN] epoch: 327, iter: 80950/120000, loss: 0.8690, lr: 0.003641, batch_cost: 0.2044, reader_cost: 0.00088, ips: 14.6780 samples/sec | ETA 02:13:01
- 2022-04-13 16:34:19 [INFO] [TRAIN] epoch: 327, iter: 81000/120000, loss: 0.8502, lr: 0.003637, batch_cost: 0.2043, reader_cost: 0.00106, ips: 14.6872 samples/sec | ETA 02:12:46
- 2022-04-13 16:34:30 [INFO] [TRAIN] epoch: 327, iter: 81050/120000, loss: 0.8459, lr: 0.003632, batch_cost: 0.2170, reader_cost: 0.00066, ips: 13.8235 samples/sec | ETA 02:20:53
- 2022-04-13 16:34:44 [INFO] [TRAIN] epoch: 328, iter: 81100/120000, loss: 0.8720, lr: 0.003628, batch_cost: 0.2773, reader_cost: 0.06179, ips: 10.8192 samples/sec | ETA 02:59:46
- 2022-04-13 16:34:55 [INFO] [TRAIN] epoch: 328, iter: 81150/120000, loss: 0.8361, lr: 0.003624, batch_cost: 0.2213, reader_cost: 0.00123, ips: 13.5540 samples/sec | ETA 02:23:18
- 2022-04-13 16:35:05 [INFO] [TRAIN] epoch: 328, iter: 81200/120000, loss: 0.8512, lr: 0.003620, batch_cost: 0.1988, reader_cost: 0.00077, ips: 15.0871 samples/sec | ETA 02:08:35
- 2022-04-13 16:35:15 [INFO] [TRAIN] epoch: 328, iter: 81250/120000, loss: 0.8488, lr: 0.003616, batch_cost: 0.1987, reader_cost: 0.00072, ips: 15.0986 samples/sec | ETA 02:08:19
- 2022-04-13 16:35:25 [INFO] [TRAIN] epoch: 328, iter: 81300/120000, loss: 0.8403, lr: 0.003611, batch_cost: 0.2012, reader_cost: 0.00088, ips: 14.9081 samples/sec | ETA 02:09:47
- 2022-04-13 16:35:38 [INFO] [TRAIN] epoch: 329, iter: 81350/120000, loss: 0.8609, lr: 0.003607, batch_cost: 0.2680, reader_cost: 0.05487, ips: 11.1951 samples/sec | ETA 02:52:37
- 2022-04-13 16:35:50 [INFO] [TRAIN] epoch: 329, iter: 81400/120000, loss: 0.8464, lr: 0.003603, batch_cost: 0.2380, reader_cost: 0.00071, ips: 12.6058 samples/sec | ETA 02:33:06
- 2022-04-13 16:36:01 [INFO] [TRAIN] epoch: 329, iter: 81450/120000, loss: 0.9046, lr: 0.003599, batch_cost: 0.2109, reader_cost: 0.00081, ips: 14.2257 samples/sec | ETA 02:15:29
- 2022-04-13 16:36:12 [INFO] [TRAIN] epoch: 329, iter: 81500/120000, loss: 0.8617, lr: 0.003595, batch_cost: 0.2190, reader_cost: 0.00081, ips: 13.7002 samples/sec | ETA 02:20:30
- 2022-04-13 16:36:23 [INFO] [TRAIN] epoch: 329, iter: 81550/120000, loss: 0.8480, lr: 0.003590, batch_cost: 0.2366, reader_cost: 0.00286, ips: 12.6776 samples/sec | ETA 02:31:38
- 2022-04-13 16:36:37 [INFO] [TRAIN] epoch: 330, iter: 81600/120000, loss: 0.8530, lr: 0.003586, batch_cost: 0.2645, reader_cost: 0.05653, ips: 11.3402 samples/sec | ETA 02:49:18
- 2022-04-13 16:36:47 [INFO] [TRAIN] epoch: 330, iter: 81650/120000, loss: 0.8487, lr: 0.003582, batch_cost: 0.2154, reader_cost: 0.00066, ips: 13.9263 samples/sec | ETA 02:17:41
- 2022-04-13 16:36:57 [INFO] [TRAIN] epoch: 330, iter: 81700/120000, loss: 0.8614, lr: 0.003578, batch_cost: 0.1904, reader_cost: 0.00103, ips: 15.7560 samples/sec | ETA 02:01:32
- 2022-04-13 16:37:07 [INFO] [TRAIN] epoch: 330, iter: 81750/120000, loss: 0.8791, lr: 0.003574, batch_cost: 0.1988, reader_cost: 0.00085, ips: 15.0885 samples/sec | ETA 02:06:45
- 2022-04-13 16:37:17 [INFO] [TRAIN] epoch: 330, iter: 81800/120000, loss: 0.8321, lr: 0.003569, batch_cost: 0.2077, reader_cost: 0.00150, ips: 14.4424 samples/sec | ETA 02:12:14
- 2022-04-13 16:37:31 [INFO] [TRAIN] epoch: 331, iter: 81850/120000, loss: 0.8606, lr: 0.003565, batch_cost: 0.2700, reader_cost: 0.05589, ips: 11.1114 samples/sec | ETA 02:51:40
- 2022-04-13 16:37:42 [INFO] [TRAIN] epoch: 331, iter: 81900/120000, loss: 0.8448, lr: 0.003561, batch_cost: 0.2153, reader_cost: 0.00105, ips: 13.9335 samples/sec | ETA 02:16:43
- 2022-04-13 16:37:52 [INFO] [TRAIN] epoch: 331, iter: 81950/120000, loss: 0.8580, lr: 0.003557, batch_cost: 0.2135, reader_cost: 0.00080, ips: 14.0490 samples/sec | ETA 02:15:25
- 2022-04-13 16:38:03 [INFO] [TRAIN] epoch: 331, iter: 82000/120000, loss: 0.8695, lr: 0.003553, batch_cost: 0.2063, reader_cost: 0.00043, ips: 14.5394 samples/sec | ETA 02:10:40
- 2022-04-13 16:38:14 [INFO] [TRAIN] epoch: 331, iter: 82050/120000, loss: 0.9076, lr: 0.003548, batch_cost: 0.2186, reader_cost: 0.00050, ips: 13.7220 samples/sec | ETA 02:18:16
- 2022-04-13 16:38:27 [INFO] [TRAIN] epoch: 332, iter: 82100/120000, loss: 0.8815, lr: 0.003544, batch_cost: 0.2642, reader_cost: 0.05598, ips: 11.3549 samples/sec | ETA 02:46:53
- 2022-04-13 16:38:37 [INFO] [TRAIN] epoch: 332, iter: 82150/120000, loss: 0.8628, lr: 0.003540, batch_cost: 0.2000, reader_cost: 0.00086, ips: 14.9980 samples/sec | ETA 02:06:11
- 2022-04-13 16:38:47 [INFO] [TRAIN] epoch: 332, iter: 82200/120000, loss: 0.8500, lr: 0.003536, batch_cost: 0.2102, reader_cost: 0.00093, ips: 14.2753 samples/sec | ETA 02:12:23
- 2022-04-13 16:38:57 [INFO] [TRAIN] epoch: 332, iter: 82250/120000, loss: 0.9060, lr: 0.003532, batch_cost: 0.2015, reader_cost: 0.00158, ips: 14.8901 samples/sec | ETA 02:06:45
- 2022-04-13 16:39:09 [INFO] [TRAIN] epoch: 332, iter: 82300/120000, loss: 0.8668, lr: 0.003527, batch_cost: 0.2300, reader_cost: 0.00066, ips: 13.0460 samples/sec | ETA 02:24:29
- 2022-04-13 16:39:24 [INFO] [TRAIN] epoch: 333, iter: 82350/120000, loss: 0.8541, lr: 0.003523, batch_cost: 0.3053, reader_cost: 0.07527, ips: 9.8266 samples/sec | ETA 03:11:34
- 2022-04-13 16:39:35 [INFO] [TRAIN] epoch: 333, iter: 82400/120000, loss: 0.9069, lr: 0.003519, batch_cost: 0.2182, reader_cost: 0.00130, ips: 13.7467 samples/sec | ETA 02:16:45
- 2022-04-13 16:39:46 [INFO] [TRAIN] epoch: 333, iter: 82450/120000, loss: 0.8565, lr: 0.003515, batch_cost: 0.2107, reader_cost: 0.00077, ips: 14.2391 samples/sec | ETA 02:11:51
- 2022-04-13 16:39:57 [INFO] [TRAIN] epoch: 333, iter: 82500/120000, loss: 0.8486, lr: 0.003511, batch_cost: 0.2285, reader_cost: 0.00066, ips: 13.1281 samples/sec | ETA 02:22:49
- 2022-04-13 16:40:07 [INFO] [TRAIN] epoch: 333, iter: 82550/120000, loss: 0.8573, lr: 0.003506, batch_cost: 0.1965, reader_cost: 0.00091, ips: 15.2705 samples/sec | ETA 02:02:37
- 2022-04-13 16:40:21 [INFO] [TRAIN] epoch: 334, iter: 82600/120000, loss: 0.9080, lr: 0.003502, batch_cost: 0.2686, reader_cost: 0.06030, ips: 11.1704 samples/sec | ETA 02:47:24
- 2022-04-13 16:40:31 [INFO] [TRAIN] epoch: 334, iter: 82650/120000, loss: 0.8784, lr: 0.003498, batch_cost: 0.2035, reader_cost: 0.00163, ips: 14.7394 samples/sec | ETA 02:06:42
- 2022-04-13 16:40:41 [INFO] [TRAIN] epoch: 334, iter: 82700/120000, loss: 0.8491, lr: 0.003494, batch_cost: 0.2061, reader_cost: 0.00074, ips: 14.5542 samples/sec | ETA 02:08:08
- 2022-04-13 16:40:51 [INFO] [TRAIN] epoch: 334, iter: 82750/120000, loss: 0.8291, lr: 0.003489, batch_cost: 0.2091, reader_cost: 0.00167, ips: 14.3455 samples/sec | ETA 02:09:49
- 2022-04-13 16:41:02 [INFO] [TRAIN] epoch: 334, iter: 82800/120000, loss: 0.8458, lr: 0.003485, batch_cost: 0.2093, reader_cost: 0.00070, ips: 14.3332 samples/sec | ETA 02:09:46
- 2022-04-13 16:41:16 [INFO] [TRAIN] epoch: 335, iter: 82850/120000, loss: 0.8899, lr: 0.003481, batch_cost: 0.2882, reader_cost: 0.05242, ips: 10.4080 samples/sec | ETA 02:58:28
- 2022-04-13 16:41:28 [INFO] [TRAIN] epoch: 335, iter: 82900/120000, loss: 0.8707, lr: 0.003477, batch_cost: 0.2244, reader_cost: 0.00132, ips: 13.3696 samples/sec | ETA 02:18:44
- 2022-04-13 16:41:38 [INFO] [TRAIN] epoch: 335, iter: 82950/120000, loss: 0.8812, lr: 0.003473, batch_cost: 0.2101, reader_cost: 0.00132, ips: 14.2820 samples/sec | ETA 02:09:42
- 2022-04-13 16:41:48 [INFO] [TRAIN] epoch: 335, iter: 83000/120000, loss: 0.8283, lr: 0.003468, batch_cost: 0.2072, reader_cost: 0.00047, ips: 14.4817 samples/sec | ETA 02:07:44
- 2022-04-13 16:41:59 [INFO] [TRAIN] epoch: 335, iter: 83050/120000, loss: 0.8548, lr: 0.003464, batch_cost: 0.2140, reader_cost: 0.00079, ips: 14.0207 samples/sec | ETA 02:11:46
- 2022-04-13 16:42:13 [INFO] [TRAIN] epoch: 336, iter: 83100/120000, loss: 0.8633, lr: 0.003460, batch_cost: 0.2742, reader_cost: 0.05461, ips: 10.9408 samples/sec | ETA 02:48:38
- 2022-04-13 16:42:24 [INFO] [TRAIN] epoch: 336, iter: 83150/120000, loss: 0.8396, lr: 0.003456, batch_cost: 0.2162, reader_cost: 0.00145, ips: 13.8750 samples/sec | ETA 02:12:47
- 2022-04-13 16:42:35 [INFO] [TRAIN] epoch: 336, iter: 83200/120000, loss: 0.8613, lr: 0.003452, batch_cost: 0.2235, reader_cost: 0.00100, ips: 13.4245 samples/sec | ETA 02:17:03
- 2022-04-13 16:42:45 [INFO] [TRAIN] epoch: 336, iter: 83250/120000, loss: 0.8418, lr: 0.003447, batch_cost: 0.2073, reader_cost: 0.00091, ips: 14.4714 samples/sec | ETA 02:06:58
- 2022-04-13 16:42:56 [INFO] [TRAIN] epoch: 336, iter: 83300/120000, loss: 0.8476, lr: 0.003443, batch_cost: 0.2126, reader_cost: 0.00080, ips: 14.1078 samples/sec | ETA 02:10:04
- 2022-04-13 16:43:09 [INFO] [TRAIN] epoch: 337, iter: 83350/120000, loss: 0.8530, lr: 0.003439, batch_cost: 0.2639, reader_cost: 0.05709, ips: 11.3665 samples/sec | ETA 02:41:13
- 2022-04-13 16:43:20 [INFO] [TRAIN] epoch: 337, iter: 83400/120000, loss: 0.8338, lr: 0.003435, batch_cost: 0.2126, reader_cost: 0.00116, ips: 14.1083 samples/sec | ETA 02:09:42
- 2022-04-13 16:43:30 [INFO] [TRAIN] epoch: 337, iter: 83450/120000, loss: 0.8479, lr: 0.003430, batch_cost: 0.2153, reader_cost: 0.00077, ips: 13.9346 samples/sec | ETA 02:11:08
- 2022-04-13 16:43:41 [INFO] [TRAIN] epoch: 337, iter: 83500/120000, loss: 0.8789, lr: 0.003426, batch_cost: 0.2040, reader_cost: 0.00108, ips: 14.7060 samples/sec | ETA 02:04:05
- 2022-04-13 16:43:51 [INFO] [TRAIN] epoch: 337, iter: 83550/120000, loss: 0.8459, lr: 0.003422, batch_cost: 0.2154, reader_cost: 0.00087, ips: 13.9297 samples/sec | ETA 02:10:50
- 2022-04-13 16:44:05 [INFO] [TRAIN] epoch: 338, iter: 83600/120000, loss: 0.8240, lr: 0.003418, batch_cost: 0.2672, reader_cost: 0.05442, ips: 11.2258 samples/sec | ETA 02:42:07
- 2022-04-13 16:44:17 [INFO] [TRAIN] epoch: 338, iter: 83650/120000, loss: 0.8563, lr: 0.003414, batch_cost: 0.2387, reader_cost: 0.00104, ips: 12.5705 samples/sec | ETA 02:24:35
- 2022-04-13 16:44:27 [INFO] [TRAIN] epoch: 338, iter: 83700/120000, loss: 0.8717, lr: 0.003409, batch_cost: 0.2092, reader_cost: 0.00162, ips: 14.3388 samples/sec | ETA 02:06:34
- 2022-04-13 16:44:37 [INFO] [TRAIN] epoch: 338, iter: 83750/120000, loss: 0.8493, lr: 0.003405, batch_cost: 0.2024, reader_cost: 0.00110, ips: 14.8196 samples/sec | ETA 02:02:18
- 2022-04-13 16:44:48 [INFO] [TRAIN] epoch: 338, iter: 83800/120000, loss: 0.8471, lr: 0.003401, batch_cost: 0.2080, reader_cost: 0.00066, ips: 14.4245 samples/sec | ETA 02:05:28
- 2022-04-13 16:45:01 [INFO] [TRAIN] epoch: 339, iter: 83850/120000, loss: 0.8267, lr: 0.003397, batch_cost: 0.2688, reader_cost: 0.05391, ips: 11.1592 samples/sec | ETA 02:41:58
- 2022-04-13 16:45:12 [INFO] [TRAIN] epoch: 339, iter: 83900/120000, loss: 0.8733, lr: 0.003392, batch_cost: 0.2247, reader_cost: 0.00075, ips: 13.3503 samples/sec | ETA 02:15:12
- 2022-04-13 16:45:23 [INFO] [TRAIN] epoch: 339, iter: 83950/120000, loss: 0.8344, lr: 0.003388, batch_cost: 0.2196, reader_cost: 0.00053, ips: 13.6591 samples/sec | ETA 02:11:57
- 2022-04-13 16:45:34 [INFO] [TRAIN] epoch: 339, iter: 84000/120000, loss: 0.8608, lr: 0.003384, batch_cost: 0.2115, reader_cost: 0.00074, ips: 14.1821 samples/sec | ETA 02:06:55
- 2022-04-13 16:45:34 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1972 - reader cost: 0.1522
- 2022-04-13 16:45:59 [INFO] [EVAL] #Images: 500 mIoU: 0.7394 Acc: 0.9525 Kappa: 0.9383 Dice: 0.8407
- 2022-04-13 16:45:59 [INFO] [EVAL] Class IoU:
- [0.9745 0.8125 0.9137 0.4545 0.6131 0.6226 0.6318 0.782 0.9051 0.4951
- 0.9179 0.8114 0.6167 0.9407 0.787 0.8084 0.6834 0.5183 0.7599]
- 2022-04-13 16:45:59 [INFO] [EVAL] Class Precision:
- [0.9867 0.8914 0.9464 0.7572 0.8385 0.8251 0.8861 0.9042 0.9413 0.8624
- 0.9369 0.897 0.7537 0.9699 0.8639 0.9344 0.9178 0.742 0.84 ]
- 2022-04-13 16:45:59 [INFO] [EVAL] Class Recall:
- [0.9875 0.9018 0.9635 0.532 0.6952 0.7173 0.6877 0.8526 0.9592 0.5376
- 0.9784 0.8948 0.7723 0.969 0.8985 0.857 0.7279 0.6323 0.8885]
- 2022-04-13 16:45:59 [INFO] [EVAL] The model with the best validation mIoU (0.7638) was saved at iter 80000.
- 2022-04-13 16:46:10 [INFO] [TRAIN] epoch: 339, iter: 84050/120000, loss: 0.8721, lr: 0.003380, batch_cost: 0.2230, reader_cost: 0.00088, ips: 13.4551 samples/sec | ETA 02:13:35
- 2022-04-13 16:46:24 [INFO] [TRAIN] epoch: 340, iter: 84100/120000, loss: 0.8474, lr: 0.003375, batch_cost: 0.2648, reader_cost: 0.05607, ips: 11.3290 samples/sec | ETA 02:38:26
- 2022-04-13 16:46:34 [INFO] [TRAIN] epoch: 340, iter: 84150/120000, loss: 0.8620, lr: 0.003371, batch_cost: 0.2146, reader_cost: 0.00085, ips: 13.9772 samples/sec | ETA 02:08:14
- 2022-04-13 16:46:45 [INFO] [TRAIN] epoch: 340, iter: 84200/120000, loss: 0.8567, lr: 0.003367, batch_cost: 0.2118, reader_cost: 0.00076, ips: 14.1647 samples/sec | ETA 02:06:22
- 2022-04-13 16:46:56 [INFO] [TRAIN] epoch: 340, iter: 84250/120000, loss: 0.8743, lr: 0.003363, batch_cost: 0.2109, reader_cost: 0.00143, ips: 14.2280 samples/sec | ETA 02:05:37
- 2022-04-13 16:47:07 [INFO] [TRAIN] epoch: 340, iter: 84300/120000, loss: 0.8698, lr: 0.003359, batch_cost: 0.2268, reader_cost: 0.00073, ips: 13.2269 samples/sec | ETA 02:14:57
- 2022-04-13 16:47:20 [INFO] [TRAIN] epoch: 341, iter: 84350/120000, loss: 0.8376, lr: 0.003354, batch_cost: 0.2660, reader_cost: 0.05787, ips: 11.2796 samples/sec | ETA 02:38:01
- 2022-04-13 16:47:30 [INFO] [TRAIN] epoch: 341, iter: 84400/120000, loss: 0.8327, lr: 0.003350, batch_cost: 0.1962, reader_cost: 0.00136, ips: 15.2891 samples/sec | ETA 01:56:25
- 2022-04-13 16:47:41 [INFO] [TRAIN] epoch: 341, iter: 84450/120000, loss: 0.8564, lr: 0.003346, batch_cost: 0.2186, reader_cost: 0.00071, ips: 13.7226 samples/sec | ETA 02:09:31
- 2022-04-13 16:47:51 [INFO] [TRAIN] epoch: 341, iter: 84500/120000, loss: 0.8470, lr: 0.003342, batch_cost: 0.2095, reader_cost: 0.00114, ips: 14.3232 samples/sec | ETA 02:03:55
- 2022-04-13 16:48:02 [INFO] [TRAIN] epoch: 341, iter: 84550/120000, loss: 0.8334, lr: 0.003337, batch_cost: 0.2093, reader_cost: 0.00110, ips: 14.3302 samples/sec | ETA 02:03:41
- 2022-04-13 16:48:15 [INFO] [TRAIN] epoch: 342, iter: 84600/120000, loss: 0.8413, lr: 0.003333, batch_cost: 0.2633, reader_cost: 0.04933, ips: 11.3957 samples/sec | ETA 02:35:19
- 2022-04-13 16:48:26 [INFO] [TRAIN] epoch: 342, iter: 84650/120000, loss: 0.8439, lr: 0.003329, batch_cost: 0.2239, reader_cost: 0.00121, ips: 13.4011 samples/sec | ETA 02:11:53
- 2022-04-13 16:48:37 [INFO] [TRAIN] epoch: 342, iter: 84700/120000, loss: 0.8679, lr: 0.003325, batch_cost: 0.2167, reader_cost: 0.00145, ips: 13.8444 samples/sec | ETA 02:07:29
- 2022-04-13 16:48:48 [INFO] [TRAIN] epoch: 342, iter: 84750/120000, loss: 0.8466, lr: 0.003320, batch_cost: 0.2085, reader_cost: 0.00202, ips: 14.3887 samples/sec | ETA 02:02:29
- 2022-04-13 16:48:58 [INFO] [TRAIN] epoch: 342, iter: 84800/120000, loss: 0.8458, lr: 0.003316, batch_cost: 0.2068, reader_cost: 0.00089, ips: 14.5098 samples/sec | ETA 02:01:17
- 2022-04-13 16:49:12 [INFO] [TRAIN] epoch: 343, iter: 84850/120000, loss: 0.8509, lr: 0.003312, batch_cost: 0.2735, reader_cost: 0.05645, ips: 10.9681 samples/sec | ETA 02:40:14
- 2022-04-13 16:49:23 [INFO] [TRAIN] epoch: 343, iter: 84900/120000, loss: 0.8406, lr: 0.003308, batch_cost: 0.2356, reader_cost: 0.00199, ips: 12.7324 samples/sec | ETA 02:17:50
- 2022-04-13 16:49:34 [INFO] [TRAIN] epoch: 343, iter: 84950/120000, loss: 0.8188, lr: 0.003303, batch_cost: 0.2172, reader_cost: 0.00086, ips: 13.8124 samples/sec | ETA 02:06:52
- 2022-04-13 16:49:46 [INFO] [TRAIN] epoch: 343, iter: 85000/120000, loss: 0.8490, lr: 0.003299, batch_cost: 0.2368, reader_cost: 0.00101, ips: 12.6715 samples/sec | ETA 02:18:06
- 2022-04-13 16:49:56 [INFO] [TRAIN] epoch: 343, iter: 85050/120000, loss: 0.8157, lr: 0.003295, batch_cost: 0.2046, reader_cost: 0.00062, ips: 14.6605 samples/sec | ETA 01:59:11
- 2022-04-13 16:50:09 [INFO] [TRAIN] epoch: 344, iter: 85100/120000, loss: 0.8471, lr: 0.003291, batch_cost: 0.2633, reader_cost: 0.05370, ips: 11.3938 samples/sec | ETA 02:33:09
- 2022-04-13 16:50:20 [INFO] [TRAIN] epoch: 344, iter: 85150/120000, loss: 0.8322, lr: 0.003286, batch_cost: 0.2011, reader_cost: 0.00126, ips: 14.9189 samples/sec | ETA 01:56:47
- 2022-04-13 16:50:30 [INFO] [TRAIN] epoch: 344, iter: 85200/120000, loss: 0.8704, lr: 0.003282, batch_cost: 0.2160, reader_cost: 0.00099, ips: 13.8913 samples/sec | ETA 02:05:15
- 2022-04-13 16:50:41 [INFO] [TRAIN] epoch: 344, iter: 85250/120000, loss: 0.8430, lr: 0.003278, batch_cost: 0.2196, reader_cost: 0.00070, ips: 13.6624 samples/sec | ETA 02:07:10
- 2022-04-13 16:50:51 [INFO] [TRAIN] epoch: 344, iter: 85300/120000, loss: 0.8595, lr: 0.003274, batch_cost: 0.2026, reader_cost: 0.00136, ips: 14.8058 samples/sec | ETA 01:57:11
- 2022-04-13 16:51:05 [INFO] [TRAIN] epoch: 345, iter: 85350/120000, loss: 0.8585, lr: 0.003269, batch_cost: 0.2703, reader_cost: 0.05795, ips: 11.0974 samples/sec | ETA 02:36:07
- 2022-04-13 16:51:16 [INFO] [TRAIN] epoch: 345, iter: 85400/120000, loss: 0.8273, lr: 0.003265, batch_cost: 0.2193, reader_cost: 0.00093, ips: 13.6777 samples/sec | ETA 02:06:28
- 2022-04-13 16:51:26 [INFO] [TRAIN] epoch: 345, iter: 85450/120000, loss: 0.8647, lr: 0.003261, batch_cost: 0.2099, reader_cost: 0.00066, ips: 14.2893 samples/sec | ETA 02:00:53
- 2022-04-13 16:51:37 [INFO] [TRAIN] epoch: 345, iter: 85500/120000, loss: 0.8409, lr: 0.003257, batch_cost: 0.2111, reader_cost: 0.00168, ips: 14.2143 samples/sec | ETA 02:01:21
- 2022-04-13 16:51:48 [INFO] [TRAIN] epoch: 345, iter: 85550/120000, loss: 0.8756, lr: 0.003253, batch_cost: 0.2124, reader_cost: 0.00086, ips: 14.1233 samples/sec | ETA 02:01:57
- 2022-04-13 16:52:02 [INFO] [TRAIN] epoch: 346, iter: 85600/120000, loss: 0.8576, lr: 0.003248, batch_cost: 0.2850, reader_cost: 0.05930, ips: 10.5260 samples/sec | ETA 02:43:24
- 2022-04-13 16:52:12 [INFO] [TRAIN] epoch: 346, iter: 85650/120000, loss: 0.8579, lr: 0.003244, batch_cost: 0.2054, reader_cost: 0.00056, ips: 14.6068 samples/sec | ETA 01:57:34
- 2022-04-13 16:52:22 [INFO] [TRAIN] epoch: 346, iter: 85700/120000, loss: 0.8926, lr: 0.003240, batch_cost: 0.2028, reader_cost: 0.00123, ips: 14.7910 samples/sec | ETA 01:55:56
- 2022-04-13 16:52:33 [INFO] [TRAIN] epoch: 346, iter: 85750/120000, loss: 0.8252, lr: 0.003236, batch_cost: 0.2063, reader_cost: 0.00114, ips: 14.5398 samples/sec | ETA 01:57:46
- 2022-04-13 16:52:43 [INFO] [TRAIN] epoch: 346, iter: 85800/120000, loss: 0.8421, lr: 0.003231, batch_cost: 0.2069, reader_cost: 0.00103, ips: 14.4977 samples/sec | ETA 01:57:56
- 2022-04-13 16:52:58 [INFO] [TRAIN] epoch: 347, iter: 85850/120000, loss: 0.8361, lr: 0.003227, batch_cost: 0.2925, reader_cost: 0.06117, ips: 10.2556 samples/sec | ETA 02:46:29
- 2022-04-13 16:53:07 [INFO] [TRAIN] epoch: 347, iter: 85900/120000, loss: 0.8272, lr: 0.003223, batch_cost: 0.1972, reader_cost: 0.00119, ips: 15.2130 samples/sec | ETA 01:52:04
- 2022-04-13 16:53:18 [INFO] [TRAIN] epoch: 347, iter: 85950/120000, loss: 0.8596, lr: 0.003219, batch_cost: 0.2121, reader_cost: 0.00122, ips: 14.1444 samples/sec | ETA 02:00:21
- 2022-04-13 16:53:30 [INFO] [TRAIN] epoch: 347, iter: 86000/120000, loss: 0.9152, lr: 0.003214, batch_cost: 0.2399, reader_cost: 0.00045, ips: 12.5058 samples/sec | ETA 02:15:56
- 2022-04-13 16:53:40 [INFO] [TRAIN] epoch: 347, iter: 86050/120000, loss: 0.8880, lr: 0.003210, batch_cost: 0.1942, reader_cost: 0.00068, ips: 15.4480 samples/sec | ETA 01:49:53
- 2022-04-13 16:53:54 [INFO] [TRAIN] epoch: 348, iter: 86100/120000, loss: 0.8248, lr: 0.003206, batch_cost: 0.2904, reader_cost: 0.06827, ips: 10.3313 samples/sec | ETA 02:44:03
- 2022-04-13 16:54:05 [INFO] [TRAIN] epoch: 348, iter: 86150/120000, loss: 0.8415, lr: 0.003201, batch_cost: 0.2114, reader_cost: 0.00078, ips: 14.1936 samples/sec | ETA 01:59:14
- 2022-04-13 16:54:16 [INFO] [TRAIN] epoch: 348, iter: 86200/120000, loss: 0.8475, lr: 0.003197, batch_cost: 0.2149, reader_cost: 0.00107, ips: 13.9598 samples/sec | ETA 02:01:03
- 2022-04-13 16:54:26 [INFO] [TRAIN] epoch: 348, iter: 86250/120000, loss: 0.8498, lr: 0.003193, batch_cost: 0.1995, reader_cost: 0.00080, ips: 15.0352 samples/sec | ETA 01:52:14
- 2022-04-13 16:54:35 [INFO] [TRAIN] epoch: 348, iter: 86300/120000, loss: 0.8241, lr: 0.003189, batch_cost: 0.1939, reader_cost: 0.00034, ips: 15.4745 samples/sec | ETA 01:48:53
- 2022-04-13 16:54:49 [INFO] [TRAIN] epoch: 349, iter: 86350/120000, loss: 0.8434, lr: 0.003184, batch_cost: 0.2788, reader_cost: 0.05820, ips: 10.7610 samples/sec | ETA 02:36:21
- 2022-04-13 16:55:00 [INFO] [TRAIN] epoch: 349, iter: 86400/120000, loss: 0.8659, lr: 0.003180, batch_cost: 0.2082, reader_cost: 0.00168, ips: 14.4084 samples/sec | ETA 01:56:35
- 2022-04-13 16:55:10 [INFO] [TRAIN] epoch: 349, iter: 86450/120000, loss: 0.8303, lr: 0.003176, batch_cost: 0.2171, reader_cost: 0.00179, ips: 13.8184 samples/sec | ETA 02:01:23
- 2022-04-13 16:55:21 [INFO] [TRAIN] epoch: 349, iter: 86500/120000, loss: 0.8729, lr: 0.003172, batch_cost: 0.2034, reader_cost: 0.00137, ips: 14.7487 samples/sec | ETA 01:53:34
- 2022-04-13 16:55:30 [INFO] [TRAIN] epoch: 349, iter: 86550/120000, loss: 0.8729, lr: 0.003167, batch_cost: 0.1966, reader_cost: 0.00098, ips: 15.2578 samples/sec | ETA 01:49:36
- 2022-04-13 16:55:44 [INFO] [TRAIN] epoch: 350, iter: 86600/120000, loss: 0.8702, lr: 0.003163, batch_cost: 0.2728, reader_cost: 0.04652, ips: 10.9986 samples/sec | ETA 02:31:50
- 2022-04-13 16:55:56 [INFO] [TRAIN] epoch: 350, iter: 86650/120000, loss: 0.8453, lr: 0.003159, batch_cost: 0.2320, reader_cost: 0.00128, ips: 12.9329 samples/sec | ETA 02:08:56
- 2022-04-13 16:56:07 [INFO] [TRAIN] epoch: 350, iter: 86700/120000, loss: 0.8440, lr: 0.003155, batch_cost: 0.2234, reader_cost: 0.00051, ips: 13.4276 samples/sec | ETA 02:03:59
- 2022-04-13 16:56:18 [INFO] [TRAIN] epoch: 350, iter: 86750/120000, loss: 0.8200, lr: 0.003150, batch_cost: 0.2145, reader_cost: 0.00149, ips: 13.9854 samples/sec | ETA 01:58:52
- 2022-04-13 16:56:28 [INFO] [TRAIN] epoch: 350, iter: 86800/120000, loss: 0.8375, lr: 0.003146, batch_cost: 0.2056, reader_cost: 0.00050, ips: 14.5935 samples/sec | ETA 01:53:44
- 2022-04-13 16:56:42 [INFO] [TRAIN] epoch: 351, iter: 86850/120000, loss: 0.8501, lr: 0.003142, batch_cost: 0.2749, reader_cost: 0.05705, ips: 10.9145 samples/sec | ETA 02:31:51
- 2022-04-13 16:56:53 [INFO] [TRAIN] epoch: 351, iter: 86900/120000, loss: 0.8250, lr: 0.003138, batch_cost: 0.2287, reader_cost: 0.00130, ips: 13.1148 samples/sec | ETA 02:06:11
- 2022-04-13 16:57:04 [INFO] [TRAIN] epoch: 351, iter: 86950/120000, loss: 0.8639, lr: 0.003133, batch_cost: 0.2166, reader_cost: 0.00089, ips: 13.8494 samples/sec | ETA 01:59:19
- 2022-04-13 16:57:14 [INFO] [TRAIN] epoch: 351, iter: 87000/120000, loss: 0.8313, lr: 0.003129, batch_cost: 0.2032, reader_cost: 0.00056, ips: 14.7651 samples/sec | ETA 01:51:44
- 2022-04-13 16:57:28 [INFO] [TRAIN] epoch: 352, iter: 87050/120000, loss: 0.8589, lr: 0.003125, batch_cost: 0.2798, reader_cost: 0.05667, ips: 10.7237 samples/sec | ETA 02:33:37
- 2022-04-13 16:57:40 [INFO] [TRAIN] epoch: 352, iter: 87100/120000, loss: 0.8500, lr: 0.003121, batch_cost: 0.2484, reader_cost: 0.00075, ips: 12.0749 samples/sec | ETA 02:16:13
- 2022-04-13 16:57:51 [INFO] [TRAIN] epoch: 352, iter: 87150/120000, loss: 0.8621, lr: 0.003116, batch_cost: 0.2118, reader_cost: 0.00062, ips: 14.1633 samples/sec | ETA 01:55:58
- 2022-04-13 16:58:02 [INFO] [TRAIN] epoch: 352, iter: 87200/120000, loss: 0.8782, lr: 0.003112, batch_cost: 0.2099, reader_cost: 0.00053, ips: 14.2895 samples/sec | ETA 01:54:46
- 2022-04-13 16:58:12 [INFO] [TRAIN] epoch: 352, iter: 87250/120000, loss: 0.8494, lr: 0.003108, batch_cost: 0.2120, reader_cost: 0.00096, ips: 14.1502 samples/sec | ETA 01:55:43
- 2022-04-13 16:58:25 [INFO] [TRAIN] epoch: 353, iter: 87300/120000, loss: 0.8369, lr: 0.003103, batch_cost: 0.2541, reader_cost: 0.05150, ips: 11.8071 samples/sec | ETA 02:18:28
- 2022-04-13 16:58:35 [INFO] [TRAIN] epoch: 353, iter: 87350/120000, loss: 0.8456, lr: 0.003099, batch_cost: 0.2058, reader_cost: 0.00054, ips: 14.5766 samples/sec | ETA 01:51:59
- 2022-04-13 16:58:46 [INFO] [TRAIN] epoch: 353, iter: 87400/120000, loss: 0.8754, lr: 0.003095, batch_cost: 0.2182, reader_cost: 0.00071, ips: 13.7473 samples/sec | ETA 01:58:34
- 2022-04-13 16:58:56 [INFO] [TRAIN] epoch: 353, iter: 87450/120000, loss: 0.8550, lr: 0.003091, batch_cost: 0.2045, reader_cost: 0.00069, ips: 14.6665 samples/sec | ETA 01:50:58
- 2022-04-13 16:59:07 [INFO] [TRAIN] epoch: 353, iter: 87500/120000, loss: 0.8375, lr: 0.003086, batch_cost: 0.2071, reader_cost: 0.00086, ips: 14.4853 samples/sec | ETA 01:52:10
- 2022-04-13 16:59:21 [INFO] [TRAIN] epoch: 354, iter: 87550/120000, loss: 0.8595, lr: 0.003082, batch_cost: 0.2898, reader_cost: 0.05999, ips: 10.3526 samples/sec | ETA 02:36:43
- 2022-04-13 16:59:32 [INFO] [TRAIN] epoch: 354, iter: 87600/120000, loss: 0.8453, lr: 0.003078, batch_cost: 0.2097, reader_cost: 0.00082, ips: 14.3087 samples/sec | ETA 01:53:13
- 2022-04-13 16:59:42 [INFO] [TRAIN] epoch: 354, iter: 87650/120000, loss: 0.8510, lr: 0.003074, batch_cost: 0.2075, reader_cost: 0.00116, ips: 14.4558 samples/sec | ETA 01:51:53
- 2022-04-13 16:59:53 [INFO] [TRAIN] epoch: 354, iter: 87700/120000, loss: 0.8339, lr: 0.003069, batch_cost: 0.2235, reader_cost: 0.00102, ips: 13.4233 samples/sec | ETA 02:00:18
- 2022-04-13 17:00:03 [INFO] [TRAIN] epoch: 354, iter: 87750/120000, loss: 0.8548, lr: 0.003065, batch_cost: 0.2020, reader_cost: 0.00109, ips: 14.8496 samples/sec | ETA 01:48:35
- 2022-04-13 17:00:17 [INFO] [TRAIN] epoch: 355, iter: 87800/120000, loss: 0.8412, lr: 0.003061, batch_cost: 0.2813, reader_cost: 0.05383, ips: 10.6655 samples/sec | ETA 02:30:57
- 2022-04-13 17:00:28 [INFO] [TRAIN] epoch: 355, iter: 87850/120000, loss: 0.8385, lr: 0.003056, batch_cost: 0.2157, reader_cost: 0.00047, ips: 13.9067 samples/sec | ETA 01:55:35
- 2022-04-13 17:00:39 [INFO] [TRAIN] epoch: 355, iter: 87900/120000, loss: 0.8432, lr: 0.003052, batch_cost: 0.2240, reader_cost: 0.00162, ips: 13.3953 samples/sec | ETA 01:59:49
- 2022-04-13 17:00:50 [INFO] [TRAIN] epoch: 355, iter: 87950/120000, loss: 0.8594, lr: 0.003048, batch_cost: 0.2202, reader_cost: 0.00073, ips: 13.6230 samples/sec | ETA 01:57:37
- 2022-04-13 17:01:02 [INFO] [TRAIN] epoch: 355, iter: 88000/120000, loss: 0.8675, lr: 0.003044, batch_cost: 0.2238, reader_cost: 0.00081, ips: 13.4067 samples/sec | ETA 01:59:20
- 2022-04-13 17:01:02 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1974 - reader cost: 0.1521
- 2022-04-13 17:01:26 [INFO] [EVAL] #Images: 500 mIoU: 0.7552 Acc: 0.9580 Kappa: 0.9454 Dice: 0.8529
- 2022-04-13 17:01:26 [INFO] [EVAL] Class IoU:
- [0.9813 0.8465 0.9197 0.5043 0.5893 0.639 0.669 0.7886 0.9193 0.6031
- 0.9467 0.8154 0.6202 0.9401 0.6715 0.8444 0.6832 0.6011 0.7653]
- 2022-04-13 17:01:26 [INFO] [EVAL] Class Precision:
- [0.9916 0.9073 0.9482 0.8203 0.8224 0.8088 0.7182 0.8991 0.9646 0.7355
- 0.9671 0.8894 0.7676 0.9607 0.911 0.9315 0.9383 0.7782 0.8245]
- 2022-04-13 17:01:26 [INFO] [EVAL] Class Recall:
- [0.9895 0.9267 0.9683 0.5669 0.6752 0.7527 0.907 0.8651 0.9513 0.7701
- 0.9782 0.9074 0.7637 0.9777 0.7186 0.9003 0.7154 0.7253 0.9142]
- 2022-04-13 17:01:27 [INFO] [EVAL] The model with the best validation mIoU (0.7638) was saved at iter 80000.
- 2022-04-13 17:01:41 [INFO] [TRAIN] epoch: 356, iter: 88050/120000, loss: 0.8268, lr: 0.003039, batch_cost: 0.2760, reader_cost: 0.05063, ips: 10.8690 samples/sec | ETA 02:26:58
- 2022-04-13 17:01:52 [INFO] [TRAIN] epoch: 356, iter: 88100/120000, loss: 0.8432, lr: 0.003035, batch_cost: 0.2159, reader_cost: 0.00068, ips: 13.8924 samples/sec | ETA 01:54:48
- 2022-04-13 17:02:02 [INFO] [TRAIN] epoch: 356, iter: 88150/120000, loss: 0.8200, lr: 0.003031, batch_cost: 0.2097, reader_cost: 0.00161, ips: 14.3046 samples/sec | ETA 01:51:19
- 2022-04-13 17:02:12 [INFO] [TRAIN] epoch: 356, iter: 88200/120000, loss: 0.8271, lr: 0.003026, batch_cost: 0.2052, reader_cost: 0.00101, ips: 14.6194 samples/sec | ETA 01:48:45
- 2022-04-13 17:02:22 [INFO] [TRAIN] epoch: 356, iter: 88250/120000, loss: 0.8499, lr: 0.003022, batch_cost: 0.2016, reader_cost: 0.00061, ips: 14.8785 samples/sec | ETA 01:46:41
- 2022-04-13 17:02:36 [INFO] [TRAIN] epoch: 357, iter: 88300/120000, loss: 0.8607, lr: 0.003018, batch_cost: 0.2796, reader_cost: 0.07220, ips: 10.7304 samples/sec | ETA 02:27:42
- 2022-04-13 17:02:47 [INFO] [TRAIN] epoch: 357, iter: 88350/120000, loss: 0.8673, lr: 0.003014, batch_cost: 0.2079, reader_cost: 0.00126, ips: 14.4285 samples/sec | ETA 01:49:40
- 2022-04-13 17:02:57 [INFO] [TRAIN] epoch: 357, iter: 88400/120000, loss: 0.8335, lr: 0.003009, batch_cost: 0.2020, reader_cost: 0.00134, ips: 14.8507 samples/sec | ETA 01:46:23
- 2022-04-13 17:03:07 [INFO] [TRAIN] epoch: 357, iter: 88450/120000, loss: 0.8267, lr: 0.003005, batch_cost: 0.2033, reader_cost: 0.00076, ips: 14.7569 samples/sec | ETA 01:46:53
- 2022-04-13 17:03:17 [INFO] [TRAIN] epoch: 357, iter: 88500/120000, loss: 0.8442, lr: 0.003001, batch_cost: 0.2093, reader_cost: 0.00100, ips: 14.3364 samples/sec | ETA 01:49:51
- 2022-04-13 17:03:31 [INFO] [TRAIN] epoch: 358, iter: 88550/120000, loss: 0.8386, lr: 0.002996, batch_cost: 0.2680, reader_cost: 0.05570, ips: 11.1949 samples/sec | ETA 02:20:27
- 2022-04-13 17:03:41 [INFO] [TRAIN] epoch: 358, iter: 88600/120000, loss: 0.8352, lr: 0.002992, batch_cost: 0.1985, reader_cost: 0.00116, ips: 15.1125 samples/sec | ETA 01:43:53
- 2022-04-13 17:03:51 [INFO] [TRAIN] epoch: 358, iter: 88650/120000, loss: 0.8504, lr: 0.002988, batch_cost: 0.2008, reader_cost: 0.00096, ips: 14.9401 samples/sec | ETA 01:44:55
- 2022-04-13 17:04:01 [INFO] [TRAIN] epoch: 358, iter: 88700/120000, loss: 0.8488, lr: 0.002984, batch_cost: 0.2086, reader_cost: 0.00135, ips: 14.3844 samples/sec | ETA 01:48:47
- 2022-04-13 17:04:13 [INFO] [TRAIN] epoch: 358, iter: 88750/120000, loss: 0.8970, lr: 0.002979, batch_cost: 0.2260, reader_cost: 0.00072, ips: 13.2766 samples/sec | ETA 01:57:41
- 2022-04-13 17:04:26 [INFO] [TRAIN] epoch: 359, iter: 88800/120000, loss: 0.8475, lr: 0.002975, batch_cost: 0.2633, reader_cost: 0.05477, ips: 11.3945 samples/sec | ETA 02:16:54
- 2022-04-13 17:04:36 [INFO] [TRAIN] epoch: 359, iter: 88850/120000, loss: 0.8567, lr: 0.002971, batch_cost: 0.2041, reader_cost: 0.00085, ips: 14.6990 samples/sec | ETA 01:45:57
- 2022-04-13 17:04:47 [INFO] [TRAIN] epoch: 359, iter: 88900/120000, loss: 0.8652, lr: 0.002966, batch_cost: 0.2230, reader_cost: 0.00065, ips: 13.4545 samples/sec | ETA 01:55:34
- 2022-04-13 17:04:57 [INFO] [TRAIN] epoch: 359, iter: 88950/120000, loss: 0.8669, lr: 0.002962, batch_cost: 0.2069, reader_cost: 0.00097, ips: 14.5020 samples/sec | ETA 01:47:03
- 2022-04-13 17:05:08 [INFO] [TRAIN] epoch: 359, iter: 89000/120000, loss: 0.8281, lr: 0.002958, batch_cost: 0.2159, reader_cost: 0.00158, ips: 13.8936 samples/sec | ETA 01:51:33
- 2022-04-13 17:05:22 [INFO] [TRAIN] epoch: 360, iter: 89050/120000, loss: 0.8404, lr: 0.002954, batch_cost: 0.2660, reader_cost: 0.06090, ips: 11.2791 samples/sec | ETA 02:17:12
- 2022-04-13 17:05:32 [INFO] [TRAIN] epoch: 360, iter: 89100/120000, loss: 0.8565, lr: 0.002949, batch_cost: 0.2167, reader_cost: 0.00091, ips: 13.8470 samples/sec | ETA 01:51:34
- 2022-04-13 17:05:44 [INFO] [TRAIN] epoch: 360, iter: 89150/120000, loss: 0.8479, lr: 0.002945, batch_cost: 0.2275, reader_cost: 0.00123, ips: 13.1875 samples/sec | ETA 01:56:58
- 2022-04-13 17:05:54 [INFO] [TRAIN] epoch: 360, iter: 89200/120000, loss: 0.8402, lr: 0.002941, batch_cost: 0.2109, reader_cost: 0.00082, ips: 14.2263 samples/sec | ETA 01:48:15
- 2022-04-13 17:06:06 [INFO] [TRAIN] epoch: 360, iter: 89250/120000, loss: 0.8400, lr: 0.002936, batch_cost: 0.2255, reader_cost: 0.00090, ips: 13.3046 samples/sec | ETA 01:55:33
- 2022-04-13 17:06:19 [INFO] [TRAIN] epoch: 361, iter: 89300/120000, loss: 0.8271, lr: 0.002932, batch_cost: 0.2713, reader_cost: 0.06217, ips: 11.0591 samples/sec | ETA 02:18:47
- 2022-04-13 17:06:29 [INFO] [TRAIN] epoch: 361, iter: 89350/120000, loss: 0.8429, lr: 0.002928, batch_cost: 0.1927, reader_cost: 0.00131, ips: 15.5686 samples/sec | ETA 01:38:26
- 2022-04-13 17:06:40 [INFO] [TRAIN] epoch: 361, iter: 89400/120000, loss: 0.8305, lr: 0.002923, batch_cost: 0.2299, reader_cost: 0.00114, ips: 13.0491 samples/sec | ETA 01:57:14
- 2022-04-13 17:06:51 [INFO] [TRAIN] epoch: 361, iter: 89450/120000, loss: 0.8456, lr: 0.002919, batch_cost: 0.2180, reader_cost: 0.00128, ips: 13.7635 samples/sec | ETA 01:50:58
- 2022-04-13 17:07:02 [INFO] [TRAIN] epoch: 361, iter: 89500/120000, loss: 0.8413, lr: 0.002915, batch_cost: 0.2121, reader_cost: 0.00253, ips: 14.1449 samples/sec | ETA 01:47:48
- 2022-04-13 17:07:15 [INFO] [TRAIN] epoch: 362, iter: 89550/120000, loss: 0.8388, lr: 0.002911, batch_cost: 0.2644, reader_cost: 0.06239, ips: 11.3472 samples/sec | ETA 02:14:10
- 2022-04-13 17:07:26 [INFO] [TRAIN] epoch: 362, iter: 89600/120000, loss: 0.8345, lr: 0.002906, batch_cost: 0.2295, reader_cost: 0.00118, ips: 13.0698 samples/sec | ETA 01:56:17
- 2022-04-13 17:07:37 [INFO] [TRAIN] epoch: 362, iter: 89650/120000, loss: 0.8192, lr: 0.002902, batch_cost: 0.2012, reader_cost: 0.00085, ips: 14.9106 samples/sec | ETA 01:41:46
- 2022-04-13 17:07:47 [INFO] [TRAIN] epoch: 362, iter: 89700/120000, loss: 0.8412, lr: 0.002898, batch_cost: 0.2046, reader_cost: 0.00042, ips: 14.6605 samples/sec | ETA 01:43:20
- 2022-04-13 17:07:57 [INFO] [TRAIN] epoch: 362, iter: 89750/120000, loss: 0.8308, lr: 0.002893, batch_cost: 0.2051, reader_cost: 0.00073, ips: 14.6292 samples/sec | ETA 01:43:23
- 2022-04-13 17:08:10 [INFO] [TRAIN] epoch: 363, iter: 89800/120000, loss: 0.8434, lr: 0.002889, batch_cost: 0.2642, reader_cost: 0.05522, ips: 11.3544 samples/sec | ETA 02:12:59
- 2022-04-13 17:08:21 [INFO] [TRAIN] epoch: 363, iter: 89850/120000, loss: 0.8548, lr: 0.002885, batch_cost: 0.2140, reader_cost: 0.00104, ips: 14.0156 samples/sec | ETA 01:47:33
- 2022-04-13 17:08:32 [INFO] [TRAIN] epoch: 363, iter: 89900/120000, loss: 0.8485, lr: 0.002880, batch_cost: 0.2156, reader_cost: 0.00085, ips: 13.9139 samples/sec | ETA 01:48:09
- 2022-04-13 17:08:44 [INFO] [TRAIN] epoch: 363, iter: 89950/120000, loss: 0.8597, lr: 0.002876, batch_cost: 0.2527, reader_cost: 0.00103, ips: 11.8723 samples/sec | ETA 02:06:33
- 2022-04-13 17:08:54 [INFO] [TRAIN] epoch: 363, iter: 90000/120000, loss: 0.8350, lr: 0.002872, batch_cost: 0.1996, reader_cost: 0.00086, ips: 15.0310 samples/sec | ETA 01:39:47
- 2022-04-13 17:09:07 [INFO] [TRAIN] epoch: 364, iter: 90050/120000, loss: 0.8356, lr: 0.002868, batch_cost: 0.2625, reader_cost: 0.05741, ips: 11.4277 samples/sec | ETA 02:11:02
- 2022-04-13 17:09:18 [INFO] [TRAIN] epoch: 364, iter: 90100/120000, loss: 0.8632, lr: 0.002863, batch_cost: 0.2093, reader_cost: 0.00198, ips: 14.3321 samples/sec | ETA 01:44:18
- 2022-04-13 17:09:29 [INFO] [TRAIN] epoch: 364, iter: 90150/120000, loss: 0.8391, lr: 0.002859, batch_cost: 0.2119, reader_cost: 0.00054, ips: 14.1609 samples/sec | ETA 01:45:23
- 2022-04-13 17:09:40 [INFO] [TRAIN] epoch: 364, iter: 90200/120000, loss: 0.8278, lr: 0.002855, batch_cost: 0.2215, reader_cost: 0.00070, ips: 13.5412 samples/sec | ETA 01:50:02
- 2022-04-13 17:09:50 [INFO] [TRAIN] epoch: 364, iter: 90250/120000, loss: 0.8391, lr: 0.002850, batch_cost: 0.2124, reader_cost: 0.00116, ips: 14.1256 samples/sec | ETA 01:45:18
- 2022-04-13 17:10:03 [INFO] [TRAIN] epoch: 365, iter: 90300/120000, loss: 0.8275, lr: 0.002846, batch_cost: 0.2620, reader_cost: 0.05503, ips: 11.4498 samples/sec | ETA 02:09:41
- 2022-04-13 17:10:14 [INFO] [TRAIN] epoch: 365, iter: 90350/120000, loss: 0.8328, lr: 0.002842, batch_cost: 0.2107, reader_cost: 0.00111, ips: 14.2349 samples/sec | ETA 01:44:08
- 2022-04-13 17:10:24 [INFO] [TRAIN] epoch: 365, iter: 90400/120000, loss: 0.8480, lr: 0.002837, batch_cost: 0.2085, reader_cost: 0.00089, ips: 14.3900 samples/sec | ETA 01:42:50
- 2022-04-13 17:10:34 [INFO] [TRAIN] epoch: 365, iter: 90450/120000, loss: 0.8689, lr: 0.002833, batch_cost: 0.2025, reader_cost: 0.00053, ips: 14.8127 samples/sec | ETA 01:39:44
- 2022-04-13 17:10:45 [INFO] [TRAIN] epoch: 365, iter: 90500/120000, loss: 0.8331, lr: 0.002829, batch_cost: 0.2024, reader_cost: 0.00069, ips: 14.8193 samples/sec | ETA 01:39:31
- 2022-04-13 17:10:58 [INFO] [TRAIN] epoch: 366, iter: 90550/120000, loss: 0.8584, lr: 0.002824, batch_cost: 0.2776, reader_cost: 0.05761, ips: 10.8082 samples/sec | ETA 02:16:14
- 2022-04-13 17:11:10 [INFO] [TRAIN] epoch: 366, iter: 90600/120000, loss: 0.8532, lr: 0.002820, batch_cost: 0.2373, reader_cost: 0.00163, ips: 12.6440 samples/sec | ETA 01:56:15
- 2022-04-13 17:11:21 [INFO] [TRAIN] epoch: 366, iter: 90650/120000, loss: 0.8371, lr: 0.002816, batch_cost: 0.2169, reader_cost: 0.00083, ips: 13.8319 samples/sec | ETA 01:46:05
- 2022-04-13 17:11:31 [INFO] [TRAIN] epoch: 366, iter: 90700/120000, loss: 0.8348, lr: 0.002811, batch_cost: 0.2024, reader_cost: 0.00056, ips: 14.8224 samples/sec | ETA 01:38:50
- 2022-04-13 17:11:42 [INFO] [TRAIN] epoch: 366, iter: 90750/120000, loss: 0.8063, lr: 0.002807, batch_cost: 0.2073, reader_cost: 0.00132, ips: 14.4712 samples/sec | ETA 01:41:03
- 2022-04-13 17:11:55 [INFO] [TRAIN] epoch: 367, iter: 90800/120000, loss: 0.8517, lr: 0.002803, batch_cost: 0.2660, reader_cost: 0.05700, ips: 11.2801 samples/sec | ETA 02:09:25
- 2022-04-13 17:12:06 [INFO] [TRAIN] epoch: 367, iter: 90850/120000, loss: 0.8301, lr: 0.002798, batch_cost: 0.2297, reader_cost: 0.00115, ips: 13.0597 samples/sec | ETA 01:51:36
- 2022-04-13 17:12:18 [INFO] [TRAIN] epoch: 367, iter: 90900/120000, loss: 0.8415, lr: 0.002794, batch_cost: 0.2409, reader_cost: 0.00074, ips: 12.4551 samples/sec | ETA 01:56:49
- 2022-04-13 17:12:29 [INFO] [TRAIN] epoch: 367, iter: 90950/120000, loss: 0.8675, lr: 0.002790, batch_cost: 0.2073, reader_cost: 0.00076, ips: 14.4708 samples/sec | ETA 01:40:22
- 2022-04-13 17:12:39 [INFO] [TRAIN] epoch: 367, iter: 91000/120000, loss: 0.8350, lr: 0.002786, batch_cost: 0.2120, reader_cost: 0.00163, ips: 14.1539 samples/sec | ETA 01:42:26
- 2022-04-13 17:12:53 [INFO] [TRAIN] epoch: 368, iter: 91050/120000, loss: 0.8348, lr: 0.002781, batch_cost: 0.2704, reader_cost: 0.05467, ips: 11.0946 samples/sec | ETA 02:10:28
- 2022-04-13 17:13:03 [INFO] [TRAIN] epoch: 368, iter: 91100/120000, loss: 0.8204, lr: 0.002777, batch_cost: 0.2003, reader_cost: 0.00139, ips: 14.9810 samples/sec | ETA 01:36:27
- 2022-04-13 17:13:14 [INFO] [TRAIN] epoch: 368, iter: 91150/120000, loss: 0.8345, lr: 0.002773, batch_cost: 0.2211, reader_cost: 0.00097, ips: 13.5655 samples/sec | ETA 01:46:20
- 2022-04-13 17:13:24 [INFO] [TRAIN] epoch: 368, iter: 91200/120000, loss: 0.8196, lr: 0.002768, batch_cost: 0.2048, reader_cost: 0.00146, ips: 14.6471 samples/sec | ETA 01:38:18
- 2022-04-13 17:13:34 [INFO] [TRAIN] epoch: 368, iter: 91250/120000, loss: 0.8285, lr: 0.002764, batch_cost: 0.2022, reader_cost: 0.00118, ips: 14.8381 samples/sec | ETA 01:36:52
- 2022-04-13 17:13:48 [INFO] [TRAIN] epoch: 369, iter: 91300/120000, loss: 0.8638, lr: 0.002760, batch_cost: 0.2724, reader_cost: 0.04974, ips: 11.0124 samples/sec | ETA 02:10:18
- 2022-04-13 17:13:59 [INFO] [TRAIN] epoch: 369, iter: 91350/120000, loss: 0.8385, lr: 0.002755, batch_cost: 0.2103, reader_cost: 0.00064, ips: 14.2635 samples/sec | ETA 01:40:25
- 2022-04-13 17:14:08 [INFO] [TRAIN] epoch: 369, iter: 91400/120000, loss: 0.8154, lr: 0.002751, batch_cost: 0.1969, reader_cost: 0.00091, ips: 15.2344 samples/sec | ETA 01:33:51
- 2022-04-13 17:14:20 [INFO] [TRAIN] epoch: 369, iter: 91450/120000, loss: 0.8280, lr: 0.002747, batch_cost: 0.2392, reader_cost: 0.00104, ips: 12.5411 samples/sec | ETA 01:53:49
- 2022-04-13 17:14:30 [INFO] [TRAIN] epoch: 369, iter: 91500/120000, loss: 0.8292, lr: 0.002742, batch_cost: 0.2006, reader_cost: 0.00105, ips: 14.9540 samples/sec | ETA 01:35:17
- 2022-04-13 17:14:45 [INFO] [TRAIN] epoch: 370, iter: 91550/120000, loss: 0.8433, lr: 0.002738, batch_cost: 0.2834, reader_cost: 0.06126, ips: 10.5856 samples/sec | ETA 02:14:22
- 2022-04-13 17:14:55 [INFO] [TRAIN] epoch: 370, iter: 91600/120000, loss: 0.8544, lr: 0.002734, batch_cost: 0.2152, reader_cost: 0.00090, ips: 13.9435 samples/sec | ETA 01:41:50
- 2022-04-13 17:15:05 [INFO] [TRAIN] epoch: 370, iter: 91650/120000, loss: 0.8720, lr: 0.002729, batch_cost: 0.1978, reader_cost: 0.00080, ips: 15.1658 samples/sec | ETA 01:33:27
- 2022-04-13 17:15:15 [INFO] [TRAIN] epoch: 370, iter: 91700/120000, loss: 0.8344, lr: 0.002725, batch_cost: 0.1969, reader_cost: 0.00151, ips: 15.2356 samples/sec | ETA 01:32:52
- 2022-04-13 17:15:25 [INFO] [TRAIN] epoch: 370, iter: 91750/120000, loss: 0.8423, lr: 0.002721, batch_cost: 0.1945, reader_cost: 0.00126, ips: 15.4265 samples/sec | ETA 01:31:33
- 2022-04-13 17:15:38 [INFO] [TRAIN] epoch: 371, iter: 91800/120000, loss: 0.8235, lr: 0.002716, batch_cost: 0.2696, reader_cost: 0.05933, ips: 11.1279 samples/sec | ETA 02:06:42
- 2022-04-13 17:15:49 [INFO] [TRAIN] epoch: 371, iter: 91850/120000, loss: 0.8252, lr: 0.002712, batch_cost: 0.2177, reader_cost: 0.00065, ips: 13.7785 samples/sec | ETA 01:42:09
- 2022-04-13 17:15:59 [INFO] [TRAIN] epoch: 371, iter: 91900/120000, loss: 0.8584, lr: 0.002708, batch_cost: 0.2041, reader_cost: 0.00097, ips: 14.6995 samples/sec | ETA 01:35:34
- 2022-04-13 17:16:09 [INFO] [TRAIN] epoch: 371, iter: 91950/120000, loss: 0.8486, lr: 0.002703, batch_cost: 0.2019, reader_cost: 0.00097, ips: 14.8593 samples/sec | ETA 01:34:23
- 2022-04-13 17:16:20 [INFO] [TRAIN] epoch: 371, iter: 92000/120000, loss: 0.8528, lr: 0.002699, batch_cost: 0.2066, reader_cost: 0.00099, ips: 14.5177 samples/sec | ETA 01:36:26
- 2022-04-13 17:16:20 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.2009 - reader cost: 0.1557
- 2022-04-13 17:16:45 [INFO] [EVAL] #Images: 500 mIoU: 0.7738 Acc: 0.9596 Kappa: 0.9476 Dice: 0.8645
- 2022-04-13 17:16:45 [INFO] [EVAL] Class IoU:
- [0.9813 0.847 0.9204 0.4367 0.5961 0.6511 0.7246 0.7961 0.9239 0.6298
- 0.9459 0.8229 0.6336 0.9499 0.7975 0.8767 0.7896 0.6034 0.776 ]
- 2022-04-13 17:16:45 [INFO] [EVAL] Class Precision:
- [0.9914 0.9047 0.9483 0.8284 0.7605 0.848 0.8484 0.892 0.9624 0.804
- 0.9613 0.9007 0.7861 0.9684 0.9339 0.9701 0.9427 0.7133 0.8667]
- 2022-04-13 17:16:45 [INFO] [EVAL] Class Recall:
- [0.9897 0.93 0.9691 0.4802 0.7338 0.7371 0.8323 0.8811 0.9585 0.744
- 0.9834 0.905 0.7655 0.9803 0.8452 0.901 0.8294 0.7966 0.8812]
- 2022-04-13 17:16:46 [INFO] [EVAL] The model with the best validation mIoU (0.7738) was saved at iter 92000.
- 2022-04-13 17:17:01 [INFO] [TRAIN] epoch: 372, iter: 92050/120000, loss: 0.8221, lr: 0.002695, batch_cost: 0.2993, reader_cost: 0.06199, ips: 10.0236 samples/sec | ETA 02:19:25
- 2022-04-13 17:17:11 [INFO] [TRAIN] epoch: 372, iter: 92100/120000, loss: 0.8297, lr: 0.002690, batch_cost: 0.2048, reader_cost: 0.00136, ips: 14.6509 samples/sec | ETA 01:35:12
- 2022-04-13 17:17:22 [INFO] [TRAIN] epoch: 372, iter: 92150/120000, loss: 0.8434, lr: 0.002686, batch_cost: 0.2031, reader_cost: 0.00118, ips: 14.7713 samples/sec | ETA 01:34:16
- 2022-04-13 17:17:32 [INFO] [TRAIN] epoch: 372, iter: 92200/120000, loss: 0.8214, lr: 0.002682, batch_cost: 0.2111, reader_cost: 0.00124, ips: 14.2104 samples/sec | ETA 01:37:48
- 2022-04-13 17:17:43 [INFO] [TRAIN] epoch: 372, iter: 92250/120000, loss: 0.8500, lr: 0.002677, batch_cost: 0.2167, reader_cost: 0.00102, ips: 13.8459 samples/sec | ETA 01:40:12
- 2022-04-13 17:17:56 [INFO] [TRAIN] epoch: 373, iter: 92300/120000, loss: 0.8278, lr: 0.002673, batch_cost: 0.2642, reader_cost: 0.05363, ips: 11.3552 samples/sec | ETA 02:01:58
- 2022-04-13 17:18:07 [INFO] [TRAIN] epoch: 373, iter: 92350/120000, loss: 0.8244, lr: 0.002669, batch_cost: 0.2159, reader_cost: 0.00042, ips: 13.8955 samples/sec | ETA 01:39:29
- 2022-04-13 17:18:17 [INFO] [TRAIN] epoch: 373, iter: 92400/120000, loss: 0.8497, lr: 0.002664, batch_cost: 0.2067, reader_cost: 0.00068, ips: 14.5123 samples/sec | ETA 01:35:05
- 2022-04-13 17:18:28 [INFO] [TRAIN] epoch: 373, iter: 92450/120000, loss: 0.8398, lr: 0.002660, batch_cost: 0.2081, reader_cost: 0.00161, ips: 14.4188 samples/sec | ETA 01:35:32
- 2022-04-13 17:18:37 [INFO] [TRAIN] epoch: 373, iter: 92500/120000, loss: 0.8264, lr: 0.002656, batch_cost: 0.1964, reader_cost: 0.00053, ips: 15.2727 samples/sec | ETA 01:30:01
- 2022-04-13 17:18:51 [INFO] [TRAIN] epoch: 374, iter: 92550/120000, loss: 0.8499, lr: 0.002651, batch_cost: 0.2766, reader_cost: 0.05966, ips: 10.8442 samples/sec | ETA 02:06:33
- 2022-04-13 17:19:02 [INFO] [TRAIN] epoch: 374, iter: 92600/120000, loss: 0.8323, lr: 0.002647, batch_cost: 0.2208, reader_cost: 0.00192, ips: 13.5860 samples/sec | ETA 01:40:50
- 2022-04-13 17:19:12 [INFO] [TRAIN] epoch: 374, iter: 92650/120000, loss: 0.8585, lr: 0.002642, batch_cost: 0.2000, reader_cost: 0.00097, ips: 15.0032 samples/sec | ETA 01:31:08
- 2022-04-13 17:19:23 [INFO] [TRAIN] epoch: 374, iter: 92700/120000, loss: 0.8507, lr: 0.002638, batch_cost: 0.2220, reader_cost: 0.00082, ips: 13.5114 samples/sec | ETA 01:41:01
- 2022-04-13 17:19:33 [INFO] [TRAIN] epoch: 374, iter: 92750/120000, loss: 0.8276, lr: 0.002634, batch_cost: 0.1996, reader_cost: 0.00150, ips: 15.0336 samples/sec | ETA 01:30:37
- 2022-04-13 17:19:48 [INFO] [TRAIN] epoch: 375, iter: 92800/120000, loss: 0.8376, lr: 0.002629, batch_cost: 0.2870, reader_cost: 0.06032, ips: 10.4533 samples/sec | ETA 02:10:06
- 2022-04-13 17:19:58 [INFO] [TRAIN] epoch: 375, iter: 92850/120000, loss: 0.8287, lr: 0.002625, batch_cost: 0.2101, reader_cost: 0.00123, ips: 14.2773 samples/sec | ETA 01:35:04
- 2022-04-13 17:20:09 [INFO] [TRAIN] epoch: 375, iter: 92900/120000, loss: 0.8084, lr: 0.002621, batch_cost: 0.2092, reader_cost: 0.00167, ips: 14.3412 samples/sec | ETA 01:34:28
- 2022-04-13 17:20:19 [INFO] [TRAIN] epoch: 375, iter: 92950/120000, loss: 0.8691, lr: 0.002616, batch_cost: 0.2120, reader_cost: 0.00057, ips: 14.1482 samples/sec | ETA 01:35:35
- 2022-04-13 17:20:29 [INFO] [TRAIN] epoch: 375, iter: 93000/120000, loss: 0.8525, lr: 0.002612, batch_cost: 0.1918, reader_cost: 0.00087, ips: 15.6446 samples/sec | ETA 01:26:17
- 2022-04-13 17:20:44 [INFO] [TRAIN] epoch: 376, iter: 93050/120000, loss: 0.8473, lr: 0.002608, batch_cost: 0.2975, reader_cost: 0.05333, ips: 10.0854 samples/sec | ETA 02:13:36
- 2022-04-13 17:20:54 [INFO] [TRAIN] epoch: 376, iter: 93100/120000, loss: 0.8362, lr: 0.002603, batch_cost: 0.2005, reader_cost: 0.00129, ips: 14.9594 samples/sec | ETA 01:29:54
- 2022-04-13 17:21:04 [INFO] [TRAIN] epoch: 376, iter: 93150/120000, loss: 0.8366, lr: 0.002599, batch_cost: 0.2067, reader_cost: 0.00085, ips: 14.5137 samples/sec | ETA 01:32:29
- 2022-04-13 17:21:15 [INFO] [TRAIN] epoch: 376, iter: 93200/120000, loss: 0.8440, lr: 0.002595, batch_cost: 0.2121, reader_cost: 0.00080, ips: 14.1417 samples/sec | ETA 01:34:45
- 2022-04-13 17:21:28 [INFO] [TRAIN] epoch: 377, iter: 93250/120000, loss: 0.8385, lr: 0.002590, batch_cost: 0.2728, reader_cost: 0.05199, ips: 10.9955 samples/sec | ETA 02:01:38
- 2022-04-13 17:21:40 [INFO] [TRAIN] epoch: 377, iter: 93300/120000, loss: 0.8227, lr: 0.002586, batch_cost: 0.2233, reader_cost: 0.00141, ips: 13.4365 samples/sec | ETA 01:39:21
- 2022-04-13 17:21:49 [INFO] [TRAIN] epoch: 377, iter: 93350/120000, loss: 0.8449, lr: 0.002582, batch_cost: 0.1975, reader_cost: 0.00121, ips: 15.1902 samples/sec | ETA 01:27:43
- 2022-04-13 17:22:00 [INFO] [TRAIN] epoch: 377, iter: 93400/120000, loss: 0.8296, lr: 0.002577, batch_cost: 0.2024, reader_cost: 0.00064, ips: 14.8196 samples/sec | ETA 01:29:44
- 2022-04-13 17:22:10 [INFO] [TRAIN] epoch: 377, iter: 93450/120000, loss: 0.8188, lr: 0.002573, batch_cost: 0.2068, reader_cost: 0.00229, ips: 14.5050 samples/sec | ETA 01:31:31
- 2022-04-13 17:22:23 [INFO] [TRAIN] epoch: 378, iter: 93500/120000, loss: 0.8248, lr: 0.002568, batch_cost: 0.2651, reader_cost: 0.06144, ips: 11.3182 samples/sec | ETA 01:57:04
- 2022-04-13 17:22:34 [INFO] [TRAIN] epoch: 378, iter: 93550/120000, loss: 0.8282, lr: 0.002564, batch_cost: 0.2249, reader_cost: 0.00100, ips: 13.3404 samples/sec | ETA 01:39:08
- 2022-04-13 17:22:45 [INFO] [TRAIN] epoch: 378, iter: 93600/120000, loss: 0.8233, lr: 0.002560, batch_cost: 0.2091, reader_cost: 0.00095, ips: 14.3458 samples/sec | ETA 01:32:00
- 2022-04-13 17:22:55 [INFO] [TRAIN] epoch: 378, iter: 93650/120000, loss: 0.8501, lr: 0.002555, batch_cost: 0.1975, reader_cost: 0.00121, ips: 15.1873 samples/sec | ETA 01:26:45
- 2022-04-13 17:23:06 [INFO] [TRAIN] epoch: 378, iter: 93700/120000, loss: 0.8348, lr: 0.002551, batch_cost: 0.2184, reader_cost: 0.00175, ips: 13.7371 samples/sec | ETA 01:35:43
- 2022-04-13 17:23:20 [INFO] [TRAIN] epoch: 379, iter: 93750/120000, loss: 0.8474, lr: 0.002547, batch_cost: 0.2759, reader_cost: 0.05572, ips: 10.8721 samples/sec | ETA 02:00:43
- 2022-04-13 17:23:30 [INFO] [TRAIN] epoch: 379, iter: 93800/120000, loss: 0.8337, lr: 0.002542, batch_cost: 0.2068, reader_cost: 0.00132, ips: 14.5049 samples/sec | ETA 01:30:18
- 2022-04-13 17:23:40 [INFO] [TRAIN] epoch: 379, iter: 93850/120000, loss: 0.8139, lr: 0.002538, batch_cost: 0.2100, reader_cost: 0.00060, ips: 14.2868 samples/sec | ETA 01:31:31
- 2022-04-13 17:23:50 [INFO] [TRAIN] epoch: 379, iter: 93900/120000, loss: 0.8196, lr: 0.002534, batch_cost: 0.1977, reader_cost: 0.00159, ips: 15.1730 samples/sec | ETA 01:26:00
- 2022-04-13 17:24:01 [INFO] [TRAIN] epoch: 379, iter: 93950/120000, loss: 0.8275, lr: 0.002529, batch_cost: 0.2062, reader_cost: 0.00148, ips: 14.5493 samples/sec | ETA 01:29:31
- 2022-04-13 17:24:14 [INFO] [TRAIN] epoch: 380, iter: 94000/120000, loss: 0.8259, lr: 0.002525, batch_cost: 0.2731, reader_cost: 0.06301, ips: 10.9850 samples/sec | ETA 01:58:20
- 2022-04-13 17:24:25 [INFO] [TRAIN] epoch: 380, iter: 94050/120000, loss: 0.8611, lr: 0.002520, batch_cost: 0.2161, reader_cost: 0.00116, ips: 13.8807 samples/sec | ETA 01:33:28
- 2022-04-13 17:24:35 [INFO] [TRAIN] epoch: 380, iter: 94100/120000, loss: 0.8435, lr: 0.002516, batch_cost: 0.2013, reader_cost: 0.00064, ips: 14.9020 samples/sec | ETA 01:26:54
- 2022-04-13 17:24:46 [INFO] [TRAIN] epoch: 380, iter: 94150/120000, loss: 0.8336, lr: 0.002512, batch_cost: 0.2206, reader_cost: 0.00175, ips: 13.6016 samples/sec | ETA 01:35:01
- 2022-04-13 17:24:56 [INFO] [TRAIN] epoch: 380, iter: 94200/120000, loss: 0.8300, lr: 0.002507, batch_cost: 0.1995, reader_cost: 0.00115, ips: 15.0359 samples/sec | ETA 01:25:47
- 2022-04-13 17:25:10 [INFO] [TRAIN] epoch: 381, iter: 94250/120000, loss: 0.8327, lr: 0.002503, batch_cost: 0.2723, reader_cost: 0.05909, ips: 11.0185 samples/sec | ETA 01:56:50
- 2022-04-13 17:25:20 [INFO] [TRAIN] epoch: 381, iter: 94300/120000, loss: 0.8303, lr: 0.002499, batch_cost: 0.2000, reader_cost: 0.00099, ips: 14.9976 samples/sec | ETA 01:25:40
- 2022-04-13 17:25:30 [INFO] [TRAIN] epoch: 381, iter: 94350/120000, loss: 0.8328, lr: 0.002494, batch_cost: 0.2108, reader_cost: 0.00074, ips: 14.2338 samples/sec | ETA 01:30:06
- 2022-04-13 17:25:41 [INFO] [TRAIN] epoch: 381, iter: 94400/120000, loss: 0.8333, lr: 0.002490, batch_cost: 0.2180, reader_cost: 0.00114, ips: 13.7586 samples/sec | ETA 01:33:01
- 2022-04-13 17:25:52 [INFO] [TRAIN] epoch: 381, iter: 94450/120000, loss: 0.8415, lr: 0.002485, batch_cost: 0.2093, reader_cost: 0.00115, ips: 14.3306 samples/sec | ETA 01:29:08
- 2022-04-13 17:26:05 [INFO] [TRAIN] epoch: 382, iter: 94500/120000, loss: 0.8203, lr: 0.002481, batch_cost: 0.2665, reader_cost: 0.05611, ips: 11.2561 samples/sec | ETA 01:53:16
- 2022-04-13 17:26:16 [INFO] [TRAIN] epoch: 382, iter: 94550/120000, loss: 0.8531, lr: 0.002477, batch_cost: 0.2119, reader_cost: 0.00082, ips: 14.1559 samples/sec | ETA 01:29:53
- 2022-04-13 17:26:26 [INFO] [TRAIN] epoch: 382, iter: 94600/120000, loss: 0.8333, lr: 0.002472, batch_cost: 0.2157, reader_cost: 0.00072, ips: 13.9070 samples/sec | ETA 01:31:19
- 2022-04-13 17:26:38 [INFO] [TRAIN] epoch: 382, iter: 94650/120000, loss: 0.8327, lr: 0.002468, batch_cost: 0.2234, reader_cost: 0.00107, ips: 13.4317 samples/sec | ETA 01:34:21
- 2022-04-13 17:26:48 [INFO] [TRAIN] epoch: 382, iter: 94700/120000, loss: 0.8409, lr: 0.002464, batch_cost: 0.2144, reader_cost: 0.00066, ips: 13.9910 samples/sec | ETA 01:30:24
- 2022-04-13 17:27:02 [INFO] [TRAIN] epoch: 383, iter: 94750/120000, loss: 0.8419, lr: 0.002459, batch_cost: 0.2784, reader_cost: 0.05941, ips: 10.7764 samples/sec | ETA 01:57:09
- 2022-04-13 17:27:12 [INFO] [TRAIN] epoch: 383, iter: 94800/120000, loss: 0.8639, lr: 0.002455, batch_cost: 0.2013, reader_cost: 0.00086, ips: 14.9033 samples/sec | ETA 01:24:32
- 2022-04-13 17:27:23 [INFO] [TRAIN] epoch: 383, iter: 94850/120000, loss: 0.8151, lr: 0.002450, batch_cost: 0.2226, reader_cost: 0.00061, ips: 13.4753 samples/sec | ETA 01:33:19
- 2022-04-13 17:27:35 [INFO] [TRAIN] epoch: 383, iter: 94900/120000, loss: 0.8253, lr: 0.002446, batch_cost: 0.2261, reader_cost: 0.00102, ips: 13.2664 samples/sec | ETA 01:34:35
- 2022-04-13 17:27:46 [INFO] [TRAIN] epoch: 383, iter: 94950/120000, loss: 0.8407, lr: 0.002442, batch_cost: 0.2269, reader_cost: 0.00197, ips: 13.2199 samples/sec | ETA 01:34:44
- 2022-04-13 17:28:00 [INFO] [TRAIN] epoch: 384, iter: 95000/120000, loss: 0.8651, lr: 0.002437, batch_cost: 0.2788, reader_cost: 0.06342, ips: 10.7591 samples/sec | ETA 01:56:10
- 2022-04-13 17:28:11 [INFO] [TRAIN] epoch: 384, iter: 95050/120000, loss: 0.7998, lr: 0.002433, batch_cost: 0.2170, reader_cost: 0.00168, ips: 13.8237 samples/sec | ETA 01:30:14
- 2022-04-13 17:28:21 [INFO] [TRAIN] epoch: 384, iter: 95100/120000, loss: 0.8613, lr: 0.002428, batch_cost: 0.1962, reader_cost: 0.00059, ips: 15.2881 samples/sec | ETA 01:21:26
- 2022-04-13 17:28:31 [INFO] [TRAIN] epoch: 384, iter: 95150/120000, loss: 0.8368, lr: 0.002424, batch_cost: 0.2038, reader_cost: 0.00115, ips: 14.7209 samples/sec | ETA 01:24:24
- 2022-04-13 17:28:41 [INFO] [TRAIN] epoch: 384, iter: 95200/120000, loss: 0.8286, lr: 0.002420, batch_cost: 0.2068, reader_cost: 0.00116, ips: 14.5072 samples/sec | ETA 01:25:28
- 2022-04-13 17:28:55 [INFO] [TRAIN] epoch: 385, iter: 95250/120000, loss: 0.8307, lr: 0.002415, batch_cost: 0.2786, reader_cost: 0.05644, ips: 10.7671 samples/sec | ETA 01:54:56
- 2022-04-13 17:29:06 [INFO] [TRAIN] epoch: 385, iter: 95300/120000, loss: 0.8498, lr: 0.002411, batch_cost: 0.2091, reader_cost: 0.00075, ips: 14.3443 samples/sec | ETA 01:26:05
- 2022-04-13 17:29:16 [INFO] [TRAIN] epoch: 385, iter: 95350/120000, loss: 0.8350, lr: 0.002407, batch_cost: 0.2155, reader_cost: 0.00111, ips: 13.9201 samples/sec | ETA 01:28:32
- 2022-04-13 17:29:26 [INFO] [TRAIN] epoch: 385, iter: 95400/120000, loss: 0.8155, lr: 0.002402, batch_cost: 0.2026, reader_cost: 0.00147, ips: 14.8052 samples/sec | ETA 01:23:04
- 2022-04-13 17:29:37 [INFO] [TRAIN] epoch: 385, iter: 95450/120000, loss: 0.8580, lr: 0.002398, batch_cost: 0.2190, reader_cost: 0.00077, ips: 13.6987 samples/sec | ETA 01:29:36
- 2022-04-13 17:29:51 [INFO] [TRAIN] epoch: 386, iter: 95500/120000, loss: 0.8523, lr: 0.002393, batch_cost: 0.2786, reader_cost: 0.05201, ips: 10.7671 samples/sec | ETA 01:53:46
- 2022-04-13 17:30:02 [INFO] [TRAIN] epoch: 386, iter: 95550/120000, loss: 0.8410, lr: 0.002389, batch_cost: 0.2135, reader_cost: 0.00104, ips: 14.0529 samples/sec | ETA 01:26:59
- 2022-04-13 17:30:13 [INFO] [TRAIN] epoch: 386, iter: 95600/120000, loss: 0.8164, lr: 0.002385, batch_cost: 0.2199, reader_cost: 0.00057, ips: 13.6404 samples/sec | ETA 01:29:26
- 2022-04-13 17:30:24 [INFO] [TRAIN] epoch: 386, iter: 95650/120000, loss: 0.8205, lr: 0.002380, batch_cost: 0.2134, reader_cost: 0.00091, ips: 14.0601 samples/sec | ETA 01:26:35
- 2022-04-13 17:30:34 [INFO] [TRAIN] epoch: 386, iter: 95700/120000, loss: 0.8442, lr: 0.002376, batch_cost: 0.2155, reader_cost: 0.00104, ips: 13.9201 samples/sec | ETA 01:27:17
- 2022-04-13 17:30:48 [INFO] [TRAIN] epoch: 387, iter: 95750/120000, loss: 0.8421, lr: 0.002371, batch_cost: 0.2699, reader_cost: 0.05550, ips: 11.1164 samples/sec | ETA 01:49:04
- 2022-04-13 17:30:58 [INFO] [TRAIN] epoch: 387, iter: 95800/120000, loss: 0.8076, lr: 0.002367, batch_cost: 0.2059, reader_cost: 0.00125, ips: 14.5714 samples/sec | ETA 01:23:02
- 2022-04-13 17:31:09 [INFO] [TRAIN] epoch: 387, iter: 95850/120000, loss: 0.8322, lr: 0.002363, batch_cost: 0.2091, reader_cost: 0.00088, ips: 14.3448 samples/sec | ETA 01:24:10
- 2022-04-13 17:31:19 [INFO] [TRAIN] epoch: 387, iter: 95900/120000, loss: 0.8196, lr: 0.002358, batch_cost: 0.2068, reader_cost: 0.00131, ips: 14.5088 samples/sec | ETA 01:23:03
- 2022-04-13 17:31:29 [INFO] [TRAIN] epoch: 387, iter: 95950/120000, loss: 0.8214, lr: 0.002354, batch_cost: 0.2077, reader_cost: 0.00107, ips: 14.4455 samples/sec | ETA 01:23:14
- 2022-04-13 17:31:43 [INFO] [TRAIN] epoch: 388, iter: 96000/120000, loss: 0.8112, lr: 0.002349, batch_cost: 0.2774, reader_cost: 0.06305, ips: 10.8128 samples/sec | ETA 01:50:58
- 2022-04-13 17:31:43 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1979 - reader cost: 0.1543
- 2022-04-13 17:32:08 [INFO] [EVAL] #Images: 500 mIoU: 0.7712 Acc: 0.9599 Kappa: 0.9479 Dice: 0.8640
- 2022-04-13 17:32:08 [INFO] [EVAL] Class IoU:
- [0.9809 0.8482 0.9257 0.5293 0.6168 0.645 0.7209 0.7905 0.9211 0.6176
- 0.9493 0.81 0.5852 0.9456 0.6958 0.8682 0.8088 0.6285 0.7656]
- 2022-04-13 17:32:08 [INFO] [EVAL] Class Precision:
- [0.9897 0.9232 0.9599 0.8493 0.7929 0.8374 0.85 0.9268 0.9461 0.8539
- 0.9687 0.898 0.6617 0.9652 0.877 0.9728 0.9429 0.7552 0.8331]
- 2022-04-13 17:32:08 [INFO] [EVAL] Class Recall:
- [0.991 0.9126 0.9629 0.5842 0.7353 0.7374 0.8259 0.8431 0.972 0.6905
- 0.9794 0.8921 0.835 0.9789 0.7711 0.8897 0.8504 0.7894 0.9043]
- 2022-04-13 17:32:09 [INFO] [EVAL] The model with the best validation mIoU (0.7738) was saved at iter 92000.
- 2022-04-13 17:32:19 [INFO] [TRAIN] epoch: 388, iter: 96050/120000, loss: 0.8167, lr: 0.002345, batch_cost: 0.2008, reader_cost: 0.00128, ips: 14.9379 samples/sec | ETA 01:20:09
- 2022-04-13 17:32:29 [INFO] [TRAIN] epoch: 388, iter: 96100/120000, loss: 0.8361, lr: 0.002341, batch_cost: 0.2059, reader_cost: 0.00066, ips: 14.5707 samples/sec | ETA 01:22:00
- 2022-04-13 17:32:40 [INFO] [TRAIN] epoch: 388, iter: 96150/120000, loss: 0.8303, lr: 0.002336, batch_cost: 0.2141, reader_cost: 0.00056, ips: 14.0129 samples/sec | ETA 01:25:06
- 2022-04-13 17:32:50 [INFO] [TRAIN] epoch: 388, iter: 96200/120000, loss: 0.8279, lr: 0.002332, batch_cost: 0.2023, reader_cost: 0.00086, ips: 14.8322 samples/sec | ETA 01:20:13
- 2022-04-13 17:33:04 [INFO] [TRAIN] epoch: 389, iter: 96250/120000, loss: 0.8271, lr: 0.002327, batch_cost: 0.2730, reader_cost: 0.06042, ips: 10.9892 samples/sec | ETA 01:48:03
- 2022-04-13 17:33:14 [INFO] [TRAIN] epoch: 389, iter: 96300/120000, loss: 0.8205, lr: 0.002323, batch_cost: 0.2134, reader_cost: 0.00103, ips: 14.0561 samples/sec | ETA 01:24:18
- 2022-04-13 17:33:25 [INFO] [TRAIN] epoch: 389, iter: 96350/120000, loss: 0.8467, lr: 0.002318, batch_cost: 0.2211, reader_cost: 0.00107, ips: 13.5713 samples/sec | ETA 01:27:07
- 2022-04-13 17:33:36 [INFO] [TRAIN] epoch: 389, iter: 96400/120000, loss: 0.8233, lr: 0.002314, batch_cost: 0.2033, reader_cost: 0.00193, ips: 14.7543 samples/sec | ETA 01:19:58
- 2022-04-13 17:33:46 [INFO] [TRAIN] epoch: 389, iter: 96450/120000, loss: 0.8245, lr: 0.002310, batch_cost: 0.2108, reader_cost: 0.00082, ips: 14.2295 samples/sec | ETA 01:22:45
- 2022-04-13 17:34:00 [INFO] [TRAIN] epoch: 390, iter: 96500/120000, loss: 0.8297, lr: 0.002305, batch_cost: 0.2700, reader_cost: 0.05794, ips: 11.1121 samples/sec | ETA 01:45:44
- 2022-04-13 17:34:10 [INFO] [TRAIN] epoch: 390, iter: 96550/120000, loss: 0.8285, lr: 0.002301, batch_cost: 0.2140, reader_cost: 0.00750, ips: 14.0214 samples/sec | ETA 01:23:37
- 2022-04-13 17:34:21 [INFO] [TRAIN] epoch: 390, iter: 96600/120000, loss: 0.8575, lr: 0.002296, batch_cost: 0.2184, reader_cost: 0.00105, ips: 13.7385 samples/sec | ETA 01:25:09
- 2022-04-13 17:34:31 [INFO] [TRAIN] epoch: 390, iter: 96650/120000, loss: 0.8312, lr: 0.002292, batch_cost: 0.2056, reader_cost: 0.00076, ips: 14.5936 samples/sec | ETA 01:20:00
- 2022-04-13 17:34:42 [INFO] [TRAIN] epoch: 390, iter: 96700/120000, loss: 0.8259, lr: 0.002288, batch_cost: 0.2131, reader_cost: 0.00120, ips: 14.0755 samples/sec | ETA 01:22:46
- 2022-04-13 17:34:55 [INFO] [TRAIN] epoch: 391, iter: 96750/120000, loss: 0.8191, lr: 0.002283, batch_cost: 0.2674, reader_cost: 0.06276, ips: 11.2201 samples/sec | ETA 01:43:36
- 2022-04-13 17:35:06 [INFO] [TRAIN] epoch: 391, iter: 96800/120000, loss: 0.8489, lr: 0.002279, batch_cost: 0.2185, reader_cost: 0.00046, ips: 13.7298 samples/sec | ETA 01:24:29
- 2022-04-13 17:35:17 [INFO] [TRAIN] epoch: 391, iter: 96850/120000, loss: 0.7966, lr: 0.002274, batch_cost: 0.2101, reader_cost: 0.00084, ips: 14.2768 samples/sec | ETA 01:21:04
- 2022-04-13 17:35:29 [INFO] [TRAIN] epoch: 391, iter: 96900/120000, loss: 0.8444, lr: 0.002270, batch_cost: 0.2338, reader_cost: 0.00109, ips: 12.8304 samples/sec | ETA 01:30:01
- 2022-04-13 17:35:39 [INFO] [TRAIN] epoch: 391, iter: 96950/120000, loss: 0.8308, lr: 0.002265, batch_cost: 0.2076, reader_cost: 0.00088, ips: 14.4539 samples/sec | ETA 01:19:44
- 2022-04-13 17:35:53 [INFO] [TRAIN] epoch: 392, iter: 97000/120000, loss: 0.8186, lr: 0.002261, batch_cost: 0.2733, reader_cost: 0.05116, ips: 10.9788 samples/sec | ETA 01:44:44
- 2022-04-13 17:36:03 [INFO] [TRAIN] epoch: 392, iter: 97050/120000, loss: 0.8192, lr: 0.002257, batch_cost: 0.2106, reader_cost: 0.00089, ips: 14.2435 samples/sec | ETA 01:20:33
- 2022-04-13 17:36:14 [INFO] [TRAIN] epoch: 392, iter: 97100/120000, loss: 0.8389, lr: 0.002252, batch_cost: 0.2173, reader_cost: 0.00062, ips: 13.8051 samples/sec | ETA 01:22:56
- 2022-04-13 17:36:24 [INFO] [TRAIN] epoch: 392, iter: 97150/120000, loss: 0.8396, lr: 0.002248, batch_cost: 0.2050, reader_cost: 0.00059, ips: 14.6308 samples/sec | ETA 01:18:05
- 2022-04-13 17:36:37 [INFO] [TRAIN] epoch: 392, iter: 97200/120000, loss: 0.8430, lr: 0.002243, batch_cost: 0.2449, reader_cost: 0.00132, ips: 12.2498 samples/sec | ETA 01:33:03
- 2022-04-13 17:36:50 [INFO] [TRAIN] epoch: 393, iter: 97250/120000, loss: 0.8202, lr: 0.002239, batch_cost: 0.2757, reader_cost: 0.05767, ips: 10.8817 samples/sec | ETA 01:44:31
- 2022-04-13 17:37:02 [INFO] [TRAIN] epoch: 393, iter: 97300/120000, loss: 0.8480, lr: 0.002234, batch_cost: 0.2262, reader_cost: 0.00086, ips: 13.2626 samples/sec | ETA 01:25:34
- 2022-04-13 17:37:12 [INFO] [TRAIN] epoch: 393, iter: 97350/120000, loss: 0.8444, lr: 0.002230, batch_cost: 0.2047, reader_cost: 0.00101, ips: 14.6532 samples/sec | ETA 01:17:17
- 2022-04-13 17:37:22 [INFO] [TRAIN] epoch: 393, iter: 97400/120000, loss: 0.8166, lr: 0.002226, batch_cost: 0.2024, reader_cost: 0.00098, ips: 14.8254 samples/sec | ETA 01:16:13
- 2022-04-13 17:37:33 [INFO] [TRAIN] epoch: 393, iter: 97450/120000, loss: 0.8239, lr: 0.002221, batch_cost: 0.2219, reader_cost: 0.00092, ips: 13.5214 samples/sec | ETA 01:23:23
- 2022-04-13 17:37:47 [INFO] [TRAIN] epoch: 394, iter: 97500/120000, loss: 0.8124, lr: 0.002217, batch_cost: 0.2781, reader_cost: 0.04551, ips: 10.7862 samples/sec | ETA 01:44:17
- 2022-04-13 17:37:58 [INFO] [TRAIN] epoch: 394, iter: 97550/120000, loss: 0.8322, lr: 0.002212, batch_cost: 0.2139, reader_cost: 0.00111, ips: 14.0272 samples/sec | ETA 01:20:01
- 2022-04-13 17:38:08 [INFO] [TRAIN] epoch: 394, iter: 97600/120000, loss: 0.8217, lr: 0.002208, batch_cost: 0.2104, reader_cost: 0.00091, ips: 14.2596 samples/sec | ETA 01:18:32
- 2022-04-13 17:38:18 [INFO] [TRAIN] epoch: 394, iter: 97650/120000, loss: 0.8239, lr: 0.002203, batch_cost: 0.2036, reader_cost: 0.00044, ips: 14.7346 samples/sec | ETA 01:15:50
- 2022-04-13 17:38:28 [INFO] [TRAIN] epoch: 394, iter: 97700/120000, loss: 0.8277, lr: 0.002199, batch_cost: 0.1990, reader_cost: 0.00143, ips: 15.0748 samples/sec | ETA 01:13:57
- 2022-04-13 17:38:42 [INFO] [TRAIN] epoch: 395, iter: 97750/120000, loss: 0.8093, lr: 0.002195, batch_cost: 0.2784, reader_cost: 0.05951, ips: 10.7741 samples/sec | ETA 01:43:15
- 2022-04-13 17:38:53 [INFO] [TRAIN] epoch: 395, iter: 97800/120000, loss: 0.8209, lr: 0.002190, batch_cost: 0.2092, reader_cost: 0.00093, ips: 14.3372 samples/sec | ETA 01:17:25
- 2022-04-13 17:39:03 [INFO] [TRAIN] epoch: 395, iter: 97850/120000, loss: 0.8217, lr: 0.002186, batch_cost: 0.2020, reader_cost: 0.00033, ips: 14.8547 samples/sec | ETA 01:14:33
- 2022-04-13 17:39:14 [INFO] [TRAIN] epoch: 395, iter: 97900/120000, loss: 0.8117, lr: 0.002181, batch_cost: 0.2140, reader_cost: 0.00102, ips: 14.0187 samples/sec | ETA 01:18:49
- 2022-04-13 17:39:24 [INFO] [TRAIN] epoch: 395, iter: 97950/120000, loss: 0.8152, lr: 0.002177, batch_cost: 0.2186, reader_cost: 0.00045, ips: 13.7268 samples/sec | ETA 01:20:19
- 2022-04-13 17:39:38 [INFO] [TRAIN] epoch: 396, iter: 98000/120000, loss: 0.8164, lr: 0.002172, batch_cost: 0.2631, reader_cost: 0.06130, ips: 11.4028 samples/sec | ETA 01:36:28
- 2022-04-13 17:39:48 [INFO] [TRAIN] epoch: 396, iter: 98050/120000, loss: 0.8191, lr: 0.002168, batch_cost: 0.1989, reader_cost: 0.00122, ips: 15.0862 samples/sec | ETA 01:12:44
- 2022-04-13 17:39:58 [INFO] [TRAIN] epoch: 396, iter: 98100/120000, loss: 0.8573, lr: 0.002163, batch_cost: 0.2109, reader_cost: 0.00111, ips: 14.2279 samples/sec | ETA 01:16:57
- 2022-04-13 17:40:09 [INFO] [TRAIN] epoch: 396, iter: 98150/120000, loss: 0.8577, lr: 0.002159, batch_cost: 0.2231, reader_cost: 0.00099, ips: 13.4493 samples/sec | ETA 01:21:13
- 2022-04-13 17:40:20 [INFO] [TRAIN] epoch: 396, iter: 98200/120000, loss: 0.8475, lr: 0.002155, batch_cost: 0.2045, reader_cost: 0.00130, ips: 14.6681 samples/sec | ETA 01:14:18
- 2022-04-13 17:40:33 [INFO] [TRAIN] epoch: 397, iter: 98250/120000, loss: 0.8416, lr: 0.002150, batch_cost: 0.2708, reader_cost: 0.06289, ips: 11.0780 samples/sec | ETA 01:38:10
- 2022-04-13 17:40:43 [INFO] [TRAIN] epoch: 397, iter: 98300/120000, loss: 0.8269, lr: 0.002146, batch_cost: 0.2067, reader_cost: 0.00130, ips: 14.5107 samples/sec | ETA 01:14:46
- 2022-04-13 17:40:54 [INFO] [TRAIN] epoch: 397, iter: 98350/120000, loss: 0.8322, lr: 0.002141, batch_cost: 0.2183, reader_cost: 0.00150, ips: 13.7395 samples/sec | ETA 01:18:47
- 2022-04-13 17:41:04 [INFO] [TRAIN] epoch: 397, iter: 98400/120000, loss: 0.8171, lr: 0.002137, batch_cost: 0.2014, reader_cost: 0.00057, ips: 14.8926 samples/sec | ETA 01:12:31
- 2022-04-13 17:41:15 [INFO] [TRAIN] epoch: 397, iter: 98450/120000, loss: 0.8420, lr: 0.002132, batch_cost: 0.2078, reader_cost: 0.00064, ips: 14.4349 samples/sec | ETA 01:14:38
- 2022-04-13 17:41:28 [INFO] [TRAIN] epoch: 398, iter: 98500/120000, loss: 0.8383, lr: 0.002128, batch_cost: 0.2734, reader_cost: 0.05843, ips: 10.9717 samples/sec | ETA 01:37:58
- 2022-04-13 17:41:41 [INFO] [TRAIN] epoch: 398, iter: 98550/120000, loss: 0.8266, lr: 0.002123, batch_cost: 0.2557, reader_cost: 0.00083, ips: 11.7310 samples/sec | ETA 01:31:25
- 2022-04-13 17:41:51 [INFO] [TRAIN] epoch: 398, iter: 98600/120000, loss: 0.8154, lr: 0.002119, batch_cost: 0.2022, reader_cost: 0.00101, ips: 14.8382 samples/sec | ETA 01:12:06
- 2022-04-13 17:42:02 [INFO] [TRAIN] epoch: 398, iter: 98650/120000, loss: 0.8222, lr: 0.002115, batch_cost: 0.2158, reader_cost: 0.00098, ips: 13.9014 samples/sec | ETA 01:16:47
- 2022-04-13 17:42:12 [INFO] [TRAIN] epoch: 398, iter: 98700/120000, loss: 0.8235, lr: 0.002110, batch_cost: 0.1981, reader_cost: 0.00130, ips: 15.1457 samples/sec | ETA 01:10:19
- 2022-04-13 17:42:26 [INFO] [TRAIN] epoch: 399, iter: 98750/120000, loss: 0.8308, lr: 0.002106, batch_cost: 0.2788, reader_cost: 0.05546, ips: 10.7615 samples/sec | ETA 01:38:43
- 2022-04-13 17:42:36 [INFO] [TRAIN] epoch: 399, iter: 98800/120000, loss: 0.8177, lr: 0.002101, batch_cost: 0.2023, reader_cost: 0.00119, ips: 14.8280 samples/sec | ETA 01:11:29
- 2022-04-13 17:42:46 [INFO] [TRAIN] epoch: 399, iter: 98850/120000, loss: 0.8789, lr: 0.002097, batch_cost: 0.1986, reader_cost: 0.00089, ips: 15.1048 samples/sec | ETA 01:10:00
- 2022-04-13 17:42:57 [INFO] [TRAIN] epoch: 399, iter: 98900/120000, loss: 0.8053, lr: 0.002092, batch_cost: 0.2125, reader_cost: 0.00160, ips: 14.1195 samples/sec | ETA 01:14:43
- 2022-04-13 17:43:07 [INFO] [TRAIN] epoch: 399, iter: 98950/120000, loss: 0.8103, lr: 0.002088, batch_cost: 0.1992, reader_cost: 0.00100, ips: 15.0626 samples/sec | ETA 01:09:52
- 2022-04-13 17:43:21 [INFO] [TRAIN] epoch: 400, iter: 99000/120000, loss: 0.8220, lr: 0.002083, batch_cost: 0.2952, reader_cost: 0.05800, ips: 10.1634 samples/sec | ETA 01:43:18
- 2022-04-13 17:43:32 [INFO] [TRAIN] epoch: 400, iter: 99050/120000, loss: 0.8086, lr: 0.002079, batch_cost: 0.2213, reader_cost: 0.00091, ips: 13.5570 samples/sec | ETA 01:17:15
- 2022-04-13 17:43:43 [INFO] [TRAIN] epoch: 400, iter: 99100/120000, loss: 0.8216, lr: 0.002074, batch_cost: 0.2160, reader_cost: 0.00102, ips: 13.8910 samples/sec | ETA 01:15:13
- 2022-04-13 17:43:55 [INFO] [TRAIN] epoch: 400, iter: 99150/120000, loss: 0.8285, lr: 0.002070, batch_cost: 0.2356, reader_cost: 0.00116, ips: 12.7335 samples/sec | ETA 01:21:52
- 2022-04-13 17:44:06 [INFO] [TRAIN] epoch: 400, iter: 99200/120000, loss: 0.8284, lr: 0.002065, batch_cost: 0.2112, reader_cost: 0.00102, ips: 14.2036 samples/sec | ETA 01:13:13
- 2022-04-13 17:44:20 [INFO] [TRAIN] epoch: 401, iter: 99250/120000, loss: 0.8120, lr: 0.002061, batch_cost: 0.2802, reader_cost: 0.05583, ips: 10.7072 samples/sec | ETA 01:36:53
- 2022-04-13 17:44:30 [INFO] [TRAIN] epoch: 401, iter: 99300/120000, loss: 0.8165, lr: 0.002057, batch_cost: 0.2155, reader_cost: 0.00115, ips: 13.9187 samples/sec | ETA 01:14:21
- 2022-04-13 17:44:41 [INFO] [TRAIN] epoch: 401, iter: 99350/120000, loss: 0.8073, lr: 0.002052, batch_cost: 0.2179, reader_cost: 0.00060, ips: 13.7653 samples/sec | ETA 01:15:00
- 2022-04-13 17:44:53 [INFO] [TRAIN] epoch: 401, iter: 99400/120000, loss: 0.8289, lr: 0.002048, batch_cost: 0.2244, reader_cost: 0.00072, ips: 13.3699 samples/sec | ETA 01:17:02
- 2022-04-13 17:45:05 [INFO] [TRAIN] epoch: 402, iter: 99450/120000, loss: 0.8403, lr: 0.002043, batch_cost: 0.2549, reader_cost: 0.05603, ips: 11.7703 samples/sec | ETA 01:27:17
- 2022-04-13 17:45:16 [INFO] [TRAIN] epoch: 402, iter: 99500/120000, loss: 0.8147, lr: 0.002039, batch_cost: 0.2175, reader_cost: 0.00141, ips: 13.7930 samples/sec | ETA 01:14:18
- 2022-04-13 17:45:27 [INFO] [TRAIN] epoch: 402, iter: 99550/120000, loss: 0.8256, lr: 0.002034, batch_cost: 0.2157, reader_cost: 0.00088, ips: 13.9105 samples/sec | ETA 01:13:30
- 2022-04-13 17:45:38 [INFO] [TRAIN] epoch: 402, iter: 99600/120000, loss: 0.8163, lr: 0.002030, batch_cost: 0.2269, reader_cost: 0.00070, ips: 13.2196 samples/sec | ETA 01:17:09
- 2022-04-13 17:45:48 [INFO] [TRAIN] epoch: 402, iter: 99650/120000, loss: 0.8182, lr: 0.002025, batch_cost: 0.1985, reader_cost: 0.00073, ips: 15.1169 samples/sec | ETA 01:07:18
- 2022-04-13 17:46:02 [INFO] [TRAIN] epoch: 403, iter: 99700/120000, loss: 0.8157, lr: 0.002021, batch_cost: 0.2766, reader_cost: 0.05245, ips: 10.8478 samples/sec | ETA 01:33:34
- 2022-04-13 17:46:12 [INFO] [TRAIN] epoch: 403, iter: 99750/120000, loss: 0.8184, lr: 0.002016, batch_cost: 0.2074, reader_cost: 0.00110, ips: 14.4624 samples/sec | ETA 01:10:00
- 2022-04-13 17:46:22 [INFO] [TRAIN] epoch: 403, iter: 99800/120000, loss: 0.8196, lr: 0.002012, batch_cost: 0.1955, reader_cost: 0.00103, ips: 15.3468 samples/sec | ETA 01:05:48
- 2022-04-13 17:46:33 [INFO] [TRAIN] epoch: 403, iter: 99850/120000, loss: 0.8754, lr: 0.002007, batch_cost: 0.2113, reader_cost: 0.00077, ips: 14.1979 samples/sec | ETA 01:10:57
- 2022-04-13 17:46:44 [INFO] [TRAIN] epoch: 403, iter: 99900/120000, loss: 0.8379, lr: 0.002003, batch_cost: 0.2315, reader_cost: 0.00091, ips: 12.9604 samples/sec | ETA 01:17:32
- 2022-04-13 17:46:58 [INFO] [TRAIN] epoch: 404, iter: 99950/120000, loss: 0.8193, lr: 0.001998, batch_cost: 0.2821, reader_cost: 0.06557, ips: 10.6340 samples/sec | ETA 01:34:16
- 2022-04-13 17:47:10 [INFO] [TRAIN] epoch: 404, iter: 100000/120000, loss: 0.8195, lr: 0.001994, batch_cost: 0.2367, reader_cost: 0.00110, ips: 12.6760 samples/sec | ETA 01:18:53
- 2022-04-13 17:47:10 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.2000 - reader cost: 0.1568
- 2022-04-13 17:47:35 [INFO] [EVAL] #Images: 500 mIoU: 0.7789 Acc: 0.9614 Kappa: 0.9499 Dice: 0.8693
- 2022-04-13 17:47:35 [INFO] [EVAL] Class IoU:
- [0.9825 0.8548 0.9268 0.522 0.6187 0.6522 0.7246 0.8019 0.9237 0.6421
- 0.9505 0.7951 0.645 0.9517 0.8128 0.8724 0.7135 0.6284 0.7803]
- 2022-04-13 17:47:35 [INFO] [EVAL] Class Precision:
- [0.9906 0.9248 0.9614 0.8421 0.7965 0.8224 0.8286 0.8963 0.948 0.8688
- 0.975 0.8591 0.78 0.9745 0.9218 0.9343 0.9749 0.7731 0.8614]
- 2022-04-13 17:47:35 [INFO] [EVAL] Class Recall:
- [0.9917 0.9187 0.9626 0.5786 0.7349 0.759 0.8524 0.884 0.973 0.711
- 0.9741 0.9143 0.7884 0.9759 0.8731 0.9294 0.7269 0.7706 0.8923]
- 2022-04-13 17:47:36 [INFO] [EVAL] The model with the best validation mIoU (0.7789) was saved at iter 100000.
- 2022-04-13 17:47:47 [INFO] [TRAIN] epoch: 404, iter: 100050/120000, loss: 0.8271, lr: 0.001989, batch_cost: 0.2069, reader_cost: 0.00156, ips: 14.5029 samples/sec | ETA 01:08:46
- 2022-04-13 17:47:58 [INFO] [TRAIN] epoch: 404, iter: 100100/120000, loss: 0.8263, lr: 0.001985, batch_cost: 0.2166, reader_cost: 0.00118, ips: 13.8509 samples/sec | ETA 01:11:50
- 2022-04-13 17:48:08 [INFO] [TRAIN] epoch: 404, iter: 100150/120000, loss: 0.8234, lr: 0.001980, batch_cost: 0.2182, reader_cost: 0.00078, ips: 13.7511 samples/sec | ETA 01:12:10
- 2022-04-13 17:48:22 [INFO] [TRAIN] epoch: 405, iter: 100200/120000, loss: 0.8150, lr: 0.001976, batch_cost: 0.2662, reader_cost: 0.05953, ips: 11.2691 samples/sec | ETA 01:27:51
- 2022-04-13 17:48:32 [INFO] [TRAIN] epoch: 405, iter: 100250/120000, loss: 0.8287, lr: 0.001971, batch_cost: 0.2075, reader_cost: 0.00150, ips: 14.4605 samples/sec | ETA 01:08:17
- 2022-04-13 17:48:42 [INFO] [TRAIN] epoch: 405, iter: 100300/120000, loss: 0.8245, lr: 0.001967, batch_cost: 0.2053, reader_cost: 0.00138, ips: 14.6099 samples/sec | ETA 01:07:25
- 2022-04-13 17:48:53 [INFO] [TRAIN] epoch: 405, iter: 100350/120000, loss: 0.8226, lr: 0.001962, batch_cost: 0.2095, reader_cost: 0.00078, ips: 14.3197 samples/sec | ETA 01:08:36
- 2022-04-13 17:49:05 [INFO] [TRAIN] epoch: 405, iter: 100400/120000, loss: 0.8209, lr: 0.001958, batch_cost: 0.2396, reader_cost: 0.00095, ips: 12.5220 samples/sec | ETA 01:18:15
- 2022-04-13 17:49:18 [INFO] [TRAIN] epoch: 406, iter: 100450/120000, loss: 0.8086, lr: 0.001953, batch_cost: 0.2664, reader_cost: 0.05816, ips: 11.2626 samples/sec | ETA 01:26:47
- 2022-04-13 17:49:29 [INFO] [TRAIN] epoch: 406, iter: 100500/120000, loss: 0.8318, lr: 0.001949, batch_cost: 0.2092, reader_cost: 0.00058, ips: 14.3405 samples/sec | ETA 01:07:59
- 2022-04-13 17:49:39 [INFO] [TRAIN] epoch: 406, iter: 100550/120000, loss: 0.8309, lr: 0.001944, batch_cost: 0.2101, reader_cost: 0.00074, ips: 14.2820 samples/sec | ETA 01:08:05
- 2022-04-13 17:49:49 [INFO] [TRAIN] epoch: 406, iter: 100600/120000, loss: 0.8199, lr: 0.001940, batch_cost: 0.2067, reader_cost: 0.00077, ips: 14.5157 samples/sec | ETA 01:06:49
- 2022-04-13 17:50:00 [INFO] [TRAIN] epoch: 406, iter: 100650/120000, loss: 0.8020, lr: 0.001935, batch_cost: 0.2179, reader_cost: 0.00092, ips: 13.7647 samples/sec | ETA 01:10:17
- 2022-04-13 17:50:14 [INFO] [TRAIN] epoch: 407, iter: 100700/120000, loss: 0.8121, lr: 0.001931, batch_cost: 0.2784, reader_cost: 0.05757, ips: 10.7741 samples/sec | ETA 01:29:33
- 2022-04-13 17:50:25 [INFO] [TRAIN] epoch: 407, iter: 100750/120000, loss: 0.8296, lr: 0.001926, batch_cost: 0.2182, reader_cost: 0.00134, ips: 13.7513 samples/sec | ETA 01:09:59
- 2022-04-13 17:50:35 [INFO] [TRAIN] epoch: 407, iter: 100800/120000, loss: 0.8064, lr: 0.001922, batch_cost: 0.2023, reader_cost: 0.00146, ips: 14.8288 samples/sec | ETA 01:04:44
- 2022-04-13 17:50:46 [INFO] [TRAIN] epoch: 407, iter: 100850/120000, loss: 0.8157, lr: 0.001917, batch_cost: 0.2144, reader_cost: 0.00033, ips: 13.9894 samples/sec | ETA 01:08:26
- 2022-04-13 17:50:57 [INFO] [TRAIN] epoch: 407, iter: 100900/120000, loss: 0.8550, lr: 0.001913, batch_cost: 0.2264, reader_cost: 0.00117, ips: 13.2496 samples/sec | ETA 01:12:04
- 2022-04-13 17:51:11 [INFO] [TRAIN] epoch: 408, iter: 100950/120000, loss: 0.8326, lr: 0.001908, batch_cost: 0.2822, reader_cost: 0.05878, ips: 10.6326 samples/sec | ETA 01:29:34
- 2022-04-13 17:51:22 [INFO] [TRAIN] epoch: 408, iter: 101000/120000, loss: 0.8237, lr: 0.001904, batch_cost: 0.2043, reader_cost: 0.00067, ips: 14.6827 samples/sec | ETA 01:04:42
- 2022-04-13 17:51:33 [INFO] [TRAIN] epoch: 408, iter: 101050/120000, loss: 0.8364, lr: 0.001899, batch_cost: 0.2240, reader_cost: 0.00162, ips: 13.3915 samples/sec | ETA 01:10:45
- 2022-04-13 17:51:43 [INFO] [TRAIN] epoch: 408, iter: 101100/120000, loss: 0.8654, lr: 0.001895, batch_cost: 0.2009, reader_cost: 0.00043, ips: 14.9352 samples/sec | ETA 01:03:16
- 2022-04-13 17:51:54 [INFO] [TRAIN] epoch: 408, iter: 101150/120000, loss: 0.8038, lr: 0.001890, batch_cost: 0.2136, reader_cost: 0.00059, ips: 14.0478 samples/sec | ETA 01:07:05
- 2022-04-13 17:52:07 [INFO] [TRAIN] epoch: 409, iter: 101200/120000, loss: 0.8254, lr: 0.001886, batch_cost: 0.2702, reader_cost: 0.04446, ips: 11.1018 samples/sec | ETA 01:24:40
- 2022-04-13 17:52:17 [INFO] [TRAIN] epoch: 409, iter: 101250/120000, loss: 0.8233, lr: 0.001881, batch_cost: 0.2046, reader_cost: 0.00085, ips: 14.6641 samples/sec | ETA 01:03:55
- 2022-04-13 17:52:27 [INFO] [TRAIN] epoch: 409, iter: 101300/120000, loss: 0.8127, lr: 0.001877, batch_cost: 0.1959, reader_cost: 0.00095, ips: 15.3143 samples/sec | ETA 01:01:03
- 2022-04-13 17:52:37 [INFO] [TRAIN] epoch: 409, iter: 101350/120000, loss: 0.8604, lr: 0.001872, batch_cost: 0.2001, reader_cost: 0.00142, ips: 14.9944 samples/sec | ETA 01:02:11
- 2022-04-13 17:52:48 [INFO] [TRAIN] epoch: 409, iter: 101400/120000, loss: 0.8197, lr: 0.001868, batch_cost: 0.2195, reader_cost: 0.00100, ips: 13.6688 samples/sec | ETA 01:08:02
- 2022-04-13 17:53:02 [INFO] [TRAIN] epoch: 410, iter: 101450/120000, loss: 0.8257, lr: 0.001863, batch_cost: 0.2758, reader_cost: 0.05869, ips: 10.8785 samples/sec | ETA 01:25:15
- 2022-04-13 17:53:12 [INFO] [TRAIN] epoch: 410, iter: 101500/120000, loss: 0.8183, lr: 0.001859, batch_cost: 0.2095, reader_cost: 0.00130, ips: 14.3192 samples/sec | ETA 01:04:35
- 2022-04-13 17:53:22 [INFO] [TRAIN] epoch: 410, iter: 101550/120000, loss: 0.8042, lr: 0.001854, batch_cost: 0.2004, reader_cost: 0.00075, ips: 14.9705 samples/sec | ETA 01:01:37
- 2022-04-13 17:53:33 [INFO] [TRAIN] epoch: 410, iter: 101600/120000, loss: 0.8202, lr: 0.001850, batch_cost: 0.2061, reader_cost: 0.00138, ips: 14.5564 samples/sec | ETA 01:03:12
- 2022-04-13 17:53:43 [INFO] [TRAIN] epoch: 410, iter: 101650/120000, loss: 0.8215, lr: 0.001845, batch_cost: 0.2010, reader_cost: 0.00070, ips: 14.9272 samples/sec | ETA 01:01:27
- 2022-04-13 17:53:57 [INFO] [TRAIN] epoch: 411, iter: 101700/120000, loss: 0.8152, lr: 0.001841, batch_cost: 0.2756, reader_cost: 0.05625, ips: 10.8852 samples/sec | ETA 01:24:03
- 2022-04-13 17:54:07 [INFO] [TRAIN] epoch: 411, iter: 101750/120000, loss: 0.8164, lr: 0.001836, batch_cost: 0.2031, reader_cost: 0.00111, ips: 14.7731 samples/sec | ETA 01:01:46
- 2022-04-13 17:54:17 [INFO] [TRAIN] epoch: 411, iter: 101800/120000, loss: 0.8185, lr: 0.001832, batch_cost: 0.2148, reader_cost: 0.00073, ips: 13.9676 samples/sec | ETA 01:05:09
- 2022-04-13 17:54:30 [INFO] [TRAIN] epoch: 411, iter: 101850/120000, loss: 0.8027, lr: 0.001827, batch_cost: 0.2420, reader_cost: 0.00089, ips: 12.3967 samples/sec | ETA 01:13:12
- 2022-04-13 17:54:41 [INFO] [TRAIN] epoch: 411, iter: 101900/120000, loss: 0.7909, lr: 0.001823, batch_cost: 0.2264, reader_cost: 0.00110, ips: 13.2525 samples/sec | ETA 01:08:17
- 2022-04-13 17:54:54 [INFO] [TRAIN] epoch: 412, iter: 101950/120000, loss: 0.8232, lr: 0.001818, batch_cost: 0.2669, reader_cost: 0.06045, ips: 11.2408 samples/sec | ETA 01:20:17
- 2022-04-13 17:55:06 [INFO] [TRAIN] epoch: 412, iter: 102000/120000, loss: 0.8216, lr: 0.001813, batch_cost: 0.2397, reader_cost: 0.00078, ips: 12.5179 samples/sec | ETA 01:11:53
- 2022-04-13 17:55:17 [INFO] [TRAIN] epoch: 412, iter: 102050/120000, loss: 0.8244, lr: 0.001809, batch_cost: 0.2234, reader_cost: 0.00096, ips: 13.4274 samples/sec | ETA 01:06:50
- 2022-04-13 17:55:28 [INFO] [TRAIN] epoch: 412, iter: 102100/120000, loss: 0.8035, lr: 0.001804, batch_cost: 0.2106, reader_cost: 0.00044, ips: 14.2418 samples/sec | ETA 01:02:50
- 2022-04-13 17:55:39 [INFO] [TRAIN] epoch: 412, iter: 102150/120000, loss: 0.8190, lr: 0.001800, batch_cost: 0.2201, reader_cost: 0.00057, ips: 13.6308 samples/sec | ETA 01:05:28
- 2022-04-13 17:55:52 [INFO] [TRAIN] epoch: 413, iter: 102200/120000, loss: 0.8101, lr: 0.001795, batch_cost: 0.2679, reader_cost: 0.06094, ips: 11.1967 samples/sec | ETA 01:19:29
- 2022-04-13 17:56:02 [INFO] [TRAIN] epoch: 413, iter: 102250/120000, loss: 0.8287, lr: 0.001791, batch_cost: 0.2027, reader_cost: 0.00114, ips: 14.7994 samples/sec | ETA 00:59:58
- 2022-04-13 17:56:13 [INFO] [TRAIN] epoch: 413, iter: 102300/120000, loss: 0.8280, lr: 0.001786, batch_cost: 0.2152, reader_cost: 0.00106, ips: 13.9381 samples/sec | ETA 01:03:29
- 2022-04-13 17:56:23 [INFO] [TRAIN] epoch: 413, iter: 102350/120000, loss: 0.8112, lr: 0.001782, batch_cost: 0.2042, reader_cost: 0.00067, ips: 14.6908 samples/sec | ETA 01:00:04
- 2022-04-13 17:56:34 [INFO] [TRAIN] epoch: 413, iter: 102400/120000, loss: 0.8124, lr: 0.001777, batch_cost: 0.2117, reader_cost: 0.00134, ips: 14.1710 samples/sec | ETA 01:02:05
- 2022-04-13 17:56:47 [INFO] [TRAIN] epoch: 414, iter: 102450/120000, loss: 0.7981, lr: 0.001773, batch_cost: 0.2680, reader_cost: 0.05868, ips: 11.1927 samples/sec | ETA 01:18:23
- 2022-04-13 17:56:58 [INFO] [TRAIN] epoch: 414, iter: 102500/120000, loss: 0.8188, lr: 0.001768, batch_cost: 0.2110, reader_cost: 0.00079, ips: 14.2191 samples/sec | ETA 01:01:32
- 2022-04-13 17:57:08 [INFO] [TRAIN] epoch: 414, iter: 102550/120000, loss: 0.8158, lr: 0.001763, batch_cost: 0.2086, reader_cost: 0.00081, ips: 14.3801 samples/sec | ETA 01:00:40
- 2022-04-13 17:57:20 [INFO] [TRAIN] epoch: 414, iter: 102600/120000, loss: 0.8135, lr: 0.001759, batch_cost: 0.2322, reader_cost: 0.00122, ips: 12.9188 samples/sec | ETA 01:07:20
- 2022-04-13 17:57:31 [INFO] [TRAIN] epoch: 414, iter: 102650/120000, loss: 0.8022, lr: 0.001754, batch_cost: 0.2170, reader_cost: 0.00086, ips: 13.8218 samples/sec | ETA 01:02:45
- 2022-04-13 17:57:45 [INFO] [TRAIN] epoch: 415, iter: 102700/120000, loss: 0.8289, lr: 0.001750, batch_cost: 0.2890, reader_cost: 0.05326, ips: 10.3796 samples/sec | ETA 01:23:20
- 2022-04-13 17:57:56 [INFO] [TRAIN] epoch: 415, iter: 102750/120000, loss: 0.8069, lr: 0.001745, batch_cost: 0.2069, reader_cost: 0.00101, ips: 14.4964 samples/sec | ETA 00:59:29
- 2022-04-13 17:58:06 [INFO] [TRAIN] epoch: 415, iter: 102800/120000, loss: 0.8199, lr: 0.001741, batch_cost: 0.2069, reader_cost: 0.00146, ips: 14.4978 samples/sec | ETA 00:59:19
- 2022-04-13 17:58:17 [INFO] [TRAIN] epoch: 415, iter: 102850/120000, loss: 0.8041, lr: 0.001736, batch_cost: 0.2280, reader_cost: 0.00097, ips: 13.1603 samples/sec | ETA 01:05:09
- 2022-04-13 17:58:28 [INFO] [TRAIN] epoch: 415, iter: 102900/120000, loss: 0.8044, lr: 0.001732, batch_cost: 0.2070, reader_cost: 0.00045, ips: 14.4916 samples/sec | ETA 00:58:59
- 2022-04-13 17:58:41 [INFO] [TRAIN] epoch: 416, iter: 102950/120000, loss: 0.8300, lr: 0.001727, batch_cost: 0.2675, reader_cost: 0.06443, ips: 11.2149 samples/sec | ETA 01:16:00
- 2022-04-13 17:58:53 [INFO] [TRAIN] epoch: 416, iter: 103000/120000, loss: 0.8365, lr: 0.001723, batch_cost: 0.2263, reader_cost: 0.00113, ips: 13.2549 samples/sec | ETA 01:04:07
- 2022-04-13 17:59:03 [INFO] [TRAIN] epoch: 416, iter: 103050/120000, loss: 0.8130, lr: 0.001718, batch_cost: 0.2029, reader_cost: 0.00055, ips: 14.7841 samples/sec | ETA 00:57:19
- 2022-04-13 17:59:13 [INFO] [TRAIN] epoch: 416, iter: 103100/120000, loss: 0.7989, lr: 0.001713, batch_cost: 0.2111, reader_cost: 0.00049, ips: 14.2132 samples/sec | ETA 00:59:27
- 2022-04-13 17:59:24 [INFO] [TRAIN] epoch: 416, iter: 103150/120000, loss: 0.8370, lr: 0.001709, batch_cost: 0.2116, reader_cost: 0.00058, ips: 14.1802 samples/sec | ETA 00:59:24
- 2022-04-13 17:59:37 [INFO] [TRAIN] epoch: 417, iter: 103200/120000, loss: 0.8281, lr: 0.001704, batch_cost: 0.2630, reader_cost: 0.05342, ips: 11.4070 samples/sec | ETA 01:13:38
- 2022-04-13 17:59:48 [INFO] [TRAIN] epoch: 417, iter: 103250/120000, loss: 0.8098, lr: 0.001700, batch_cost: 0.2129, reader_cost: 0.00074, ips: 14.0942 samples/sec | ETA 00:59:25
- 2022-04-13 17:59:58 [INFO] [TRAIN] epoch: 417, iter: 103300/120000, loss: 0.8258, lr: 0.001695, batch_cost: 0.2149, reader_cost: 0.00074, ips: 13.9623 samples/sec | ETA 00:59:48
- 2022-04-13 18:00:09 [INFO] [TRAIN] epoch: 417, iter: 103350/120000, loss: 0.8055, lr: 0.001691, batch_cost: 0.2063, reader_cost: 0.00059, ips: 14.5435 samples/sec | ETA 00:57:14
- 2022-04-13 18:00:19 [INFO] [TRAIN] epoch: 417, iter: 103400/120000, loss: 0.8182, lr: 0.001686, batch_cost: 0.2134, reader_cost: 0.00130, ips: 14.0561 samples/sec | ETA 00:59:02
- 2022-04-13 18:00:32 [INFO] [TRAIN] epoch: 418, iter: 103450/120000, loss: 0.8094, lr: 0.001681, batch_cost: 0.2635, reader_cost: 0.05816, ips: 11.3850 samples/sec | ETA 01:12:40
- 2022-04-13 18:00:43 [INFO] [TRAIN] epoch: 418, iter: 103500/120000, loss: 0.7957, lr: 0.001677, batch_cost: 0.2119, reader_cost: 0.00148, ips: 14.1590 samples/sec | ETA 00:58:16
- 2022-04-13 18:00:54 [INFO] [TRAIN] epoch: 418, iter: 103550/120000, loss: 0.8226, lr: 0.001672, batch_cost: 0.2083, reader_cost: 0.00098, ips: 14.4034 samples/sec | ETA 00:57:06
- 2022-04-13 18:01:04 [INFO] [TRAIN] epoch: 418, iter: 103600/120000, loss: 0.8070, lr: 0.001668, batch_cost: 0.2069, reader_cost: 0.00090, ips: 14.4972 samples/sec | ETA 00:56:33
- 2022-04-13 18:01:15 [INFO] [TRAIN] epoch: 418, iter: 103650/120000, loss: 0.8116, lr: 0.001663, batch_cost: 0.2179, reader_cost: 0.00173, ips: 13.7695 samples/sec | ETA 00:59:22
- 2022-04-13 18:01:28 [INFO] [TRAIN] epoch: 419, iter: 103700/120000, loss: 0.8098, lr: 0.001659, batch_cost: 0.2700, reader_cost: 0.05311, ips: 11.1116 samples/sec | ETA 01:13:20
- 2022-04-13 18:01:39 [INFO] [TRAIN] epoch: 419, iter: 103750/120000, loss: 0.7964, lr: 0.001654, batch_cost: 0.2100, reader_cost: 0.00122, ips: 14.2885 samples/sec | ETA 00:56:51
- 2022-04-13 18:01:49 [INFO] [TRAIN] epoch: 419, iter: 103800/120000, loss: 0.8238, lr: 0.001649, batch_cost: 0.2067, reader_cost: 0.00145, ips: 14.5119 samples/sec | ETA 00:55:48
- 2022-04-13 18:02:00 [INFO] [TRAIN] epoch: 419, iter: 103850/120000, loss: 0.8097, lr: 0.001645, batch_cost: 0.2171, reader_cost: 0.00101, ips: 13.8186 samples/sec | ETA 00:58:26
- 2022-04-13 18:02:11 [INFO] [TRAIN] epoch: 419, iter: 103900/120000, loss: 0.8185, lr: 0.001640, batch_cost: 0.2164, reader_cost: 0.00062, ips: 13.8651 samples/sec | ETA 00:58:03
- 2022-04-13 18:02:24 [INFO] [TRAIN] epoch: 420, iter: 103950/120000, loss: 0.8133, lr: 0.001636, batch_cost: 0.2691, reader_cost: 0.05531, ips: 11.1482 samples/sec | ETA 01:11:59
- 2022-04-13 18:02:35 [INFO] [TRAIN] epoch: 420, iter: 104000/120000, loss: 0.8201, lr: 0.001631, batch_cost: 0.2206, reader_cost: 0.00090, ips: 13.5987 samples/sec | ETA 00:58:49
- 2022-04-13 18:02:35 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1947 - reader cost: 0.1514
- 2022-04-13 18:03:00 [INFO] [EVAL] #Images: 500 mIoU: 0.7870 Acc: 0.9626 Kappa: 0.9514 Dice: 0.8751
- 2022-04-13 18:03:00 [INFO] [EVAL] Class IoU:
- [0.9829 0.8597 0.9274 0.5925 0.5887 0.6443 0.7284 0.8027 0.9267 0.6742
- 0.9504 0.8218 0.6473 0.9535 0.824 0.8693 0.7271 0.6472 0.7839]
- 2022-04-13 18:03:00 [INFO] [EVAL] Class Precision:
- [0.9923 0.923 0.9558 0.8163 0.8231 0.858 0.8623 0.9087 0.9556 0.822
- 0.9695 0.8946 0.7798 0.9725 0.9423 0.9276 0.954 0.7781 0.8636]
- 2022-04-13 18:03:00 [INFO] [EVAL] Class Recall:
- [0.9904 0.9262 0.9689 0.6836 0.674 0.7212 0.8243 0.8731 0.9684 0.7895
- 0.9797 0.9099 0.7921 0.98 0.8679 0.9327 0.7535 0.7936 0.8947]
- 2022-04-13 18:03:01 [INFO] [EVAL] The model with the best validation mIoU (0.7870) was saved at iter 104000.
- 2022-04-13 18:03:12 [INFO] [TRAIN] epoch: 420, iter: 104050/120000, loss: 0.8041, lr: 0.001626, batch_cost: 0.2231, reader_cost: 0.00176, ips: 13.4439 samples/sec | ETA 00:59:19
- 2022-04-13 18:03:23 [INFO] [TRAIN] epoch: 420, iter: 104100/120000, loss: 0.8191, lr: 0.001622, batch_cost: 0.2297, reader_cost: 0.00085, ips: 13.0630 samples/sec | ETA 01:00:51
- 2022-04-13 18:03:34 [INFO] [TRAIN] epoch: 420, iter: 104150/120000, loss: 0.8079, lr: 0.001617, batch_cost: 0.2103, reader_cost: 0.00093, ips: 14.2627 samples/sec | ETA 00:55:33
- 2022-04-13 18:03:48 [INFO] [TRAIN] epoch: 421, iter: 104200/120000, loss: 0.8187, lr: 0.001613, batch_cost: 0.2825, reader_cost: 0.05596, ips: 10.6192 samples/sec | ETA 01:14:23
- 2022-04-13 18:03:59 [INFO] [TRAIN] epoch: 421, iter: 104250/120000, loss: 0.8079, lr: 0.001608, batch_cost: 0.2195, reader_cost: 0.00079, ips: 13.6671 samples/sec | ETA 00:57:37
- 2022-04-13 18:04:09 [INFO] [TRAIN] epoch: 421, iter: 104300/120000, loss: 0.7915, lr: 0.001604, batch_cost: 0.1958, reader_cost: 0.00121, ips: 15.3220 samples/sec | ETA 00:51:14
- 2022-04-13 18:04:19 [INFO] [TRAIN] epoch: 421, iter: 104350/120000, loss: 0.8162, lr: 0.001599, batch_cost: 0.2137, reader_cost: 0.00110, ips: 14.0391 samples/sec | ETA 00:55:44
- 2022-04-13 18:04:30 [INFO] [TRAIN] epoch: 421, iter: 104400/120000, loss: 0.8209, lr: 0.001594, batch_cost: 0.2115, reader_cost: 0.00082, ips: 14.1867 samples/sec | ETA 00:54:58
- 2022-04-13 18:04:44 [INFO] [TRAIN] epoch: 422, iter: 104450/120000, loss: 0.8144, lr: 0.001590, batch_cost: 0.2727, reader_cost: 0.04950, ips: 11.0002 samples/sec | ETA 01:10:40
- 2022-04-13 18:04:54 [INFO] [TRAIN] epoch: 422, iter: 104500/120000, loss: 0.8179, lr: 0.001585, batch_cost: 0.2151, reader_cost: 0.00122, ips: 13.9491 samples/sec | ETA 00:55:33
- 2022-04-13 18:05:05 [INFO] [TRAIN] epoch: 422, iter: 104550/120000, loss: 0.8205, lr: 0.001581, batch_cost: 0.2128, reader_cost: 0.00186, ips: 14.0964 samples/sec | ETA 00:54:48
- 2022-04-13 18:05:16 [INFO] [TRAIN] epoch: 422, iter: 104600/120000, loss: 0.8137, lr: 0.001576, batch_cost: 0.2231, reader_cost: 0.00075, ips: 13.4466 samples/sec | ETA 00:57:15
- 2022-04-13 18:05:26 [INFO] [TRAIN] epoch: 422, iter: 104650/120000, loss: 0.8117, lr: 0.001571, batch_cost: 0.2037, reader_cost: 0.00110, ips: 14.7306 samples/sec | ETA 00:52:06
- 2022-04-13 18:05:40 [INFO] [TRAIN] epoch: 423, iter: 104700/120000, loss: 0.8141, lr: 0.001567, batch_cost: 0.2743, reader_cost: 0.05317, ips: 10.9355 samples/sec | ETA 01:09:57
- 2022-04-13 18:05:50 [INFO] [TRAIN] epoch: 423, iter: 104750/120000, loss: 0.8212, lr: 0.001562, batch_cost: 0.1968, reader_cost: 0.00162, ips: 15.2457 samples/sec | ETA 00:50:00
- 2022-04-13 18:06:00 [INFO] [TRAIN] epoch: 423, iter: 104800/120000, loss: 0.7931, lr: 0.001557, batch_cost: 0.2020, reader_cost: 0.00138, ips: 14.8546 samples/sec | ETA 00:51:09
- 2022-04-13 18:06:12 [INFO] [TRAIN] epoch: 423, iter: 104850/120000, loss: 0.8251, lr: 0.001553, batch_cost: 0.2437, reader_cost: 0.00149, ips: 12.3120 samples/sec | ETA 01:01:31
- 2022-04-13 18:06:22 [INFO] [TRAIN] epoch: 423, iter: 104900/120000, loss: 0.8141, lr: 0.001548, batch_cost: 0.1930, reader_cost: 0.00063, ips: 15.5433 samples/sec | ETA 00:48:34
- 2022-04-13 18:06:37 [INFO] [TRAIN] epoch: 424, iter: 104950/120000, loss: 0.8241, lr: 0.001544, batch_cost: 0.2932, reader_cost: 0.05113, ips: 10.2310 samples/sec | ETA 01:13:33
- 2022-04-13 18:06:47 [INFO] [TRAIN] epoch: 424, iter: 105000/120000, loss: 0.8284, lr: 0.001539, batch_cost: 0.2059, reader_cost: 0.00089, ips: 14.5673 samples/sec | ETA 00:51:29
- 2022-04-13 18:06:58 [INFO] [TRAIN] epoch: 424, iter: 105050/120000, loss: 0.7937, lr: 0.001534, batch_cost: 0.2177, reader_cost: 0.00156, ips: 13.7781 samples/sec | ETA 00:54:15
- 2022-04-13 18:07:09 [INFO] [TRAIN] epoch: 424, iter: 105100/120000, loss: 0.8189, lr: 0.001530, batch_cost: 0.2182, reader_cost: 0.00074, ips: 13.7484 samples/sec | ETA 00:54:11
- 2022-04-13 18:07:19 [INFO] [TRAIN] epoch: 424, iter: 105150/120000, loss: 0.8080, lr: 0.001525, batch_cost: 0.2015, reader_cost: 0.00101, ips: 14.8908 samples/sec | ETA 00:49:51
- 2022-04-13 18:07:33 [INFO] [TRAIN] epoch: 425, iter: 105200/120000, loss: 0.8057, lr: 0.001521, batch_cost: 0.2911, reader_cost: 0.05891, ips: 10.3046 samples/sec | ETA 01:11:48
- 2022-04-13 18:07:45 [INFO] [TRAIN] epoch: 425, iter: 105250/120000, loss: 0.8070, lr: 0.001516, batch_cost: 0.2284, reader_cost: 0.00129, ips: 13.1330 samples/sec | ETA 00:56:09
- 2022-04-13 18:07:56 [INFO] [TRAIN] epoch: 425, iter: 105300/120000, loss: 0.8111, lr: 0.001511, batch_cost: 0.2269, reader_cost: 0.00129, ips: 13.2197 samples/sec | ETA 00:55:35
- 2022-04-13 18:08:06 [INFO] [TRAIN] epoch: 425, iter: 105350/120000, loss: 0.8088, lr: 0.001507, batch_cost: 0.2059, reader_cost: 0.00133, ips: 14.5734 samples/sec | ETA 00:50:15
- 2022-04-13 18:08:17 [INFO] [TRAIN] epoch: 425, iter: 105400/120000, loss: 0.7944, lr: 0.001502, batch_cost: 0.2094, reader_cost: 0.00117, ips: 14.3238 samples/sec | ETA 00:50:57
- 2022-04-13 18:08:31 [INFO] [TRAIN] epoch: 426, iter: 105450/120000, loss: 0.8002, lr: 0.001497, batch_cost: 0.2768, reader_cost: 0.05894, ips: 10.8397 samples/sec | ETA 01:07:06
- 2022-04-13 18:08:41 [INFO] [TRAIN] epoch: 426, iter: 105500/120000, loss: 0.8048, lr: 0.001493, batch_cost: 0.2105, reader_cost: 0.00117, ips: 14.2543 samples/sec | ETA 00:50:51
- 2022-04-13 18:08:52 [INFO] [TRAIN] epoch: 426, iter: 105550/120000, loss: 0.8254, lr: 0.001488, batch_cost: 0.2092, reader_cost: 0.00112, ips: 14.3386 samples/sec | ETA 00:50:23
- 2022-04-13 18:09:02 [INFO] [TRAIN] epoch: 426, iter: 105600/120000, loss: 0.8062, lr: 0.001484, batch_cost: 0.2157, reader_cost: 0.00127, ips: 13.9071 samples/sec | ETA 00:51:46
- 2022-04-13 18:09:16 [INFO] [TRAIN] epoch: 427, iter: 105650/120000, loss: 0.8062, lr: 0.001479, batch_cost: 0.2680, reader_cost: 0.06001, ips: 11.1949 samples/sec | ETA 01:04:05
- 2022-04-13 18:09:26 [INFO] [TRAIN] epoch: 427, iter: 105700/120000, loss: 0.7983, lr: 0.001474, batch_cost: 0.2043, reader_cost: 0.00071, ips: 14.6875 samples/sec | ETA 00:48:40
- 2022-04-13 18:09:36 [INFO] [TRAIN] epoch: 427, iter: 105750/120000, loss: 0.8266, lr: 0.001470, batch_cost: 0.2066, reader_cost: 0.00115, ips: 14.5211 samples/sec | ETA 00:49:03
- 2022-04-13 18:09:46 [INFO] [TRAIN] epoch: 427, iter: 105800/120000, loss: 0.7995, lr: 0.001465, batch_cost: 0.1992, reader_cost: 0.00044, ips: 15.0569 samples/sec | ETA 00:47:09
- 2022-04-13 18:09:58 [INFO] [TRAIN] epoch: 427, iter: 105850/120000, loss: 0.8268, lr: 0.001460, batch_cost: 0.2255, reader_cost: 0.00136, ips: 13.3056 samples/sec | ETA 00:53:10
- 2022-04-13 18:10:11 [INFO] [TRAIN] epoch: 428, iter: 105900/120000, loss: 0.7953, lr: 0.001456, batch_cost: 0.2676, reader_cost: 0.05837, ips: 11.2106 samples/sec | ETA 01:02:53
- 2022-04-13 18:10:22 [INFO] [TRAIN] epoch: 428, iter: 105950/120000, loss: 0.8226, lr: 0.001451, batch_cost: 0.2235, reader_cost: 0.00128, ips: 13.4242 samples/sec | ETA 00:52:19
- 2022-04-13 18:10:33 [INFO] [TRAIN] epoch: 428, iter: 106000/120000, loss: 0.8187, lr: 0.001446, batch_cost: 0.2231, reader_cost: 0.00293, ips: 13.4494 samples/sec | ETA 00:52:02
- 2022-04-13 18:10:44 [INFO] [TRAIN] epoch: 428, iter: 106050/120000, loss: 0.8124, lr: 0.001442, batch_cost: 0.2049, reader_cost: 0.00058, ips: 14.6388 samples/sec | ETA 00:47:38
- 2022-04-13 18:10:54 [INFO] [TRAIN] epoch: 428, iter: 106100/120000, loss: 0.8060, lr: 0.001437, batch_cost: 0.2090, reader_cost: 0.00131, ips: 14.3542 samples/sec | ETA 00:48:25
- 2022-04-13 18:11:07 [INFO] [TRAIN] epoch: 429, iter: 106150/120000, loss: 0.8195, lr: 0.001432, batch_cost: 0.2662, reader_cost: 0.06179, ips: 11.2697 samples/sec | ETA 01:01:26
- 2022-04-13 18:11:18 [INFO] [TRAIN] epoch: 429, iter: 106200/120000, loss: 0.8087, lr: 0.001428, batch_cost: 0.2060, reader_cost: 0.00134, ips: 14.5608 samples/sec | ETA 00:47:23
- 2022-04-13 18:11:28 [INFO] [TRAIN] epoch: 429, iter: 106250/120000, loss: 0.8061, lr: 0.001423, batch_cost: 0.2031, reader_cost: 0.00086, ips: 14.7724 samples/sec | ETA 00:46:32
- 2022-04-13 18:11:38 [INFO] [TRAIN] epoch: 429, iter: 106300/120000, loss: 0.8120, lr: 0.001418, batch_cost: 0.2095, reader_cost: 0.00057, ips: 14.3188 samples/sec | ETA 00:47:50
- 2022-04-13 18:11:50 [INFO] [TRAIN] epoch: 429, iter: 106350/120000, loss: 0.8060, lr: 0.001414, batch_cost: 0.2247, reader_cost: 0.00066, ips: 13.3503 samples/sec | ETA 00:51:07
- 2022-04-13 18:12:03 [INFO] [TRAIN] epoch: 430, iter: 106400/120000, loss: 0.8187, lr: 0.001409, batch_cost: 0.2766, reader_cost: 0.06484, ips: 10.8477 samples/sec | ETA 01:02:41
- 2022-04-13 18:12:14 [INFO] [TRAIN] epoch: 430, iter: 106450/120000, loss: 0.8022, lr: 0.001404, batch_cost: 0.2030, reader_cost: 0.00230, ips: 14.7811 samples/sec | ETA 00:45:50
- 2022-04-13 18:12:24 [INFO] [TRAIN] epoch: 430, iter: 106500/120000, loss: 0.7925, lr: 0.001400, batch_cost: 0.2010, reader_cost: 0.00043, ips: 14.9234 samples/sec | ETA 00:45:13
- 2022-04-13 18:12:34 [INFO] [TRAIN] epoch: 430, iter: 106550/120000, loss: 0.8048, lr: 0.001395, batch_cost: 0.2165, reader_cost: 0.00056, ips: 13.8551 samples/sec | ETA 00:48:32
- 2022-04-13 18:12:45 [INFO] [TRAIN] epoch: 430, iter: 106600/120000, loss: 0.8031, lr: 0.001390, batch_cost: 0.2201, reader_cost: 0.00086, ips: 13.6306 samples/sec | ETA 00:49:09
- 2022-04-13 18:12:59 [INFO] [TRAIN] epoch: 431, iter: 106650/120000, loss: 0.8255, lr: 0.001386, batch_cost: 0.2698, reader_cost: 0.05977, ips: 11.1194 samples/sec | ETA 01:00:01
- 2022-04-13 18:13:10 [INFO] [TRAIN] epoch: 431, iter: 106700/120000, loss: 0.8049, lr: 0.001381, batch_cost: 0.2273, reader_cost: 0.00114, ips: 13.2001 samples/sec | ETA 00:50:22
- 2022-04-13 18:13:21 [INFO] [TRAIN] epoch: 431, iter: 106750/120000, loss: 0.8014, lr: 0.001376, batch_cost: 0.2180, reader_cost: 0.00144, ips: 13.7630 samples/sec | ETA 00:48:08
- 2022-04-13 18:13:32 [INFO] [TRAIN] epoch: 431, iter: 106800/120000, loss: 0.8190, lr: 0.001372, batch_cost: 0.2191, reader_cost: 0.00152, ips: 13.6921 samples/sec | ETA 00:48:12
- 2022-04-13 18:13:43 [INFO] [TRAIN] epoch: 431, iter: 106850/120000, loss: 0.8153, lr: 0.001367, batch_cost: 0.2179, reader_cost: 0.00085, ips: 13.7669 samples/sec | ETA 00:47:45
- 2022-04-13 18:13:57 [INFO] [TRAIN] epoch: 432, iter: 106900/120000, loss: 0.8149, lr: 0.001362, batch_cost: 0.2727, reader_cost: 0.05885, ips: 11.0008 samples/sec | ETA 00:59:32
- 2022-04-13 18:14:07 [INFO] [TRAIN] epoch: 432, iter: 106950/120000, loss: 0.7995, lr: 0.001358, batch_cost: 0.2021, reader_cost: 0.00126, ips: 14.8434 samples/sec | ETA 00:43:57
- 2022-04-13 18:14:17 [INFO] [TRAIN] epoch: 432, iter: 107000/120000, loss: 0.8010, lr: 0.001353, batch_cost: 0.2060, reader_cost: 0.00032, ips: 14.5664 samples/sec | ETA 00:44:37
- 2022-04-13 18:14:27 [INFO] [TRAIN] epoch: 432, iter: 107050/120000, loss: 0.7932, lr: 0.001348, batch_cost: 0.2031, reader_cost: 0.00120, ips: 14.7707 samples/sec | ETA 00:43:50
- 2022-04-13 18:14:39 [INFO] [TRAIN] epoch: 432, iter: 107100/120000, loss: 0.8023, lr: 0.001344, batch_cost: 0.2348, reader_cost: 0.00103, ips: 12.7754 samples/sec | ETA 00:50:29
- 2022-04-13 18:14:53 [INFO] [TRAIN] epoch: 433, iter: 107150/120000, loss: 0.8158, lr: 0.001339, batch_cost: 0.2720, reader_cost: 0.05708, ips: 11.0285 samples/sec | ETA 00:58:15
- 2022-04-13 18:15:03 [INFO] [TRAIN] epoch: 433, iter: 107200/120000, loss: 0.8111, lr: 0.001334, batch_cost: 0.2059, reader_cost: 0.00077, ips: 14.5701 samples/sec | ETA 00:43:55
- 2022-04-13 18:15:14 [INFO] [TRAIN] epoch: 433, iter: 107250/120000, loss: 0.8280, lr: 0.001330, batch_cost: 0.2142, reader_cost: 0.00108, ips: 14.0060 samples/sec | ETA 00:45:30
- 2022-04-13 18:15:24 [INFO] [TRAIN] epoch: 433, iter: 107300/120000, loss: 0.8081, lr: 0.001325, batch_cost: 0.2167, reader_cost: 0.00099, ips: 13.8452 samples/sec | ETA 00:45:51
- 2022-04-13 18:15:34 [INFO] [TRAIN] epoch: 433, iter: 107350/120000, loss: 0.8073, lr: 0.001320, batch_cost: 0.1989, reader_cost: 0.00082, ips: 15.0818 samples/sec | ETA 00:41:56
- 2022-04-13 18:15:48 [INFO] [TRAIN] epoch: 434, iter: 107400/120000, loss: 0.8285, lr: 0.001316, batch_cost: 0.2767, reader_cost: 0.05681, ips: 10.8430 samples/sec | ETA 00:58:06
- 2022-04-13 18:15:59 [INFO] [TRAIN] epoch: 434, iter: 107450/120000, loss: 0.7964, lr: 0.001311, batch_cost: 0.2212, reader_cost: 0.00095, ips: 13.5632 samples/sec | ETA 00:46:15
- 2022-04-13 18:16:10 [INFO] [TRAIN] epoch: 434, iter: 107500/120000, loss: 0.8113, lr: 0.001306, batch_cost: 0.2059, reader_cost: 0.00058, ips: 14.5710 samples/sec | ETA 00:42:53
- 2022-04-13 18:16:20 [INFO] [TRAIN] epoch: 434, iter: 107550/120000, loss: 0.8183, lr: 0.001301, batch_cost: 0.2125, reader_cost: 0.00044, ips: 14.1158 samples/sec | ETA 00:44:05
- 2022-04-13 18:16:31 [INFO] [TRAIN] epoch: 434, iter: 107600/120000, loss: 0.7965, lr: 0.001297, batch_cost: 0.2152, reader_cost: 0.00095, ips: 13.9417 samples/sec | ETA 00:44:28
- 2022-04-13 18:16:45 [INFO] [TRAIN] epoch: 435, iter: 107650/120000, loss: 0.8127, lr: 0.001292, batch_cost: 0.2796, reader_cost: 0.06170, ips: 10.7288 samples/sec | ETA 00:57:33
- 2022-04-13 18:16:56 [INFO] [TRAIN] epoch: 435, iter: 107700/120000, loss: 0.8093, lr: 0.001287, batch_cost: 0.2143, reader_cost: 0.00083, ips: 13.9968 samples/sec | ETA 00:43:56
- 2022-04-13 18:17:06 [INFO] [TRAIN] epoch: 435, iter: 107750/120000, loss: 0.8084, lr: 0.001283, batch_cost: 0.2133, reader_cost: 0.00147, ips: 14.0672 samples/sec | ETA 00:43:32
- 2022-04-13 18:17:17 [INFO] [TRAIN] epoch: 435, iter: 107800/120000, loss: 0.8125, lr: 0.001278, batch_cost: 0.2046, reader_cost: 0.00138, ips: 14.6610 samples/sec | ETA 00:41:36
- 2022-04-13 18:17:27 [INFO] [TRAIN] epoch: 435, iter: 107850/120000, loss: 0.8241, lr: 0.001273, batch_cost: 0.2022, reader_cost: 0.00072, ips: 14.8344 samples/sec | ETA 00:40:57
- 2022-04-13 18:17:40 [INFO] [TRAIN] epoch: 436, iter: 107900/120000, loss: 0.8203, lr: 0.001268, batch_cost: 0.2657, reader_cost: 0.05784, ips: 11.2908 samples/sec | ETA 00:53:35
- 2022-04-13 18:17:51 [INFO] [TRAIN] epoch: 436, iter: 107950/120000, loss: 0.8055, lr: 0.001264, batch_cost: 0.2151, reader_cost: 0.00197, ips: 13.9469 samples/sec | ETA 00:43:11
- 2022-04-13 18:18:01 [INFO] [TRAIN] epoch: 436, iter: 108000/120000, loss: 0.8067, lr: 0.001259, batch_cost: 0.2120, reader_cost: 0.00056, ips: 14.1537 samples/sec | ETA 00:42:23
- 2022-04-13 18:18:01 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1958 - reader cost: 0.1524
- 2022-04-13 18:18:26 [INFO] [EVAL] #Images: 500 mIoU: 0.7878 Acc: 0.9612 Kappa: 0.9497 Dice: 0.8750
- 2022-04-13 18:18:26 [INFO] [EVAL] Class IoU:
- [0.9809 0.8467 0.9252 0.5185 0.6122 0.6625 0.73 0.7994 0.9248 0.6258
- 0.952 0.8278 0.6498 0.9535 0.807 0.8996 0.8167 0.6544 0.7822]
- 2022-04-13 18:18:26 [INFO] [EVAL] Class Precision:
- [0.9933 0.8989 0.9567 0.8683 0.8511 0.8338 0.8542 0.9133 0.9497 0.831
- 0.9726 0.8881 0.7535 0.9758 0.8853 0.9679 0.9282 0.8055 0.8777]
- 2022-04-13 18:18:26 [INFO] [EVAL] Class Recall:
- [0.9875 0.9358 0.9657 0.5627 0.6856 0.7633 0.8339 0.865 0.9724 0.717
- 0.9782 0.9241 0.8252 0.9766 0.9013 0.9273 0.8718 0.7772 0.8778]
- 2022-04-13 18:18:27 [INFO] [EVAL] The model with the best validation mIoU (0.7878) was saved at iter 108000.
- 2022-04-13 18:18:38 [INFO] [TRAIN] epoch: 436, iter: 108050/120000, loss: 0.8302, lr: 0.001254, batch_cost: 0.2277, reader_cost: 0.00210, ips: 13.1774 samples/sec | ETA 00:45:20
- 2022-04-13 18:18:49 [INFO] [TRAIN] epoch: 436, iter: 108100/120000, loss: 0.8068, lr: 0.001250, batch_cost: 0.2067, reader_cost: 0.00181, ips: 14.5170 samples/sec | ETA 00:40:59
- 2022-04-13 18:19:02 [INFO] [TRAIN] epoch: 437, iter: 108150/120000, loss: 0.8074, lr: 0.001245, batch_cost: 0.2670, reader_cost: 0.05207, ips: 11.2360 samples/sec | ETA 00:52:43
- 2022-04-13 18:19:13 [INFO] [TRAIN] epoch: 437, iter: 108200/120000, loss: 0.7927, lr: 0.001240, batch_cost: 0.2222, reader_cost: 0.00085, ips: 13.5024 samples/sec | ETA 00:43:41
- 2022-04-13 18:19:23 [INFO] [TRAIN] epoch: 437, iter: 108250/120000, loss: 0.8093, lr: 0.001235, batch_cost: 0.1980, reader_cost: 0.00070, ips: 15.1539 samples/sec | ETA 00:38:46
- 2022-04-13 18:19:35 [INFO] [TRAIN] epoch: 437, iter: 108300/120000, loss: 0.8042, lr: 0.001231, batch_cost: 0.2375, reader_cost: 0.00123, ips: 12.6292 samples/sec | ETA 00:46:19
- 2022-04-13 18:19:46 [INFO] [TRAIN] epoch: 437, iter: 108350/120000, loss: 0.8308, lr: 0.001226, batch_cost: 0.2164, reader_cost: 0.00089, ips: 13.8653 samples/sec | ETA 00:42:00
- 2022-04-13 18:20:00 [INFO] [TRAIN] epoch: 438, iter: 108400/120000, loss: 0.7921, lr: 0.001221, batch_cost: 0.2788, reader_cost: 0.05667, ips: 10.7587 samples/sec | ETA 00:53:54
- 2022-04-13 18:20:10 [INFO] [TRAIN] epoch: 438, iter: 108450/120000, loss: 0.7950, lr: 0.001216, batch_cost: 0.2081, reader_cost: 0.00083, ips: 14.4152 samples/sec | ETA 00:40:03
- 2022-04-13 18:20:21 [INFO] [TRAIN] epoch: 438, iter: 108500/120000, loss: 0.8287, lr: 0.001212, batch_cost: 0.2208, reader_cost: 0.00053, ips: 13.5898 samples/sec | ETA 00:42:18
- 2022-04-13 18:20:32 [INFO] [TRAIN] epoch: 438, iter: 108550/120000, loss: 0.8172, lr: 0.001207, batch_cost: 0.2180, reader_cost: 0.00094, ips: 13.7613 samples/sec | ETA 00:41:36
- 2022-04-13 18:20:42 [INFO] [TRAIN] epoch: 438, iter: 108600/120000, loss: 0.8135, lr: 0.001202, batch_cost: 0.2039, reader_cost: 0.00043, ips: 14.7121 samples/sec | ETA 00:38:44
- 2022-04-13 18:20:56 [INFO] [TRAIN] epoch: 439, iter: 108650/120000, loss: 0.7893, lr: 0.001197, batch_cost: 0.2685, reader_cost: 0.06077, ips: 11.1713 samples/sec | ETA 00:50:48
- 2022-04-13 18:21:06 [INFO] [TRAIN] epoch: 439, iter: 108700/120000, loss: 0.7928, lr: 0.001193, batch_cost: 0.2020, reader_cost: 0.00140, ips: 14.8548 samples/sec | ETA 00:38:02
- 2022-04-13 18:21:16 [INFO] [TRAIN] epoch: 439, iter: 108750/120000, loss: 0.8086, lr: 0.001188, batch_cost: 0.2130, reader_cost: 0.00060, ips: 14.0877 samples/sec | ETA 00:39:55
- 2022-04-13 18:21:27 [INFO] [TRAIN] epoch: 439, iter: 108800/120000, loss: 0.8233, lr: 0.001183, batch_cost: 0.2136, reader_cost: 0.00055, ips: 14.0456 samples/sec | ETA 00:39:52
- 2022-04-13 18:21:37 [INFO] [TRAIN] epoch: 439, iter: 108850/120000, loss: 0.8121, lr: 0.001178, batch_cost: 0.2067, reader_cost: 0.00056, ips: 14.5152 samples/sec | ETA 00:38:24
- 2022-04-13 18:21:51 [INFO] [TRAIN] epoch: 440, iter: 108900/120000, loss: 0.8072, lr: 0.001174, batch_cost: 0.2684, reader_cost: 0.05630, ips: 11.1792 samples/sec | ETA 00:49:38
- 2022-04-13 18:22:02 [INFO] [TRAIN] epoch: 440, iter: 108950/120000, loss: 0.7982, lr: 0.001169, batch_cost: 0.2251, reader_cost: 0.00095, ips: 13.3285 samples/sec | ETA 00:41:27
- 2022-04-13 18:22:12 [INFO] [TRAIN] epoch: 440, iter: 109000/120000, loss: 0.7972, lr: 0.001164, batch_cost: 0.1999, reader_cost: 0.00034, ips: 15.0048 samples/sec | ETA 00:36:39
- 2022-04-13 18:22:23 [INFO] [TRAIN] epoch: 440, iter: 109050/120000, loss: 0.7997, lr: 0.001159, batch_cost: 0.2165, reader_cost: 0.00144, ips: 13.8564 samples/sec | ETA 00:39:30
- 2022-04-13 18:22:33 [INFO] [TRAIN] epoch: 440, iter: 109100/120000, loss: 0.8080, lr: 0.001155, batch_cost: 0.2020, reader_cost: 0.00090, ips: 14.8511 samples/sec | ETA 00:36:41
- 2022-04-13 18:22:47 [INFO] [TRAIN] epoch: 441, iter: 109150/120000, loss: 0.7908, lr: 0.001150, batch_cost: 0.2741, reader_cost: 0.05650, ips: 10.9432 samples/sec | ETA 00:49:34
- 2022-04-13 18:22:57 [INFO] [TRAIN] epoch: 441, iter: 109200/120000, loss: 0.8058, lr: 0.001145, batch_cost: 0.2058, reader_cost: 0.00097, ips: 14.5797 samples/sec | ETA 00:37:02
- 2022-04-13 18:23:08 [INFO] [TRAIN] epoch: 441, iter: 109250/120000, loss: 0.8045, lr: 0.001140, batch_cost: 0.2247, reader_cost: 0.00080, ips: 13.3526 samples/sec | ETA 00:40:15
- 2022-04-13 18:23:18 [INFO] [TRAIN] epoch: 441, iter: 109300/120000, loss: 0.8060, lr: 0.001136, batch_cost: 0.2032, reader_cost: 0.00106, ips: 14.7624 samples/sec | ETA 00:36:14
- 2022-04-13 18:23:29 [INFO] [TRAIN] epoch: 441, iter: 109350/120000, loss: 0.7876, lr: 0.001131, batch_cost: 0.2097, reader_cost: 0.00117, ips: 14.3032 samples/sec | ETA 00:37:13
- 2022-04-13 18:23:43 [INFO] [TRAIN] epoch: 442, iter: 109400/120000, loss: 0.8069, lr: 0.001126, batch_cost: 0.2864, reader_cost: 0.06374, ips: 10.4739 samples/sec | ETA 00:50:36
- 2022-04-13 18:23:54 [INFO] [TRAIN] epoch: 442, iter: 109450/120000, loss: 0.7973, lr: 0.001121, batch_cost: 0.2248, reader_cost: 0.00131, ips: 13.3458 samples/sec | ETA 00:39:31
- 2022-04-13 18:24:05 [INFO] [TRAIN] epoch: 442, iter: 109500/120000, loss: 0.8010, lr: 0.001116, batch_cost: 0.2196, reader_cost: 0.00080, ips: 13.6633 samples/sec | ETA 00:38:25
- 2022-04-13 18:24:16 [INFO] [TRAIN] epoch: 442, iter: 109550/120000, loss: 0.8235, lr: 0.001112, batch_cost: 0.2163, reader_cost: 0.00134, ips: 13.8728 samples/sec | ETA 00:37:39
- 2022-04-13 18:24:27 [INFO] [TRAIN] epoch: 442, iter: 109600/120000, loss: 0.8086, lr: 0.001107, batch_cost: 0.2089, reader_cost: 0.00078, ips: 14.3600 samples/sec | ETA 00:36:12
- 2022-04-13 18:24:41 [INFO] [TRAIN] epoch: 443, iter: 109650/120000, loss: 0.8057, lr: 0.001102, batch_cost: 0.2836, reader_cost: 0.05838, ips: 10.5789 samples/sec | ETA 00:48:55
- 2022-04-13 18:24:51 [INFO] [TRAIN] epoch: 443, iter: 109700/120000, loss: 0.7900, lr: 0.001097, batch_cost: 0.2036, reader_cost: 0.00099, ips: 14.7370 samples/sec | ETA 00:34:56
- 2022-04-13 18:25:02 [INFO] [TRAIN] epoch: 443, iter: 109750/120000, loss: 0.8067, lr: 0.001093, batch_cost: 0.2195, reader_cost: 0.00140, ips: 13.6684 samples/sec | ETA 00:37:29
- 2022-04-13 18:25:14 [INFO] [TRAIN] epoch: 443, iter: 109800/120000, loss: 0.7862, lr: 0.001088, batch_cost: 0.2344, reader_cost: 0.00093, ips: 12.7965 samples/sec | ETA 00:39:51
- 2022-04-13 18:25:24 [INFO] [TRAIN] epoch: 443, iter: 109850/120000, loss: 0.8221, lr: 0.001083, batch_cost: 0.2103, reader_cost: 0.00083, ips: 14.2652 samples/sec | ETA 00:35:34
- 2022-04-13 18:25:39 [INFO] [TRAIN] epoch: 444, iter: 109900/120000, loss: 0.7973, lr: 0.001078, batch_cost: 0.2898, reader_cost: 0.05865, ips: 10.3504 samples/sec | ETA 00:48:47
- 2022-04-13 18:25:49 [INFO] [TRAIN] epoch: 444, iter: 109950/120000, loss: 0.8154, lr: 0.001073, batch_cost: 0.2124, reader_cost: 0.00137, ips: 14.1234 samples/sec | ETA 00:35:34
- 2022-04-13 18:26:00 [INFO] [TRAIN] epoch: 444, iter: 110000/120000, loss: 0.8032, lr: 0.001069, batch_cost: 0.2193, reader_cost: 0.00070, ips: 13.6820 samples/sec | ETA 00:36:32
- 2022-04-13 18:26:11 [INFO] [TRAIN] epoch: 444, iter: 110050/120000, loss: 0.7967, lr: 0.001064, batch_cost: 0.2162, reader_cost: 0.00105, ips: 13.8789 samples/sec | ETA 00:35:50
- 2022-04-13 18:26:22 [INFO] [TRAIN] epoch: 444, iter: 110100/120000, loss: 0.8076, lr: 0.001059, batch_cost: 0.2130, reader_cost: 0.00042, ips: 14.0867 samples/sec | ETA 00:35:08
- 2022-04-13 18:26:36 [INFO] [TRAIN] epoch: 445, iter: 110150/120000, loss: 0.7902, lr: 0.001054, batch_cost: 0.2752, reader_cost: 0.05525, ips: 10.9016 samples/sec | ETA 00:45:10
- 2022-04-13 18:26:45 [INFO] [TRAIN] epoch: 445, iter: 110200/120000, loss: 0.8006, lr: 0.001049, batch_cost: 0.1982, reader_cost: 0.00147, ips: 15.1356 samples/sec | ETA 00:32:22
- 2022-04-13 18:26:56 [INFO] [TRAIN] epoch: 445, iter: 110250/120000, loss: 0.8054, lr: 0.001044, batch_cost: 0.2034, reader_cost: 0.00105, ips: 14.7477 samples/sec | ETA 00:33:03
- 2022-04-13 18:27:07 [INFO] [TRAIN] epoch: 445, iter: 110300/120000, loss: 0.8036, lr: 0.001040, batch_cost: 0.2205, reader_cost: 0.00125, ips: 13.6050 samples/sec | ETA 00:35:38
- 2022-04-13 18:27:17 [INFO] [TRAIN] epoch: 445, iter: 110350/120000, loss: 0.7931, lr: 0.001035, batch_cost: 0.2070, reader_cost: 0.00054, ips: 14.4906 samples/sec | ETA 00:33:17
- 2022-04-13 18:27:31 [INFO] [TRAIN] epoch: 446, iter: 110400/120000, loss: 0.7983, lr: 0.001030, batch_cost: 0.2738, reader_cost: 0.05512, ips: 10.9570 samples/sec | ETA 00:43:48
- 2022-04-13 18:27:41 [INFO] [TRAIN] epoch: 446, iter: 110450/120000, loss: 0.8033, lr: 0.001025, batch_cost: 0.2062, reader_cost: 0.00141, ips: 14.5502 samples/sec | ETA 00:32:49
- 2022-04-13 18:27:52 [INFO] [TRAIN] epoch: 446, iter: 110500/120000, loss: 0.7899, lr: 0.001020, batch_cost: 0.2131, reader_cost: 0.00079, ips: 14.0791 samples/sec | ETA 00:33:44
- 2022-04-13 18:28:03 [INFO] [TRAIN] epoch: 446, iter: 110550/120000, loss: 0.8127, lr: 0.001015, batch_cost: 0.2214, reader_cost: 0.00085, ips: 13.5531 samples/sec | ETA 00:34:51
- 2022-04-13 18:28:12 [INFO] [TRAIN] epoch: 446, iter: 110600/120000, loss: 0.8153, lr: 0.001011, batch_cost: 0.1926, reader_cost: 0.00055, ips: 15.5730 samples/sec | ETA 00:30:10
- 2022-04-13 18:28:27 [INFO] [TRAIN] epoch: 447, iter: 110650/120000, loss: 0.8041, lr: 0.001006, batch_cost: 0.2840, reader_cost: 0.05283, ips: 10.5643 samples/sec | ETA 00:44:15
- 2022-04-13 18:28:37 [INFO] [TRAIN] epoch: 447, iter: 110700/120000, loss: 0.8117, lr: 0.001001, batch_cost: 0.2070, reader_cost: 0.00103, ips: 14.4896 samples/sec | ETA 00:32:05
- 2022-04-13 18:28:47 [INFO] [TRAIN] epoch: 447, iter: 110750/120000, loss: 0.8220, lr: 0.000996, batch_cost: 0.2108, reader_cost: 0.00098, ips: 14.2282 samples/sec | ETA 00:32:30
- 2022-04-13 18:28:58 [INFO] [TRAIN] epoch: 447, iter: 110800/120000, loss: 0.7946, lr: 0.000991, batch_cost: 0.2036, reader_cost: 0.00056, ips: 14.7337 samples/sec | ETA 00:31:13
- 2022-04-13 18:29:08 [INFO] [TRAIN] epoch: 447, iter: 110850/120000, loss: 0.7997, lr: 0.000986, batch_cost: 0.2014, reader_cost: 0.00102, ips: 14.8946 samples/sec | ETA 00:30:42
- 2022-04-13 18:29:22 [INFO] [TRAIN] epoch: 448, iter: 110900/120000, loss: 0.8064, lr: 0.000982, batch_cost: 0.2888, reader_cost: 0.05404, ips: 10.3862 samples/sec | ETA 00:43:48
- 2022-04-13 18:29:33 [INFO] [TRAIN] epoch: 448, iter: 110950/120000, loss: 0.8253, lr: 0.000977, batch_cost: 0.2184, reader_cost: 0.00082, ips: 13.7384 samples/sec | ETA 00:32:56
- 2022-04-13 18:29:43 [INFO] [TRAIN] epoch: 448, iter: 111000/120000, loss: 0.7989, lr: 0.000972, batch_cost: 0.2023, reader_cost: 0.00097, ips: 14.8328 samples/sec | ETA 00:30:20
- 2022-04-13 18:29:55 [INFO] [TRAIN] epoch: 448, iter: 111050/120000, loss: 0.7984, lr: 0.000967, batch_cost: 0.2359, reader_cost: 0.00145, ips: 12.7149 samples/sec | ETA 00:35:11
- 2022-04-13 18:30:06 [INFO] [TRAIN] epoch: 448, iter: 111100/120000, loss: 0.7988, lr: 0.000962, batch_cost: 0.2118, reader_cost: 0.00110, ips: 14.1665 samples/sec | ETA 00:31:24
- 2022-04-13 18:30:20 [INFO] [TRAIN] epoch: 449, iter: 111150/120000, loss: 0.8129, lr: 0.000957, batch_cost: 0.2802, reader_cost: 0.06089, ips: 10.7069 samples/sec | ETA 00:41:19
- 2022-04-13 18:30:30 [INFO] [TRAIN] epoch: 449, iter: 111200/120000, loss: 0.7970, lr: 0.000952, batch_cost: 0.2020, reader_cost: 0.00125, ips: 14.8503 samples/sec | ETA 00:29:37
- 2022-04-13 18:30:40 [INFO] [TRAIN] epoch: 449, iter: 111250/120000, loss: 0.8157, lr: 0.000948, batch_cost: 0.2013, reader_cost: 0.00125, ips: 14.9026 samples/sec | ETA 00:29:21
- 2022-04-13 18:30:50 [INFO] [TRAIN] epoch: 449, iter: 111300/120000, loss: 0.7915, lr: 0.000943, batch_cost: 0.2075, reader_cost: 0.00158, ips: 14.4605 samples/sec | ETA 00:30:04
- 2022-04-13 18:31:02 [INFO] [TRAIN] epoch: 449, iter: 111350/120000, loss: 0.7895, lr: 0.000938, batch_cost: 0.2467, reader_cost: 0.00054, ips: 12.1609 samples/sec | ETA 00:35:33
- 2022-04-13 18:31:16 [INFO] [TRAIN] epoch: 450, iter: 111400/120000, loss: 0.8010, lr: 0.000933, batch_cost: 0.2797, reader_cost: 0.05784, ips: 10.7275 samples/sec | ETA 00:40:05
- 2022-04-13 18:31:27 [INFO] [TRAIN] epoch: 450, iter: 111450/120000, loss: 0.8022, lr: 0.000928, batch_cost: 0.2080, reader_cost: 0.00097, ips: 14.4246 samples/sec | ETA 00:29:38
- 2022-04-13 18:31:37 [INFO] [TRAIN] epoch: 450, iter: 111500/120000, loss: 0.7982, lr: 0.000923, batch_cost: 0.1995, reader_cost: 0.00149, ips: 15.0411 samples/sec | ETA 00:28:15
- 2022-04-13 18:31:48 [INFO] [TRAIN] epoch: 450, iter: 111550/120000, loss: 0.8124, lr: 0.000918, batch_cost: 0.2278, reader_cost: 0.00103, ips: 13.1700 samples/sec | ETA 00:32:04
- 2022-04-13 18:31:59 [INFO] [TRAIN] epoch: 450, iter: 111600/120000, loss: 0.7901, lr: 0.000913, batch_cost: 0.2058, reader_cost: 0.00126, ips: 14.5806 samples/sec | ETA 00:28:48
- 2022-04-13 18:32:13 [INFO] [TRAIN] epoch: 451, iter: 111650/120000, loss: 0.7936, lr: 0.000908, batch_cost: 0.2948, reader_cost: 0.05877, ips: 10.1747 samples/sec | ETA 00:41:01
- 2022-04-13 18:32:24 [INFO] [TRAIN] epoch: 451, iter: 111700/120000, loss: 0.8033, lr: 0.000904, batch_cost: 0.2119, reader_cost: 0.00107, ips: 14.1549 samples/sec | ETA 00:29:19
- 2022-04-13 18:32:35 [INFO] [TRAIN] epoch: 451, iter: 111750/120000, loss: 0.7941, lr: 0.000899, batch_cost: 0.2161, reader_cost: 0.00086, ips: 13.8801 samples/sec | ETA 00:29:43
- 2022-04-13 18:32:45 [INFO] [TRAIN] epoch: 451, iter: 111800/120000, loss: 0.8068, lr: 0.000894, batch_cost: 0.2124, reader_cost: 0.00147, ips: 14.1213 samples/sec | ETA 00:29:02
- 2022-04-13 18:32:58 [INFO] [TRAIN] epoch: 452, iter: 111850/120000, loss: 0.7932, lr: 0.000889, batch_cost: 0.2505, reader_cost: 0.05285, ips: 11.9783 samples/sec | ETA 00:34:01
- 2022-04-13 18:33:09 [INFO] [TRAIN] epoch: 452, iter: 111900/120000, loss: 0.8008, lr: 0.000884, batch_cost: 0.2186, reader_cost: 0.00160, ips: 13.7214 samples/sec | ETA 00:29:30
- 2022-04-13 18:33:19 [INFO] [TRAIN] epoch: 452, iter: 111950/120000, loss: 0.7995, lr: 0.000879, batch_cost: 0.2122, reader_cost: 0.00175, ips: 14.1404 samples/sec | ETA 00:28:27
- 2022-04-13 18:33:30 [INFO] [TRAIN] epoch: 452, iter: 112000/120000, loss: 0.7877, lr: 0.000874, batch_cost: 0.2115, reader_cost: 0.00096, ips: 14.1876 samples/sec | ETA 00:28:11
- 2022-04-13 18:33:30 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1981 - reader cost: 0.1528
- 2022-04-13 18:33:55 [INFO] [EVAL] #Images: 500 mIoU: 0.7969 Acc: 0.9630 Kappa: 0.9520 Dice: 0.8811
- 2022-04-13 18:33:55 [INFO] [EVAL] Class IoU:
- [0.9827 0.8593 0.93 0.5379 0.6327 0.6651 0.7368 0.8105 0.9247 0.636
- 0.9521 0.8298 0.6585 0.955 0.8425 0.9058 0.8238 0.6673 0.7899]
- 2022-04-13 18:33:55 [INFO] [EVAL] Class Precision:
- [0.9917 0.9203 0.9617 0.8264 0.8111 0.8393 0.852 0.9141 0.9521 0.8149
- 0.9723 0.8886 0.8169 0.9745 0.9402 0.9613 0.9499 0.8289 0.8745]
- 2022-04-13 18:33:55 [INFO] [EVAL] Class Recall:
- [0.9909 0.9284 0.9657 0.6064 0.7421 0.7622 0.845 0.8773 0.9699 0.7433
- 0.9786 0.9261 0.7726 0.9795 0.8902 0.9401 0.8612 0.7739 0.8909]
- 2022-04-13 18:33:56 [INFO] [EVAL] The model with the best validation mIoU (0.7969) was saved at iter 112000.
- 2022-04-13 18:34:06 [INFO] [TRAIN] epoch: 452, iter: 112050/120000, loss: 0.7953, lr: 0.000869, batch_cost: 0.2000, reader_cost: 0.00107, ips: 14.9967 samples/sec | ETA 00:26:30
- 2022-04-13 18:34:19 [INFO] [TRAIN] epoch: 453, iter: 112100/120000, loss: 0.7953, lr: 0.000864, batch_cost: 0.2559, reader_cost: 0.06083, ips: 11.7225 samples/sec | ETA 00:33:41
- 2022-04-13 18:34:29 [INFO] [TRAIN] epoch: 453, iter: 112150/120000, loss: 0.7934, lr: 0.000859, batch_cost: 0.2033, reader_cost: 0.00113, ips: 14.7545 samples/sec | ETA 00:26:36
- 2022-04-13 18:34:39 [INFO] [TRAIN] epoch: 453, iter: 112200/120000, loss: 0.8007, lr: 0.000854, batch_cost: 0.1973, reader_cost: 0.00094, ips: 15.2036 samples/sec | ETA 00:25:39
- 2022-04-13 18:34:50 [INFO] [TRAIN] epoch: 453, iter: 112250/120000, loss: 0.7929, lr: 0.000849, batch_cost: 0.2280, reader_cost: 0.00097, ips: 13.1603 samples/sec | ETA 00:29:26
- 2022-04-13 18:35:00 [INFO] [TRAIN] epoch: 453, iter: 112300/120000, loss: 0.8083, lr: 0.000845, batch_cost: 0.2040, reader_cost: 0.00079, ips: 14.7067 samples/sec | ETA 00:26:10
- 2022-04-13 18:35:14 [INFO] [TRAIN] epoch: 454, iter: 112350/120000, loss: 0.7779, lr: 0.000840, batch_cost: 0.2657, reader_cost: 0.06078, ips: 11.2893 samples/sec | ETA 00:33:52
- 2022-04-13 18:35:24 [INFO] [TRAIN] epoch: 454, iter: 112400/120000, loss: 0.7782, lr: 0.000835, batch_cost: 0.2062, reader_cost: 0.00073, ips: 14.5481 samples/sec | ETA 00:26:07
- 2022-04-13 18:35:35 [INFO] [TRAIN] epoch: 454, iter: 112450/120000, loss: 0.8173, lr: 0.000830, batch_cost: 0.2137, reader_cost: 0.00087, ips: 14.0414 samples/sec | ETA 00:26:53
- 2022-04-13 18:35:45 [INFO] [TRAIN] epoch: 454, iter: 112500/120000, loss: 0.7877, lr: 0.000825, batch_cost: 0.2034, reader_cost: 0.00079, ips: 14.7476 samples/sec | ETA 00:25:25
- 2022-04-13 18:35:55 [INFO] [TRAIN] epoch: 454, iter: 112550/120000, loss: 0.8079, lr: 0.000820, batch_cost: 0.2108, reader_cost: 0.00157, ips: 14.2346 samples/sec | ETA 00:26:10
- 2022-04-13 18:36:09 [INFO] [TRAIN] epoch: 455, iter: 112600/120000, loss: 0.7952, lr: 0.000815, batch_cost: 0.2721, reader_cost: 0.06302, ips: 11.0258 samples/sec | ETA 00:33:33
- 2022-04-13 18:36:20 [INFO] [TRAIN] epoch: 455, iter: 112650/120000, loss: 0.7992, lr: 0.000810, batch_cost: 0.2269, reader_cost: 0.00090, ips: 13.2206 samples/sec | ETA 00:27:47
- 2022-04-13 18:36:31 [INFO] [TRAIN] epoch: 455, iter: 112700/120000, loss: 0.7884, lr: 0.000805, batch_cost: 0.2189, reader_cost: 0.00044, ips: 13.7034 samples/sec | ETA 00:26:38
- 2022-04-13 18:36:42 [INFO] [TRAIN] epoch: 455, iter: 112750/120000, loss: 0.8086, lr: 0.000800, batch_cost: 0.2091, reader_cost: 0.00071, ips: 14.3504 samples/sec | ETA 00:25:15
- 2022-04-13 18:36:53 [INFO] [TRAIN] epoch: 455, iter: 112800/120000, loss: 0.7866, lr: 0.000795, batch_cost: 0.2195, reader_cost: 0.00099, ips: 13.6667 samples/sec | ETA 00:26:20
- 2022-04-13 18:37:06 [INFO] [TRAIN] epoch: 456, iter: 112850/120000, loss: 0.8095, lr: 0.000790, batch_cost: 0.2731, reader_cost: 0.06324, ips: 10.9853 samples/sec | ETA 00:32:32
- 2022-04-13 18:37:17 [INFO] [TRAIN] epoch: 456, iter: 112900/120000, loss: 0.7971, lr: 0.000785, batch_cost: 0.2198, reader_cost: 0.00114, ips: 13.6507 samples/sec | ETA 00:26:00
- 2022-04-13 18:37:29 [INFO] [TRAIN] epoch: 456, iter: 112950/120000, loss: 0.7970, lr: 0.000780, batch_cost: 0.2230, reader_cost: 0.00125, ips: 13.4524 samples/sec | ETA 00:26:12
- 2022-04-13 18:37:39 [INFO] [TRAIN] epoch: 456, iter: 113000/120000, loss: 0.7884, lr: 0.000775, batch_cost: 0.2175, reader_cost: 0.00129, ips: 13.7905 samples/sec | ETA 00:25:22
- 2022-04-13 18:37:50 [INFO] [TRAIN] epoch: 456, iter: 113050/120000, loss: 0.8058, lr: 0.000770, batch_cost: 0.2204, reader_cost: 0.00097, ips: 13.6100 samples/sec | ETA 00:25:31
- 2022-04-13 18:38:05 [INFO] [TRAIN] epoch: 457, iter: 113100/120000, loss: 0.7942, lr: 0.000765, batch_cost: 0.2832, reader_cost: 0.05133, ips: 10.5932 samples/sec | ETA 00:32:34
- 2022-04-13 18:38:15 [INFO] [TRAIN] epoch: 457, iter: 113150/120000, loss: 0.7967, lr: 0.000760, batch_cost: 0.2032, reader_cost: 0.00057, ips: 14.7627 samples/sec | ETA 00:23:12
- 2022-04-13 18:38:26 [INFO] [TRAIN] epoch: 457, iter: 113200/120000, loss: 0.8084, lr: 0.000755, batch_cost: 0.2257, reader_cost: 0.00113, ips: 13.2944 samples/sec | ETA 00:25:34
- 2022-04-13 18:38:36 [INFO] [TRAIN] epoch: 457, iter: 113250/120000, loss: 0.8075, lr: 0.000750, batch_cost: 0.2011, reader_cost: 0.00032, ips: 14.9172 samples/sec | ETA 00:22:37
- 2022-04-13 18:38:46 [INFO] [TRAIN] epoch: 457, iter: 113300/120000, loss: 0.7965, lr: 0.000745, batch_cost: 0.2007, reader_cost: 0.00086, ips: 14.9497 samples/sec | ETA 00:22:24
- 2022-04-13 18:39:00 [INFO] [TRAIN] epoch: 458, iter: 113350/120000, loss: 0.8042, lr: 0.000740, batch_cost: 0.2853, reader_cost: 0.05921, ips: 10.5169 samples/sec | ETA 00:31:36
- 2022-04-13 18:39:12 [INFO] [TRAIN] epoch: 458, iter: 113400/120000, loss: 0.7916, lr: 0.000735, batch_cost: 0.2231, reader_cost: 0.00139, ips: 13.4474 samples/sec | ETA 00:24:32
- 2022-04-13 18:39:23 [INFO] [TRAIN] epoch: 458, iter: 113450/120000, loss: 0.7902, lr: 0.000730, batch_cost: 0.2180, reader_cost: 0.00075, ips: 13.7631 samples/sec | ETA 00:23:47
- 2022-04-13 18:39:33 [INFO] [TRAIN] epoch: 458, iter: 113500/120000, loss: 0.8133, lr: 0.000725, batch_cost: 0.2108, reader_cost: 0.00116, ips: 14.2308 samples/sec | ETA 00:22:50
- 2022-04-13 18:39:44 [INFO] [TRAIN] epoch: 458, iter: 113550/120000, loss: 0.7974, lr: 0.000720, batch_cost: 0.2213, reader_cost: 0.00094, ips: 13.5536 samples/sec | ETA 00:23:47
- 2022-04-13 18:39:57 [INFO] [TRAIN] epoch: 459, iter: 113600/120000, loss: 0.7920, lr: 0.000715, batch_cost: 0.2666, reader_cost: 0.05150, ips: 11.2526 samples/sec | ETA 00:28:26
- 2022-04-13 18:40:08 [INFO] [TRAIN] epoch: 459, iter: 113650/120000, loss: 0.7917, lr: 0.000710, batch_cost: 0.2037, reader_cost: 0.00054, ips: 14.7270 samples/sec | ETA 00:21:33
- 2022-04-13 18:40:18 [INFO] [TRAIN] epoch: 459, iter: 113700/120000, loss: 0.7994, lr: 0.000705, batch_cost: 0.2149, reader_cost: 0.00043, ips: 13.9568 samples/sec | ETA 00:22:34
- 2022-04-13 18:40:29 [INFO] [TRAIN] epoch: 459, iter: 113750/120000, loss: 0.8002, lr: 0.000700, batch_cost: 0.2092, reader_cost: 0.00119, ips: 14.3382 samples/sec | ETA 00:21:47
- 2022-04-13 18:40:40 [INFO] [TRAIN] epoch: 459, iter: 113800/120000, loss: 0.7971, lr: 0.000695, batch_cost: 0.2201, reader_cost: 0.00052, ips: 13.6278 samples/sec | ETA 00:22:44
- 2022-04-13 18:40:53 [INFO] [TRAIN] epoch: 460, iter: 113850/120000, loss: 0.7953, lr: 0.000690, batch_cost: 0.2693, reader_cost: 0.06514, ips: 11.1392 samples/sec | ETA 00:27:36
- 2022-04-13 18:41:04 [INFO] [TRAIN] epoch: 460, iter: 113900/120000, loss: 0.8040, lr: 0.000685, batch_cost: 0.2144, reader_cost: 0.00086, ips: 13.9912 samples/sec | ETA 00:21:47
- 2022-04-13 18:41:14 [INFO] [TRAIN] epoch: 460, iter: 113950/120000, loss: 0.7899, lr: 0.000680, batch_cost: 0.2051, reader_cost: 0.00167, ips: 14.6276 samples/sec | ETA 00:20:40
- 2022-04-13 18:41:27 [INFO] [TRAIN] epoch: 460, iter: 114000/120000, loss: 0.7862, lr: 0.000675, batch_cost: 0.2495, reader_cost: 0.00081, ips: 12.0239 samples/sec | ETA 00:24:57
- 2022-04-13 18:41:37 [INFO] [TRAIN] epoch: 460, iter: 114050/120000, loss: 0.8072, lr: 0.000670, batch_cost: 0.2054, reader_cost: 0.00148, ips: 14.6078 samples/sec | ETA 00:20:21
- 2022-04-13 18:41:51 [INFO] [TRAIN] epoch: 461, iter: 114100/120000, loss: 0.7905, lr: 0.000665, batch_cost: 0.2737, reader_cost: 0.05611, ips: 10.9627 samples/sec | ETA 00:26:54
- 2022-04-13 18:42:02 [INFO] [TRAIN] epoch: 461, iter: 114150/120000, loss: 0.7829, lr: 0.000660, batch_cost: 0.2193, reader_cost: 0.00098, ips: 13.6779 samples/sec | ETA 00:21:23
- 2022-04-13 18:42:13 [INFO] [TRAIN] epoch: 461, iter: 114200/120000, loss: 0.7931, lr: 0.000654, batch_cost: 0.2269, reader_cost: 0.00135, ips: 13.2235 samples/sec | ETA 00:21:55
- 2022-04-13 18:42:24 [INFO] [TRAIN] epoch: 461, iter: 114250/120000, loss: 0.8089, lr: 0.000649, batch_cost: 0.2157, reader_cost: 0.00108, ips: 13.9060 samples/sec | ETA 00:20:40
- 2022-04-13 18:42:34 [INFO] [TRAIN] epoch: 461, iter: 114300/120000, loss: 0.7967, lr: 0.000644, batch_cost: 0.2022, reader_cost: 0.00087, ips: 14.8391 samples/sec | ETA 00:19:12
- 2022-04-13 18:42:47 [INFO] [TRAIN] epoch: 462, iter: 114350/120000, loss: 0.8164, lr: 0.000639, batch_cost: 0.2660, reader_cost: 0.05494, ips: 11.2796 samples/sec | ETA 00:25:02
- 2022-04-13 18:42:58 [INFO] [TRAIN] epoch: 462, iter: 114400/120000, loss: 0.8013, lr: 0.000634, batch_cost: 0.2216, reader_cost: 0.00071, ips: 13.5388 samples/sec | ETA 00:20:40
- 2022-04-13 18:43:09 [INFO] [TRAIN] epoch: 462, iter: 114450/120000, loss: 0.7918, lr: 0.000629, batch_cost: 0.2177, reader_cost: 0.00109, ips: 13.7810 samples/sec | ETA 00:20:08
- 2022-04-13 18:43:20 [INFO] [TRAIN] epoch: 462, iter: 114500/120000, loss: 0.7985, lr: 0.000624, batch_cost: 0.2246, reader_cost: 0.00113, ips: 13.3561 samples/sec | ETA 00:20:35
- 2022-04-13 18:43:31 [INFO] [TRAIN] epoch: 462, iter: 114550/120000, loss: 0.7861, lr: 0.000619, batch_cost: 0.2193, reader_cost: 0.00077, ips: 13.6800 samples/sec | ETA 00:19:55
- 2022-04-13 18:43:45 [INFO] [TRAIN] epoch: 463, iter: 114600/120000, loss: 0.7819, lr: 0.000614, batch_cost: 0.2667, reader_cost: 0.05722, ips: 11.2470 samples/sec | ETA 00:24:00
- 2022-04-13 18:43:55 [INFO] [TRAIN] epoch: 463, iter: 114650/120000, loss: 0.7903, lr: 0.000609, batch_cost: 0.2066, reader_cost: 0.00138, ips: 14.5194 samples/sec | ETA 00:18:25
- 2022-04-13 18:44:07 [INFO] [TRAIN] epoch: 463, iter: 114700/120000, loss: 0.7802, lr: 0.000603, batch_cost: 0.2403, reader_cost: 0.00054, ips: 12.4830 samples/sec | ETA 00:21:13
- 2022-04-13 18:44:17 [INFO] [TRAIN] epoch: 463, iter: 114750/120000, loss: 0.8110, lr: 0.000598, batch_cost: 0.2048, reader_cost: 0.00078, ips: 14.6453 samples/sec | ETA 00:17:55
- 2022-04-13 18:44:28 [INFO] [TRAIN] epoch: 463, iter: 114800/120000, loss: 0.8029, lr: 0.000593, batch_cost: 0.2099, reader_cost: 0.00066, ips: 14.2892 samples/sec | ETA 00:18:11
- 2022-04-13 18:44:41 [INFO] [TRAIN] epoch: 464, iter: 114850/120000, loss: 0.8057, lr: 0.000588, batch_cost: 0.2727, reader_cost: 0.04796, ips: 11.0031 samples/sec | ETA 00:23:24
- 2022-04-13 18:44:53 [INFO] [TRAIN] epoch: 464, iter: 114900/120000, loss: 0.7978, lr: 0.000583, batch_cost: 0.2380, reader_cost: 0.00174, ips: 12.6077 samples/sec | ETA 00:20:13
- 2022-04-13 18:45:04 [INFO] [TRAIN] epoch: 464, iter: 114950/120000, loss: 0.7950, lr: 0.000578, batch_cost: 0.2053, reader_cost: 0.00118, ips: 14.6098 samples/sec | ETA 00:17:16
- 2022-04-13 18:45:14 [INFO] [TRAIN] epoch: 464, iter: 115000/120000, loss: 0.7925, lr: 0.000573, batch_cost: 0.2126, reader_cost: 0.00149, ips: 14.1080 samples/sec | ETA 00:17:43
- 2022-04-13 18:45:25 [INFO] [TRAIN] epoch: 464, iter: 115050/120000, loss: 0.7939, lr: 0.000567, batch_cost: 0.2154, reader_cost: 0.00090, ips: 13.9280 samples/sec | ETA 00:17:46
- 2022-04-13 18:45:38 [INFO] [TRAIN] epoch: 465, iter: 115100/120000, loss: 0.7831, lr: 0.000562, batch_cost: 0.2591, reader_cost: 0.05331, ips: 11.5777 samples/sec | ETA 00:21:09
- 2022-04-13 18:45:48 [INFO] [TRAIN] epoch: 465, iter: 115150/120000, loss: 0.7865, lr: 0.000557, batch_cost: 0.2067, reader_cost: 0.00122, ips: 14.5146 samples/sec | ETA 00:16:42
- 2022-04-13 18:45:58 [INFO] [TRAIN] epoch: 465, iter: 115200/120000, loss: 0.7926, lr: 0.000552, batch_cost: 0.2011, reader_cost: 0.00063, ips: 14.9215 samples/sec | ETA 00:16:05
- 2022-04-13 18:46:08 [INFO] [TRAIN] epoch: 465, iter: 115250/120000, loss: 0.7988, lr: 0.000547, batch_cost: 0.1989, reader_cost: 0.00116, ips: 15.0795 samples/sec | ETA 00:15:44
- 2022-04-13 18:46:20 [INFO] [TRAIN] epoch: 465, iter: 115300/120000, loss: 0.7787, lr: 0.000542, batch_cost: 0.2319, reader_cost: 0.00086, ips: 12.9380 samples/sec | ETA 00:18:09
- 2022-04-13 18:46:34 [INFO] [TRAIN] epoch: 466, iter: 115350/120000, loss: 0.7841, lr: 0.000536, batch_cost: 0.2726, reader_cost: 0.06273, ips: 11.0048 samples/sec | ETA 00:21:07
- 2022-04-13 18:46:44 [INFO] [TRAIN] epoch: 466, iter: 115400/120000, loss: 0.7842, lr: 0.000531, batch_cost: 0.2100, reader_cost: 0.00105, ips: 14.2824 samples/sec | ETA 00:16:06
- 2022-04-13 18:46:55 [INFO] [TRAIN] epoch: 466, iter: 115450/120000, loss: 0.7849, lr: 0.000526, batch_cost: 0.2116, reader_cost: 0.00055, ips: 14.1772 samples/sec | ETA 00:16:02
- 2022-04-13 18:47:05 [INFO] [TRAIN] epoch: 466, iter: 115500/120000, loss: 0.8027, lr: 0.000521, batch_cost: 0.2056, reader_cost: 0.00111, ips: 14.5935 samples/sec | ETA 00:15:25
- 2022-04-13 18:47:15 [INFO] [TRAIN] epoch: 466, iter: 115550/120000, loss: 0.7822, lr: 0.000516, batch_cost: 0.1995, reader_cost: 0.00132, ips: 15.0399 samples/sec | ETA 00:14:47
- 2022-04-13 18:47:30 [INFO] [TRAIN] epoch: 467, iter: 115600/120000, loss: 0.7937, lr: 0.000510, batch_cost: 0.2917, reader_cost: 0.05822, ips: 10.2855 samples/sec | ETA 00:21:23
- 2022-04-13 18:47:40 [INFO] [TRAIN] epoch: 467, iter: 115650/120000, loss: 0.7886, lr: 0.000505, batch_cost: 0.2022, reader_cost: 0.00104, ips: 14.8393 samples/sec | ETA 00:14:39
- 2022-04-13 18:47:52 [INFO] [TRAIN] epoch: 467, iter: 115700/120000, loss: 0.7917, lr: 0.000500, batch_cost: 0.2545, reader_cost: 0.00090, ips: 11.7860 samples/sec | ETA 00:18:14
- 2022-04-13 18:48:02 [INFO] [TRAIN] epoch: 467, iter: 115750/120000, loss: 0.7905, lr: 0.000495, batch_cost: 0.1986, reader_cost: 0.00082, ips: 15.1023 samples/sec | ETA 00:14:04
- 2022-04-13 18:48:14 [INFO] [TRAIN] epoch: 467, iter: 115800/120000, loss: 0.7951, lr: 0.000490, batch_cost: 0.2280, reader_cost: 0.00060, ips: 13.1604 samples/sec | ETA 00:15:57
- 2022-04-13 18:48:27 [INFO] [TRAIN] epoch: 468, iter: 115850/120000, loss: 0.7986, lr: 0.000484, batch_cost: 0.2693, reader_cost: 0.05669, ips: 11.1393 samples/sec | ETA 00:18:37
- 2022-04-13 18:48:38 [INFO] [TRAIN] epoch: 468, iter: 115900/120000, loss: 0.7949, lr: 0.000479, batch_cost: 0.2203, reader_cost: 0.00065, ips: 13.6179 samples/sec | ETA 00:15:03
- 2022-04-13 18:48:48 [INFO] [TRAIN] epoch: 468, iter: 115950/120000, loss: 0.7993, lr: 0.000474, batch_cost: 0.2033, reader_cost: 0.00068, ips: 14.7581 samples/sec | ETA 00:13:43
- 2022-04-13 18:48:58 [INFO] [TRAIN] epoch: 468, iter: 116000/120000, loss: 0.7939, lr: 0.000468, batch_cost: 0.2003, reader_cost: 0.00094, ips: 14.9752 samples/sec | ETA 00:13:21
- 2022-04-13 18:48:58 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 25s - batch_cost: 0.1978 - reader cost: 0.1537
- 2022-04-13 18:49:23 [INFO] [EVAL] #Images: 500 mIoU: 0.7935 Acc: 0.9635 Kappa: 0.9526 Dice: 0.8793
- 2022-04-13 18:49:23 [INFO] [EVAL] Class IoU:
- [0.9835 0.8617 0.9293 0.5634 0.63 0.6694 0.7372 0.8068 0.9268 0.6341
- 0.9521 0.832 0.6625 0.9556 0.7868 0.8908 0.8021 0.6638 0.789 ]
- 2022-04-13 18:49:23 [INFO] [EVAL] Class Precision:
- [0.9916 0.9229 0.9578 0.8818 0.8063 0.8411 0.8491 0.9177 0.9562 0.8302
- 0.9698 0.9031 0.8032 0.975 0.9309 0.945 0.93 0.8236 0.8713]
- 2022-04-13 18:49:23 [INFO] [EVAL] Class Recall:
- [0.9918 0.9286 0.9689 0.6094 0.7423 0.7662 0.8484 0.8698 0.9679 0.7286
- 0.9811 0.9136 0.7909 0.9796 0.8355 0.9395 0.8537 0.7738 0.8931]
- 2022-04-13 18:49:24 [INFO] [EVAL] The model with the best validation mIoU (0.7969) was saved at iter 112000.
- 2022-04-13 18:49:35 [INFO] [TRAIN] epoch: 468, iter: 116050/120000, loss: 0.7845, lr: 0.000463, batch_cost: 0.2238, reader_cost: 0.00089, ips: 13.4026 samples/sec | ETA 00:14:44
- 2022-04-13 18:49:49 [INFO] [TRAIN] epoch: 469, iter: 116100/120000, loss: 0.7902, lr: 0.000458, batch_cost: 0.2706, reader_cost: 0.05416, ips: 11.0885 samples/sec | ETA 00:17:35
- 2022-04-13 18:49:59 [INFO] [TRAIN] epoch: 469, iter: 116150/120000, loss: 0.7901, lr: 0.000453, batch_cost: 0.2061, reader_cost: 0.00096, ips: 14.5592 samples/sec | ETA 00:13:13
- 2022-04-13 18:50:09 [INFO] [TRAIN] epoch: 469, iter: 116200/120000, loss: 0.7869, lr: 0.000447, batch_cost: 0.1985, reader_cost: 0.00156, ips: 15.1144 samples/sec | ETA 00:12:34
- 2022-04-13 18:50:20 [INFO] [TRAIN] epoch: 469, iter: 116250/120000, loss: 0.7864, lr: 0.000442, batch_cost: 0.2193, reader_cost: 0.00076, ips: 13.6796 samples/sec | ETA 00:13:42
- 2022-04-13 18:50:31 [INFO] [TRAIN] epoch: 469, iter: 116300/120000, loss: 0.7879, lr: 0.000437, batch_cost: 0.2151, reader_cost: 0.00176, ips: 13.9492 samples/sec | ETA 00:13:15
- 2022-04-13 18:50:44 [INFO] [TRAIN] epoch: 470, iter: 116350/120000, loss: 0.7968, lr: 0.000431, batch_cost: 0.2683, reader_cost: 0.05639, ips: 11.1801 samples/sec | ETA 00:16:19
- 2022-04-13 18:50:55 [INFO] [TRAIN] epoch: 470, iter: 116400/120000, loss: 0.8003, lr: 0.000426, batch_cost: 0.2229, reader_cost: 0.00090, ips: 13.4608 samples/sec | ETA 00:13:22
- 2022-04-13 18:51:06 [INFO] [TRAIN] epoch: 470, iter: 116450/120000, loss: 0.7963, lr: 0.000421, batch_cost: 0.2104, reader_cost: 0.00107, ips: 14.2607 samples/sec | ETA 00:12:26
- 2022-04-13 18:51:16 [INFO] [TRAIN] epoch: 470, iter: 116500/120000, loss: 0.7848, lr: 0.000415, batch_cost: 0.2023, reader_cost: 0.00043, ips: 14.8264 samples/sec | ETA 00:11:48
- 2022-04-13 18:51:27 [INFO] [TRAIN] epoch: 470, iter: 116550/120000, loss: 0.7737, lr: 0.000410, batch_cost: 0.2226, reader_cost: 0.00170, ips: 13.4742 samples/sec | ETA 00:12:48
- 2022-04-13 18:51:41 [INFO] [TRAIN] epoch: 471, iter: 116600/120000, loss: 0.7854, lr: 0.000405, batch_cost: 0.2741, reader_cost: 0.06638, ips: 10.9434 samples/sec | ETA 00:15:32
- 2022-04-13 18:51:50 [INFO] [TRAIN] epoch: 471, iter: 116650/120000, loss: 0.7936, lr: 0.000399, batch_cost: 0.1987, reader_cost: 0.00137, ips: 15.1016 samples/sec | ETA 00:11:05
- 2022-04-13 18:52:01 [INFO] [TRAIN] epoch: 471, iter: 116700/120000, loss: 0.7739, lr: 0.000394, batch_cost: 0.2013, reader_cost: 0.00060, ips: 14.9012 samples/sec | ETA 00:11:04
- 2022-04-13 18:52:11 [INFO] [TRAIN] epoch: 471, iter: 116750/120000, loss: 0.7898, lr: 0.000389, batch_cost: 0.2139, reader_cost: 0.00131, ips: 14.0229 samples/sec | ETA 00:11:35
- 2022-04-13 18:52:21 [INFO] [TRAIN] epoch: 471, iter: 116800/120000, loss: 0.7949, lr: 0.000383, batch_cost: 0.1964, reader_cost: 0.00096, ips: 15.2777 samples/sec | ETA 00:10:28
- 2022-04-13 18:52:35 [INFO] [TRAIN] epoch: 472, iter: 116850/120000, loss: 0.7816, lr: 0.000378, batch_cost: 0.2736, reader_cost: 0.06256, ips: 10.9664 samples/sec | ETA 00:14:21
- 2022-04-13 18:52:45 [INFO] [TRAIN] epoch: 472, iter: 116900/120000, loss: 0.7872, lr: 0.000372, batch_cost: 0.2079, reader_cost: 0.00055, ips: 14.4281 samples/sec | ETA 00:10:44
- 2022-04-13 18:52:56 [INFO] [TRAIN] epoch: 472, iter: 116950/120000, loss: 0.7895, lr: 0.000367, batch_cost: 0.2137, reader_cost: 0.00142, ips: 14.0357 samples/sec | ETA 00:10:51
- 2022-04-13 18:53:06 [INFO] [TRAIN] epoch: 472, iter: 117000/120000, loss: 0.7720, lr: 0.000362, batch_cost: 0.2029, reader_cost: 0.00116, ips: 14.7839 samples/sec | ETA 00:10:08
- 2022-04-13 18:53:16 [INFO] [TRAIN] epoch: 472, iter: 117050/120000, loss: 0.8055, lr: 0.000356, batch_cost: 0.1969, reader_cost: 0.00068, ips: 15.2343 samples/sec | ETA 00:09:40
- 2022-04-13 18:53:30 [INFO] [TRAIN] epoch: 473, iter: 117100/120000, loss: 0.7960, lr: 0.000351, batch_cost: 0.2930, reader_cost: 0.05467, ips: 10.2405 samples/sec | ETA 00:14:09
- 2022-04-13 18:53:41 [INFO] [TRAIN] epoch: 473, iter: 117150/120000, loss: 0.7979, lr: 0.000345, batch_cost: 0.2095, reader_cost: 0.00087, ips: 14.3217 samples/sec | ETA 00:09:56
- 2022-04-13 18:53:52 [INFO] [TRAIN] epoch: 473, iter: 117200/120000, loss: 0.8204, lr: 0.000340, batch_cost: 0.2129, reader_cost: 0.00051, ips: 14.0899 samples/sec | ETA 00:09:56
- 2022-04-13 18:54:02 [INFO] [TRAIN] epoch: 473, iter: 117250/120000, loss: 0.7925, lr: 0.000334, batch_cost: 0.2120, reader_cost: 0.00101, ips: 14.1534 samples/sec | ETA 00:09:42
- 2022-04-13 18:54:12 [INFO] [TRAIN] epoch: 473, iter: 117300/120000, loss: 0.7905, lr: 0.000329, batch_cost: 0.1984, reader_cost: 0.00110, ips: 15.1192 samples/sec | ETA 00:08:55
- 2022-04-13 18:54:26 [INFO] [TRAIN] epoch: 474, iter: 117350/120000, loss: 0.7734, lr: 0.000323, batch_cost: 0.2746, reader_cost: 0.05253, ips: 10.9235 samples/sec | ETA 00:12:07
- 2022-04-13 18:54:36 [INFO] [TRAIN] epoch: 474, iter: 117400/120000, loss: 0.7969, lr: 0.000318, batch_cost: 0.2071, reader_cost: 0.00073, ips: 14.4861 samples/sec | ETA 00:08:58
- 2022-04-13 18:54:47 [INFO] [TRAIN] epoch: 474, iter: 117450/120000, loss: 0.7792, lr: 0.000312, batch_cost: 0.2095, reader_cost: 0.00143, ips: 14.3194 samples/sec | ETA 00:08:54
- 2022-04-13 18:54:57 [INFO] [TRAIN] epoch: 474, iter: 117500/120000, loss: 0.8041, lr: 0.000307, batch_cost: 0.2050, reader_cost: 0.00086, ips: 14.6354 samples/sec | ETA 00:08:32
- 2022-04-13 18:55:07 [INFO] [TRAIN] epoch: 474, iter: 117550/120000, loss: 0.7834, lr: 0.000301, batch_cost: 0.2100, reader_cost: 0.00083, ips: 14.2824 samples/sec | ETA 00:08:34
- 2022-04-13 18:55:22 [INFO] [TRAIN] epoch: 475, iter: 117600/120000, loss: 0.7777, lr: 0.000296, batch_cost: 0.2931, reader_cost: 0.06380, ips: 10.2357 samples/sec | ETA 00:11:43
- 2022-04-13 18:55:33 [INFO] [TRAIN] epoch: 475, iter: 117650/120000, loss: 0.7846, lr: 0.000290, batch_cost: 0.2108, reader_cost: 0.00094, ips: 14.2341 samples/sec | ETA 00:08:15
- 2022-04-13 18:55:43 [INFO] [TRAIN] epoch: 475, iter: 117700/120000, loss: 0.7880, lr: 0.000285, batch_cost: 0.2061, reader_cost: 0.00073, ips: 14.5595 samples/sec | ETA 00:07:53
- 2022-04-13 18:55:54 [INFO] [TRAIN] epoch: 475, iter: 117750/120000, loss: 0.7999, lr: 0.000279, batch_cost: 0.2118, reader_cost: 0.00107, ips: 14.1655 samples/sec | ETA 00:07:56
- 2022-04-13 18:56:03 [INFO] [TRAIN] epoch: 475, iter: 117800/120000, loss: 0.7916, lr: 0.000274, batch_cost: 0.1893, reader_cost: 0.00074, ips: 15.8437 samples/sec | ETA 00:06:56
- 2022-04-13 18:56:17 [INFO] [TRAIN] epoch: 476, iter: 117850/120000, loss: 0.7982, lr: 0.000268, batch_cost: 0.2826, reader_cost: 0.05105, ips: 10.6146 samples/sec | ETA 00:10:07
- 2022-04-13 18:56:27 [INFO] [TRAIN] epoch: 476, iter: 117900/120000, loss: 0.7809, lr: 0.000262, batch_cost: 0.2057, reader_cost: 0.00083, ips: 14.5873 samples/sec | ETA 00:07:11
- 2022-04-13 18:56:38 [INFO] [TRAIN] epoch: 476, iter: 117950/120000, loss: 0.7804, lr: 0.000257, batch_cost: 0.2109, reader_cost: 0.00042, ips: 14.2243 samples/sec | ETA 00:07:12
- 2022-04-13 18:56:49 [INFO] [TRAIN] epoch: 476, iter: 118000/120000, loss: 0.7814, lr: 0.000251, batch_cost: 0.2144, reader_cost: 0.00141, ips: 13.9906 samples/sec | ETA 00:07:08
- 2022-04-13 18:57:02 [INFO] [TRAIN] epoch: 477, iter: 118050/120000, loss: 0.7964, lr: 0.000245, batch_cost: 0.2709, reader_cost: 0.05814, ips: 11.0722 samples/sec | ETA 00:08:48
- 2022-04-13 18:57:14 [INFO] [TRAIN] epoch: 477, iter: 118100/120000, loss: 0.7984, lr: 0.000240, batch_cost: 0.2250, reader_cost: 0.00222, ips: 13.3305 samples/sec | ETA 00:07:07
- 2022-04-13 18:57:24 [INFO] [TRAIN] epoch: 477, iter: 118150/120000, loss: 0.7771, lr: 0.000234, batch_cost: 0.2038, reader_cost: 0.00120, ips: 14.7176 samples/sec | ETA 00:06:17
- 2022-04-13 18:57:34 [INFO] [TRAIN] epoch: 477, iter: 118200/120000, loss: 0.7921, lr: 0.000228, batch_cost: 0.2075, reader_cost: 0.00069, ips: 14.4546 samples/sec | ETA 00:06:13
- 2022-04-13 18:57:45 [INFO] [TRAIN] epoch: 477, iter: 118250/120000, loss: 0.7791, lr: 0.000223, batch_cost: 0.2232, reader_cost: 0.00069, ips: 13.4393 samples/sec | ETA 00:06:30
- 2022-04-13 18:57:58 [INFO] [TRAIN] epoch: 478, iter: 118300/120000, loss: 0.7868, lr: 0.000217, batch_cost: 0.2620, reader_cost: 0.06088, ips: 11.4518 samples/sec | ETA 00:07:25
- 2022-04-13 18:58:09 [INFO] [TRAIN] epoch: 478, iter: 118350/120000, loss: 0.7851, lr: 0.000211, batch_cost: 0.2208, reader_cost: 0.00120, ips: 13.5850 samples/sec | ETA 00:06:04
- 2022-04-13 18:58:20 [INFO] [TRAIN] epoch: 478, iter: 118400/120000, loss: 0.7856, lr: 0.000205, batch_cost: 0.2152, reader_cost: 0.00094, ips: 13.9409 samples/sec | ETA 00:05:44
- 2022-04-13 18:58:30 [INFO] [TRAIN] epoch: 478, iter: 118450/120000, loss: 0.7872, lr: 0.000200, batch_cost: 0.2011, reader_cost: 0.00092, ips: 14.9144 samples/sec | ETA 00:05:11
- 2022-04-13 18:58:41 [INFO] [TRAIN] epoch: 478, iter: 118500/120000, loss: 0.7823, lr: 0.000194, batch_cost: 0.2087, reader_cost: 0.00158, ips: 14.3764 samples/sec | ETA 00:05:13
- 2022-04-13 18:58:55 [INFO] [TRAIN] epoch: 479, iter: 118550/120000, loss: 0.7964, lr: 0.000188, batch_cost: 0.2863, reader_cost: 0.06000, ips: 10.4788 samples/sec | ETA 00:06:55
- 2022-04-13 18:59:05 [INFO] [TRAIN] epoch: 479, iter: 118600/120000, loss: 0.7803, lr: 0.000182, batch_cost: 0.2056, reader_cost: 0.00075, ips: 14.5896 samples/sec | ETA 00:04:47
- 2022-04-13 18:59:16 [INFO] [TRAIN] epoch: 479, iter: 118650/120000, loss: 0.7895, lr: 0.000176, batch_cost: 0.2147, reader_cost: 0.00105, ips: 13.9761 samples/sec | ETA 00:04:49
- 2022-04-13 18:59:26 [INFO] [TRAIN] epoch: 479, iter: 118700/120000, loss: 0.7727, lr: 0.000170, batch_cost: 0.2040, reader_cost: 0.00120, ips: 14.7087 samples/sec | ETA 00:04:25
- 2022-04-13 18:59:37 [INFO] [TRAIN] epoch: 479, iter: 118750/120000, loss: 0.7990, lr: 0.000165, batch_cost: 0.2145, reader_cost: 0.00135, ips: 13.9889 samples/sec | ETA 00:04:28
- 2022-04-13 18:59:51 [INFO] [TRAIN] epoch: 480, iter: 118800/120000, loss: 0.7945, lr: 0.000159, batch_cost: 0.2780, reader_cost: 0.05853, ips: 10.7932 samples/sec | ETA 00:05:33
- 2022-04-13 19:00:01 [INFO] [TRAIN] epoch: 480, iter: 118850/120000, loss: 0.7790, lr: 0.000153, batch_cost: 0.2087, reader_cost: 0.00178, ips: 14.3745 samples/sec | ETA 00:04:00
- 2022-04-13 19:00:12 [INFO] [TRAIN] epoch: 480, iter: 118900/120000, loss: 0.7870, lr: 0.000147, batch_cost: 0.2153, reader_cost: 0.00340, ips: 13.9357 samples/sec | ETA 00:03:56
- 2022-04-13 19:00:23 [INFO] [TRAIN] epoch: 480, iter: 118950/120000, loss: 0.7832, lr: 0.000141, batch_cost: 0.2133, reader_cost: 0.00140, ips: 14.0652 samples/sec | ETA 00:03:43
- 2022-04-13 19:00:33 [INFO] [TRAIN] epoch: 480, iter: 119000/120000, loss: 0.7941, lr: 0.000135, batch_cost: 0.2112, reader_cost: 0.00072, ips: 14.2053 samples/sec | ETA 00:03:31
- 2022-04-13 19:00:47 [INFO] [TRAIN] epoch: 481, iter: 119050/120000, loss: 0.7788, lr: 0.000129, batch_cost: 0.2741, reader_cost: 0.05690, ips: 10.9453 samples/sec | ETA 00:04:20
- 2022-04-13 19:00:57 [INFO] [TRAIN] epoch: 481, iter: 119100/120000, loss: 0.7846, lr: 0.000122, batch_cost: 0.2092, reader_cost: 0.00061, ips: 14.3381 samples/sec | ETA 00:03:08
- INFO 2022-04-13 19:04:47,325 launch.py:311] Local processes completed.
- 2022-04-13 19:01:08 [INFO] [TRAIN] epoch: 481, iter: 119150/120000, loss: 0.7971, lr: 0.000116, batch_cost: 0.2106, reader_cost: 0.00120, ips: 14.2465 samples/sec | ETA 00:02:58
- 2022-04-13 19:01:20 [INFO] [TRAIN] epoch: 481, iter: 119200/120000, loss: 0.7903, lr: 0.000110, batch_cost: 0.2401, reader_cost: 0.00149, ips: 12.4948 samples/sec | ETA 00:03:12
- 2022-04-13 19:01:31 [INFO] [TRAIN] epoch: 481, iter: 119250/120000, loss: 0.7932, lr: 0.000104, batch_cost: 0.2306, reader_cost: 0.00091, ips: 13.0107 samples/sec | ETA 00:02:52
- 2022-04-13 19:01:45 [INFO] [TRAIN] epoch: 482, iter: 119300/120000, loss: 0.7900, lr: 0.000098, batch_cost: 0.2650, reader_cost: 0.05940, ips: 11.3226 samples/sec | ETA 00:03:05
- 2022-04-13 19:01:55 [INFO] [TRAIN] epoch: 482, iter: 119350/120000, loss: 0.7974, lr: 0.000091, batch_cost: 0.2063, reader_cost: 0.00088, ips: 14.5438 samples/sec | ETA 00:02:14
- 2022-04-13 19:02:06 [INFO] [TRAIN] epoch: 482, iter: 119400/120000, loss: 0.7868, lr: 0.000085, batch_cost: 0.2190, reader_cost: 0.00079, ips: 13.6970 samples/sec | ETA 00:02:11
- 2022-04-13 19:02:17 [INFO] [TRAIN] epoch: 482, iter: 119450/120000, loss: 0.7827, lr: 0.000079, batch_cost: 0.2166, reader_cost: 0.00106, ips: 13.8500 samples/sec | ETA 00:01:59
- 2022-04-13 19:02:27 [INFO] [TRAIN] epoch: 482, iter: 119500/120000, loss: 0.7820, lr: 0.000072, batch_cost: 0.2000, reader_cost: 0.00182, ips: 15.0002 samples/sec | ETA 00:01:39
- 2022-04-13 19:02:40 [INFO] [TRAIN] epoch: 483, iter: 119550/120000, loss: 0.7773, lr: 0.000066, batch_cost: 0.2721, reader_cost: 0.06384, ips: 11.0241 samples/sec | ETA 00:02:02
- 2022-04-13 19:02:51 [INFO] [TRAIN] epoch: 483, iter: 119600/120000, loss: 0.8027, lr: 0.000059, batch_cost: 0.2027, reader_cost: 0.00075, ips: 14.8012 samples/sec | ETA 00:01:21
- 2022-04-13 19:03:01 [INFO] [TRAIN] epoch: 483, iter: 119650/120000, loss: 0.7965, lr: 0.000052, batch_cost: 0.2163, reader_cost: 0.00097, ips: 13.8673 samples/sec | ETA 00:01:15
- 2022-04-13 19:03:12 [INFO] [TRAIN] epoch: 483, iter: 119700/120000, loss: 0.7817, lr: 0.000046, batch_cost: 0.2126, reader_cost: 0.00114, ips: 14.1090 samples/sec | ETA 00:01:03
- 2022-04-13 19:03:23 [INFO] [TRAIN] epoch: 483, iter: 119750/120000, loss: 0.7868, lr: 0.000039, batch_cost: 0.2111, reader_cost: 0.00128, ips: 14.2125 samples/sec | ETA 00:00:52
- 2022-04-13 19:03:36 [INFO] [TRAIN] epoch: 484, iter: 119800/120000, loss: 0.7889, lr: 0.000032, batch_cost: 0.2713, reader_cost: 0.06081, ips: 11.0576 samples/sec | ETA 00:00:54
- 2022-04-13 19:03:47 [INFO] [TRAIN] epoch: 484, iter: 119850/120000, loss: 0.7883, lr: 0.000025, batch_cost: 0.2146, reader_cost: 0.00066, ips: 13.9802 samples/sec | ETA 00:00:32
- 2022-04-13 19:03:57 [INFO] [TRAIN] epoch: 484, iter: 119900/120000, loss: 0.7889, lr: 0.000017, batch_cost: 0.2045, reader_cost: 0.00108, ips: 14.6724 samples/sec | ETA 00:00:20
- 2022-04-13 19:04:08 [INFO] [TRAIN] epoch: 484, iter: 119950/120000, loss: 0.8007, lr: 0.000009, batch_cost: 0.2094, reader_cost: 0.00095, ips: 14.3275 samples/sec | ETA 00:00:10
- 2022-04-13 19:04:18 [INFO] [TRAIN] epoch: 484, iter: 120000/120000, loss: 0.7907, lr: 0.000000, batch_cost: 0.2092, reader_cost: 0.00080, ips: 14.3432 samples/sec | ETA 00:00:00
- 2022-04-13 19:04:18 [INFO] Start evaluating (total_samples: 500, total_iters: 125)...
- 125/125 - 24s - batch_cost: 0.1958 - reader cost: 0.1506
- 2022-04-13 19:04:43 [INFO] [EVAL] #Images: 500 mIoU: 0.7985 Acc: 0.9638 Kappa: 0.9530 Dice: 0.8825
- 2022-04-13 19:04:43 [INFO] [EVAL] Class IoU:
- [0.9836 0.863 0.9302 0.5629 0.6355 0.6724 0.738 0.8103 0.9267 0.6401
- 0.9521 0.8335 0.6664 0.9561 0.8302 0.9042 0.8032 0.6741 0.7886]
- 2022-04-13 19:04:43 [INFO] [EVAL] Class Precision:
- [0.9918 0.9227 0.9589 0.8684 0.8272 0.8405 0.853 0.9206 0.9559 0.825
- 0.9708 0.9015 0.7845 0.9752 0.9395 0.9567 0.946 0.8383 0.8721]
- 2022-04-13 19:04:43 [INFO] [EVAL] Class Recall:
- [0.9916 0.9303 0.9689 0.6154 0.7328 0.7707 0.8456 0.8712 0.9681 0.7407
- 0.9801 0.917 0.8157 0.9799 0.877 0.9428 0.8418 0.7748 0.8918]
- 2022-04-13 19:04:44 [INFO] [EVAL] The model with the best validation mIoU (0.7985) was saved at iter 120000.
- <class 'paddle.nn.layer.conv.Conv2D'>'s flops has been counted
- Customize Function has been applied to <class 'paddle.nn.layer.norm.SyncBatchNorm'>
- <class 'paddle.nn.layer.activation.ReLU'>'s flops has been counted
- <class 'paddle.nn.layer.pooling.AvgPool2D'>'s flops has been counted
- <class 'paddle.nn.layer.pooling.AdaptiveAvgPool2D'>'s flops has been counted
- Total Flops: 71746173952 Total Params: 20186323
- /mnt
- [INFO]: train job success!
|