MMSegmentation provides pre-trained models for semantic segmentation in Model Zoo, and supports multiple standard datasets, including Cityscapes, ADE20K, etc.
This note will show how to use existing models to inference on given images.
As for how to test existing models on standard datasets, please see this guide
MMSegmentation provides several interfaces for users to easily use pre-trained models for inference.
We provide the most convenient way to use the model in MMSegmentation MMSegInferencer
. You can get segmentation mask for an image with only 3 lines of code.
The following example shows how to use MMSegInferencer
to perform inference on a single image.
>>> from mmseg.apis import MMSegInferencer
>>> # Load models into memory
>>> inferencer = MMSegInferencer(model='deeplabv3plus_r18-d8_4xb2-80k_cityscapes-512x1024')
>>> # Inference
>>> inferencer('demo/demo.png', show=True)
The visualization result should look like:
Moreover, you can use MMSegInferencer
to process a list of images:
# Input a list of images
>>> images = [image1, image2, ...] # image1 can be a file path or a np.ndarray
>>> inferencer(images, show=True, wait_time=0.5) # wait_time is delay time, and 0 means forever
# Or input image directory
>>> images = $IMAGESDIR
>>> inferencer(images, show=True, wait_time=0.5)
# Save visualized rendering color maps and predicted results
# out_dir is the directory to save the output results, img_out_dir and pred_out_dir are subdirectories of out_dir
# to save visualized rendering color maps and predicted results
>>> inferencer(images, out_dir='outputs', img_out_dir='vis', pred_out_dir='pred')
There is a optional parameter of inferencer, return_datasamples
, whose default value is False, and return value of inferencer is a dict
type by default, including 2 keys 'visualization' and 'predictions'.
If return_datasamples=True
inferencer will return SegDataSample
, or list of it.
result = inferencer('demo/demo.png')
# result is a `dict` including 2 keys 'visualization' and 'predictions'
# 'visualization' includes color segmentation map
print(result['visualization'].shape)
# (512, 683, 3)
# 'predictions' includes segmentation mask with label indice
print(result['predictions'].shape)
# (512, 683)
result = inferencer('demo/demo.png', return_datasamples=True)
print(type(result))
# <class 'mmseg.structures.seg_data_sample.SegDataSample'>
# Input a list of images
results = inferencer(images)
# The output is list
print(type(results['visualization']), results['visualization'][0].shape)
# <class 'list'> (512, 683, 3)
print(type(results['predictions']), results['predictions'][0].shape)
# <class 'list'> (512, 683)
results = inferencer(images, return_datasamples=True)
# <class 'list'>
print(type(results[0]))
# <class 'mmseg.structures.seg_data_sample.SegDataSample'>
MMSegInferencer
must be initialized from a model
, which can be a model name or a Config
even a path of config file.
The model names can be found in models' metafile (configs/xxx/metafile.yaml), like one model name of maskformer is maskformer_r50-d32_8xb2-160k_ade20k-512x512
, and if input model name and the weights of the model will be download automatically. Below are other input parameters:
classes
is a list which includes items responding to the label indices. If classes is not defined, visualizer will take cityscapes
classes by default. Defaults to None.cityscapes
by default. Defaults to None.classes
and palette
have higher priority. Defaults to None.MMSegInferencer
supports 4 parameters for visualize prediction, you can use them when call initialized inferencer:
out_dir
, used to save rendering color segmentation mask, so out_dir
must be defined if you would like to save predicted mask. Defaults to 'vis'.The examples of these parameters is in Basic Usage
There is a very easy to list all model names in MMSegmentation
>>> from mmseg.apis import MMSegInferencer
# models is a list of model names, and them will print automatically
>>> models = MMSegInferencer.list_models('mmseg')
Initialize a segmentor from config file.
Parameters:
Path
, or mmengine.Config
) - Config file path or the config object.Returns:
Example:
from mmseg.apis import init_model
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
# initialize model without checkpoint
model = init_model(config_path)
# init model and load checkpoint
model = init_model(config_path, checkpoint_path)
# init model and load checkpoint on CPU
model = init_model(config_path, checkpoint_path, 'cpu')
Inference image(s) with the segmentor.
Parameters:
Returns:
SegDataSample
or list[SegDataSample
]: If imgs is a list or tuple, the same length list type results will be returned, otherwise return the segmentation results directly.Note: SegDataSample is a data structure interface of MMSegmentation, it is used as interfaces between different components. SegDataSample
implement the abstract data element mmengine.structures.BaseDataElement
, please refer to data element documentation in MMEngine for more information.
The attributes in SegDataSample
are divided into several parts:
gt_sem_seg
(PixelData
) - Ground truth of semantic segmentation.pred_sem_seg
(PixelData
) - Prediction of semantic segmentation.seg_logits
(PixelData
) - Predicted logits of semantic segmentation.Note PixelData is the data structure for pixel-level annotations or predictions, please refer to PixelData documentation in MMEngine for more information.
Example:
from mmseg.apis import init_model, inference_model
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
img_path = 'demo/demo.png'
model = init_model(config_path, checkpoint_path)
result = inference_model(model, img_path)
Visualize the segmentation results on the image.
Parameters:
SegDataSample
) - The prediction SegDataSample result.0.5
, must be in (0, 1]
range.True
.True
.0
.True
.None
, the backend storage will not save any data.None
.Returns:
Example:
from mmseg.apis import init_model, inference_model, show_result_pyplot
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
img_path = 'demo/demo.png'
# build the model from a config file and a checkpoint file
model = init_model(config_path, checkpoint_path, device='cuda:0')
# inference on given image
result = inference_model(model, img_path)
# display the segmentation result
vis_image = show_result_pyplot(model, img_path, result)
# save the visualization result, the output image would be found at the path `work_dirs/result.png`
vis_iamge = show_result_pyplot(model, img_path, result, out_file='work_dirs/result.png')
# Modify the time of displaying images, note that 0 is the special value that means "forever"
vis_image = show_result_pyplot(model, img_path, result, wait_time=5)
Note: If your current device doesn't have graphical user interface, it is recommended that setting show
to False
and specify the out_file
or save_dir
to save the results. If you would like to display the result on a window, no special settings are required.
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》