OpenPointCloud
OpenPointCloud is An Open-Source Point Cloud Algorithm Library based on Deep Learning. We collect the algorithms on the area of point cloud compression, process, and analysis. For these methods, we introduce their principles and contributions, as well as provide source codes implemented with different deep learning programming frameworks, such as TensorFlow, Pytorch and TensorLayer. We also conduct a comprehensive benchmarking test to systematically evaluate the performances of all these methods. We provide analyses and comparisons of their performances according to their categories and draw constructive conclusions.
1.Point Cloud Compression
Figure 1: Chronological overview and publishers of the state-of-the-art point cloud compression (PCC) methods.
1.1 lossy compression
1.1.1 pcc_geo_cnn_v1
- ICIP 2019, geometry compression.
- It is the first work to combine arithmetic coding with convolutional neural
network (CNN) and makes entropy coding applicable.
- Code in the framework of tensorflow & pytorch are provided.
- For more information, please go to pcc_geo_cnn_v1.
Figure 2: Network structure of pcc_geo_cnn_v1
1.1.2 pcc_geo_cnn_v2
- MMSP 2020, geometry compression.
- Based on pcc_geo_cnn_v1, improvements on 5 aspects are made on pcc_geo_cnn_v2, and ablation tests are carried out accordingly.
- Code in the framework of tensorflow & pytorch are provided.
- For more information, please go to pcc_geo_cnn_v2.
1.1.3 PCGCv1
- TCSVT 2021, geometry compression.
- PCGCv1 is inherited from pcc_geo_cnn_v2. It uses Voxception-ResNet-based blocks for analysis and synthesis transform to make network deeper and avoid gradient vanishing in training.
- Code in the framework of tensorflow & pytorch & tensorlayer are provided.
- For more information, please go to PCGCv1.
Figure 3: Network structure of pcc_geo_cnn_v2/PCGCv1. The main difference of the two methods is that the hyper prior network of PCGCv1 provides both μ and σ, while pcc_geo_cnn_v2 only provides σ.
1.1.4 PCGCv2
- DCC 2021, geometry compression.
- PCGCv2 is upgraded from PCGCv1. It introduces Inception-Residual Network (IRN) unit, uses the multi-scale strategy, and applies sparse convolutions mechanism.
- Code in the framework of pytorch is provided.
- For more information, please go to PCGCv2.
Figure 4: Network structure of PCGCv2.
1.1.5 pcc_attr_folding
- ICIP 2020, attribute compression.
- It proposes a method to map 3D coordinates into 2D grids through overfitting deep learning, which makes it possible to encode point cloud attributes with conventional 2D image codec.
- Code in the framework of tensorflow & tensorlayer is provided.
- For more information, please go to pcc_attr_folding.
Figure 5: Proposed system of pcc_attr_folding.
1.1.6 NGS_PCC
邓宇更新一下关于这个算法的内容
1.2 lossless compression
1.2.1 VoxelDNN
- TCSVT 2021, geometry compression.
- It applies a deep generative model which contains masked convolutions to approximate the occupancy probability for arithmetic coding.
- Code in the framework of tensorflow & tensorlayer are provided.
- For more information, please go to VoxelDNN.
Figure 6: Working flow of VoxelDNN
1.2.2 MSVoxelDNN
- ICMEW 2021, geometry compression.
- It leverages multi-scale structures which encodes the subblocks in a course-to-fine order. Using the multi-scale structures, it avoids high time complexity problem generated by voxel-by-voxel decoding.
- Code in the framework of pytorch & tensorlayer are provided.
- For more information, please go to MSVoxelDNN.
Figure 7: Working flow of MSVoxelDNN
1.2.3 NNCTX
- MMSP 2021, geometry compression.
- It designs a coding process that realizes efficient decoding and avoids doing it voxel-by-voxel like VoxelDNN.
- Code in the framework of pytorch & tensorflow are provided.
- For more information, please go to NNCTX.
Figure 8: Working flow of NNCTX
1.2.4 OctAttention
- AAAI 2022, geometry compression.
- It combines octree and transformer together to predict the distribution for entropy coding. As octree structure consumes less space than voxelization, the performance of OctAttention is state-of-the-art on compression rate by far.
- Code in the framework of pytorch & tensorflow are provided.
- For more information, please go to OctAttention.
Figure 9: Working flow of OctAttention
1.3 Post-Process
- ICME 2022.
- It can improve the quality of the point cloud after reconstruction from compression of different scales. The network is formed by a series of down-sample and up-sample convolutional modules to effectively study the geometry feature of point clouds.
- Code in the framework of pytorch & tensorflow are provided.
- For more information, please go to Post-Process.
Figure 10: The architecture of the post-process model. We first split the point cloud into cubes. Then the neural network is used to predict the occupancy probability for each voxel. Finally, we obtain the refined point clouds based on the predicted probabilities.
1.4 Benchmarking Results and Analysis
1.4.1 Datasets
In order to completely evaluate all these methods, we perform tests over 3 publicly available datasets: 8i voxelized full bodies version 2(8iVFBv2), Microsoft voxelized upper bodies (MVUB), and Owlii dynamic human mesh. These PC files cover different bit widths and scales of point numbers. For more details, please go to PCC_benchmark_testsets.
1.4.2 Lossy Geometry PCC Methods Evaluation
For lossy methods, in order to jointly evaluate the bpp and D1, D2 metrics, we calculate the Bjøntegaard delta PSNR (BD-PSNR) gains as shown in Table 3.
