Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
jeffding ce74a2ab6f | 1 year ago | |
---|---|---|
.. | ||
Data | 1 year ago | |
models | 1 year ago | |
affinity_fusion.py | 1 year ago | |
pretrain_affinity_fusion.py | 1 year ago | |
readme.md | 1 year ago | |
tune_result_20220206_151634628_pid46371.json | 1 year ago | |
tune_result_20220206_152116501_pid50005.json | 1 year ago | |
tune_result_20220207_06513027_pid39016.json | 1 year ago | |
tune_result_20220208_082839633_pid132608.json | 1 year ago |
This repository contains the implementation of the paper "Deep multimodal subspace clustering networks" by Mahdi Abavisani and Vishal M. Patel. The paper was posted on arXiv in May 2018.
"Deep multimodal subspace clustering networks" (DMSC) investigated various fusion methods for the task of multimodal subspace clustering, and suggested a new fusion technique called "affinity fusion" as the idea of integrating complementary information from two modalities with respect to the similarities between datapoints across different modalities.
Please use the following to refer to this work in publications:
@ARTICLE{8488484,
author={M. {Abavisani} and V. M. {Patel}},
journal={IEEE Journal of Selected Topics in Signal Processing},
title={Deep Multimodal Subspace Clustering Networks},
year={2018},
volume={12},
number={6},
pages={1601-1614},
doi={10.1109/JSTSP.2018.2875385},
ISSN={1932-4553},
month={Dec},}
Tensorflow, numpy, sklearn, munkres, scipy.
Resize the input images of all the modalities to 32 × 32, and rescale them to have pixel values between 0 and 255. This is for keeping the hyperparameter selections suggested in Deep subspace clustering networks valid.
Save the data in a .mat
file that includes verctorized modalities as separate matrices with the names modality_0
,modality_1
, ... ; labels in a vector with the name Labels
; and number of modalities in the variable num_modalities
.
A sample preprocessed dataset is available in: Data/EYB_fc.mat
Run affinity_fusion.py
to do mutlimodal subspace clustering. For demo a pretrained model trained on EYB_fc
is avilable in models/EYBfc_af.ckpt
Run the demo as:
python affinity_fusion.py --mat EYB_fc --model EYBfc_af
Run pretrain_affinity_fusion.py
to pretrain your networks.
For example:
python pretrain_affinity_fusion.py --mat EYB_fc --model mymodel --epoch 100000
Huwei Ascend 910 Code
Text Unity3D Asset nesC Python Jupyter Notebook other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》