Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
linyebin123 982d0cf7a0 | 1 year ago | |
---|---|---|
auto | 1 year ago | |
data | 1 year ago | |
models | 1 year ago | |
opts | 1 year ago | |
preprocess | 1 year ago | |
scripts | 1 year ago | |
utils | 1 year ago | |
.gitignore | 1 year ago | |
LICENSE | 1 year ago | |
README.md | 1 year ago | |
test.py | 1 year ago | |
train_baseline.py | 1 year ago | |
train_miss.py | 1 year ago |
This repo implements the Missing Modality Imagination Network(MMIN) for the following paper:
"Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities"
python 3.7.0
pytorch >= 1.0.0
First you should change the data folder path in data/config
and preprocess your data follwing the code in preprocess/
.
The preprocess of feature was done handcrafted in several steps, we will make it a automatical running script in the next update. You can download the preprocessed feature to run the code.
For Training MMIN on IEMOCAP:
First training a model fusion model with all audio, visual and lexical modality as the pretrained encoder.
bash scripts/CAP_utt_fusion.sh AVL [num_of_expr] [GPU_index]
Then
bash scripts/CAP_mmin.sh [num_of_expr] [GPU_index]
For Training MMIN on MSP-improv:
bash scripts/MSP_utt_fusion.sh AVL [num_of_expr] [GPU_index]
bash scripts/MSP_mmin.sh [num_of_expr] [GPU_index]
Note that you can run the code with default hyper-parameters defined in shell scripts, for changing these arguments, please refer to options/get_opt.py and the modify_commandline_options
method of each model you choose.
Baidu Yun Link
IEMOCAP A V L modality Features
链接: https://pan.baidu.com/s/1WmuqNlvcs5XzLKfz5i4iqQ 提取码: gn6w
Google Drive Link
https://drive.google.com/file/d/1X5wjY-eMnLPV2qkFaaRi9ZPkrMcCAv7Q/view?usp=sharing
Baidu Yun Link
MSP A V L modality Features
链接: https://pan.baidu.com/s/17E44x84pdR2AQIts0aJfKg 提取码: 6dzq
MIT license.
Copyright (c) 2021 AIM3-RUC lab, School of Information, Renmin University of China.
If you find our paper and this code usefull, please consider cite
@inproceedings{zhao2021missing,
title={Missing modality imagination network for emotion recognition with uncertain missing modalities},
author={Zhao, Jinming and Li, Ruichen and Jin, Qin},
booktitle={Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
pages={2608--2618},
year={2021}
}
No Description
Python Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》