Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
root c18b07e766 | 1 year ago | |
---|---|---|
assets | 1 year ago | |
auxiliary | 1 year ago | |
.gitignore | 1 year ago | |
LICENSE | 1 year ago | |
README.md | 1 year ago | |
dataset.py | 1 year ago | |
install.sh | 1 year ago | |
main.py | 1 year ago | |
network.py | 1 year ago | |
run.sh | 1 year ago |
Official code of the CVPR 2020 paper
Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications
available on arxiv.
Learn a video representation that can generalize to unseen actions.
Semantic information are used as supervision. In particular, the visual representation is mapped into the Word2Vec embedding, where words semantically related are closer to each other in an euclidean sense.
The trained models, used to produce the numbers in the paper, can be downloaded here.
Run install.sh
to get the uncommon libraries (faiss, tensorboardx, joblib) and the latest version of pytorch compatible with cuda 9.2 installed in the docker.
sudo chmod + assets/download_word2vec.sh
./assets/download_word2vec.sh
wget http://imagelab.ing.unimore.it/files/c3d_pytorch/c3d.pickle -P /workplace/
The script run.sh
shows an example of parameters for starting the training of the e2e model.
In case you want to train your model on Kinetics, you need to adapt the function get_kinetics() in auxiliary/auxuliary_dataset.py according to the format in which Kinetics is stored on your machine. The current version is just a placeholder and will NOT work right away.
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2both --save_path PATH_TO_RESULT_FOLDER --nopretrained
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2others --save_path PATH_TO_RESULT_FOLDER --nopretrained
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2both --save_path PATH_TO_RESULT_FOLDER --fixed
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2others --save_path PATH_TO_RESULT_FOLDER --fixed
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2both --save_path PATH_TO_RESULT_FOLDER --weights [path_to_SUN_pretraining]
python3 main.py --n_epochs 150 --bs 22 --lr 1e-3 --network r2plus1d_18 --dataset kinetics2others --save_path PATH_TO_RESULT_FOLDER --weights [path_to_SUN_pretraining]
No Description
Python Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》