Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
mm1978 987de14ad1 | 8 months ago | |
---|---|---|
ckpts | 8 months ago | |
datasets | 8 months ago | |
images | 9 months ago | |
losses | 8 months ago | |
networks | 8 months ago | |
utils | 8 months ago | |
.gitignore | 9 months ago | |
LICENSE | 11 months ago | |
README.md | 11 months ago | |
__init__.py | 11 months ago | |
debug.ipynb | 8 months ago | |
engine.py | 8 months ago | |
infer_image2feature.py | 8 months ago | |
infer_image_igev.py | 10 months ago | |
infer_image_litemono.py | 10 months ago | |
layers.py | 8 months ago | |
main.py | 10 months ago | |
options.py | 8 months ago | |
train.py | 10 months ago | |
zhisuan_train.py | 8 months ago |
This repository contains the network architectures, losses and pretrained models from DeFeat-Net.
In the current monocular depth research, the dominant approach is to employ unsupervised training on large datasets, driven by warped photometric consistency.
Such approaches lack robustness and are unable to generalize to challenging domains such as nighttime scenes or adverse weather conditions where assumptions about photometric consistency break down.
We propose DeFeat-Net (Depth & Feature network), an approach to simultaneously learn a cross-domain dense feature representation, alongside a robust depth-estimation framework based on warped feature consistency.
The resulting feature representation is learned in an unsupervised manner with no explicit ground-truth correspondences required.
We show that within a single domain, our technique is comparable to both the current state of the art in monocular depth estimation and supervised feature representation learning.
However, by simultaneously learning features, depth and motion, our technique is able to generalize to challenging domains, allowing DeFeat-Net to outperform the current state-of-the-art with around 10% reduction in all error measures on more challenging sequences such as nighttime driving.
A simple script is provided as an example of running the depth and feature networks.
# Create and load model
model = DeFeatNet.from_ckpt(ckpt_file, key=lambda x: x['model']).to(device)
# Sub-networks can be run separately
model.depth_net(images)
model.feat_net(images)
model.pose_net(images, support_images)
To visualize the depth maps produced by the network:
# Assuming a batched tensor (b, 1, h, w)
disp_np = disp.squeeze(1).cpu().numpy()
ax.imshow(disp_np[0], cmap='magma', vmax=np.percentile(disp_np[0], 95))
NOTE: Images should be downsampled to (480, 352)
to match training resolution.
Checkpoints can be found here,
which contains variants trained on Kitti and RobotCar-Seasons, and should be placed in the ckpt
directory.
When restoring from a checkpoint with DeFeatNet.from_ckpt
, all optional parameters are set to the correct configuration,
so no additional initialization is required.
Please cite the following paper if you find DeFeat-Net useful in your research:
@inproceedings{spencer2020,
title={DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning},
author={Spencer, Jaime and Bowden, Richard and Hadfield, Simon},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2020}
}
We would also like to thank the authors of Monodepth2 for their contribution.
You can contact me at jaime.spencer@surrey.ac.uk
CVPR2020 单目深度估计
Jupyter Notebook Python
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》