HashNeRF
Model description
A PyTorch implementation (Hash) of the NeRF part (grid encoder, density grid ray sampler) in instant-ngp, as described in Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.
Step 1: Installation
pip3 install -r requirements.txt
Step 2: Preparing datasets
We use the same data format as instant-ngp, fox and blender dataset nerf_synthetic.Please download and put them under ./data
.
For custom dataset, you should:
- take a video / many photos from different views
- put the video under a path like ./data/custom/video.mp4 or the images under ./data/custom/images/*.jpg.
- call the preprocess code: (should install ffmpeg and colmap first! refer to the file for more options)
python3 scripts/colmap2nerf.py --video ./data/custom/video.mp4 --run_colmap # if use video
python3 scripts/colmap2nerf.py --images ./data/custom/images/ --run_colmap # if use images
Step 3: Training and test
One single GPU
First time running will take some time to compile the CUDA extensions.
# train with fox dataset
python3 main_nerf.py data/fox --workspace trial_nerf -O
# data/fox is dataset path; --workspace means output path; -O means --fp16 --cuda_ray --preload, which usually gives the best results balanced on speed & performance.
# test mode
python3 main_nerf.py data/fox --workspace trial_nerf -O --test
# train with the blender dataset, you should add `--bound 1.0 --scale 0.8 --dt_gamma 0`
# --bound means the scene is assumed to be inside box[-bound, bound]
# --scale adjusts the camera locaction to make sure it falls inside the above bounding box.
# --dt_gamma controls the adaptive ray marching speed, set to 0 turns it off.
python3 main_nerf.py data/nerf_synthetic/lego --workspace trial_nerf -O --bound 1.0 --scale 0.8 --dt_gamma 0
# train with custom dataset(you'll need to tune the scale & bound if necessary):
python3 main_nerf.py data/custom_data --workspace trial_nerf -O
Results on BI-V100
@@ -65,90 +60,4 @@ python3 main_nerf.py data/nerf_synthetic/lego --workspace trial_nerf -O
Convergence criteria |
Configuration (x denotes number of GPUs) |
Performance |
Accuracy |
Power(W) |
Scalability |
Memory utilization(G) |
Stability |
0.0652 |
SDK V2.2,bs:1,1x,fp16 |
10 |
11.9 |
82 |
0.903 |
28.1 |
1 |
Reference