📌 Track 1 - Task 3: Color Reconstruction
Track 1 of the SpikeCV Wuji Challenge includes a Color Reconstruction task designed to explore the advantages of spike cameras in ultra-fast vision, enabling the generation of high-fidelity color scenes from continuous spike streams.
📁 Dataset Overview
We provide two types of spike datasets for this task:
- Synthetic Spike Dataset
- Real-world Spike Dataset
📦 Download address:
https://openi.pcl.ac.cn/Cordium/SpikeCV/datasets
Both of the two datasets are included in RBSS.zip
🔷 1. Synthetic Spike Dataset
The training data is in the ./train folder, the folder structure is
---- train
|---- spk
| |---- 0000-0000.dat
| |---- 0000-0001.dat
| |---- ...
|---- gt
| |---- 0000-0000.npy
| |---- 0000-0001.npy
| |---- ...
The files with the same filenames in the spk folder and the gt folder are a pair of data. For example, the spk/0000-0000.dat corresponds to gt/0000-0000.npy, where the former include a spike tensor with a shape of $41 \times 720 \times 1280$, corresponding to temporal length, height, width, respectively.
The 21st frame in the spike stream (index 20 in python) correspond to the time of ground truth images. The ground truth images can be loaded by np.load(gt_path), which is a numpy array with a shape of $1 \times 3 \times 720 \times 1280$ and a value range of [0, 255].
Note that during the spike synthesis process, there is a brightness factor 0.6 between the spikes and ground truth. If you use a non-learning-based method such as TFP (with some demosaicing method) for reconstruction, you need to implement img = img / 0.6 to align the reconstructed image to the gt.
If you use a learning-based method, the factor 0.6 can be normalized after the network, or you can directly construct the mapping between spikes and gt to learning this factor in the network. Both of the two ways are OK. However, for real-captured data, note that these two ways will bring results with different light intensities (the network contains the brightness factor or not). You can adjust the results of real-captured data as you like, we will only evaluate real-captured data through non-reference-based metrics.
The test data is in the ./test/RBSS folder, only the image correspond to the 21st frame of each spike stream need to be reconstructed. The submitted results should be png files with a size of $720 \times 1280 \times 3$, where 3 represents the R, G, and B channels in sequence.
🔷 2. Real-world Spike Dataset
The data is in the ./test/real_captured folder, the folder structure is
---- test
|---- real_captured
|---- [name of scene 1].dat
|---- [name of scene 2].dat
|---- ...
|---- [name of scene N].dat
Each of the dat file contains a $399 \times 1000 \times 1000$, only the 101st, 111st, 121st, ..., 291st time stamp (index 100th, 110th, 120th, 290th in python) of the spike streams need to be reconstructed, 20 frames for each scene in total.
The submitted results should be png files with a size of $1000 \times 1000 \times 3$, where 3 represents the R, G, and B channels in sequence. The name of each png should be [scene_name]_[python_index].png, for example, poker_150.png.
📂 Submission Format
A zip file should be submitted, the name of the zip file should be the team name. The structure is:
---- BSSF
|---- 0000-0000.png
|---- 0000-0001.png
|---- ...
|---- 0029-0004.png
---- real_captured
|---- [name of scene 1]\_100.png
|---- [name of scene 1]\_110.png
|---- ...
|---- [name of scene N]_290.png
Compress and upload the results.zip as your final result.
Note that in this track, the reconstructed images should have the same spatial size with the input spikes.
🛠 Reference Code
You can use the following official scripts to help with data parsing and evaluation:
🙏 Acknowledgements
We would like to thank Yanchen Dong for his support and for providing access to the high-quality datasets used in this challenge.
📚 Reference
Yanchen Dong, et al.
"Self-Supervised Learning for Color Spike Camera Reconstruction"
CVPR 2025
