MorpheuS: Neural Dynamic 360° Surface Reconstruction from Monocular RGB-D Video

Paper | Project Page | Video

MorpheuS: Neural Dynamic 360° Surface Reconstruction from Monocular RGB-D Video
Hengyi Wang, Jingwen Wang, Lourdes Agapito
CVPR 2024

This repository contains the code for the paper MorpheuS: Neural Dynamic 360° Surface Reconstruction from Monocular RGB-D Video, a dynamic surface reconstruction method that leverages a diffusion prior to achieve 360° surface reconstruction.

Update

• Code for visualization of the results [2024-3-25]
• Code for data pre-processing [2024-3-21]
• Code for MorpheuS [2024-3-18]

Installation

For detailed instructions, please refer to INSTALL.md.

Dataset

Please download here: Google Drive. Alternatively, use our download script:

bash scripts/download_data.sh

To create your own dataset, please refer to preprocess.

Run

You can run MorpheuS using the code below:

python morpheus.py --config './configs/snoopy.yaml'

For visualizing the results, you can use:

python visualizer.py --config './configs/snoopy.yaml'

Advanced tips

I have included my tips in the comments prefixed with "NOTE". Given the extensive complexity of this project and the numerous experiments conducted, reading these tips is strongly encouraged to gain a deeper understanding.

Please be aware that this project has undergone significant refactoring. As a result, certain sections may differ from the original codebase. However, these modifications are intended to enhance overall performance and results. If anything is broken, don't hesitate to open an issue:)

Acknowledgement

We have borrowed codes from following awesome repositories, many thanks to authors for sharing their code:

• stable-dreamfusion
• NDR
• VolSDF

The research presented here has been supported by a sponsored research award from Cisco Research and the UCL Centre for Doctoral Training in Foundational AI under UKRI grant number EP/S021566/1. This project made use of time on Tier 2 HPC facility JADE2, funded by EPSRC (EP/T022205/1).

Citation

If you find our code or paper useful for your research, please consider citing:

@inproceedings{wang2024morpheus,
        title={MorpheuS: Neural Dynamic 360deg Surface Reconstruction from Monocular RGB-D Video},
        author={Wang, Hengyi and Wang, Jingwen and Agapito, Lourdes},
        booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
        pages={20965--20976},
        year={2024}
      }