Skip to content

Latest commit

 

History

History
123 lines (89 loc) · 5.42 KB

README.md

File metadata and controls

123 lines (89 loc) · 5.42 KB
Raw

DFM-Net (ACM MM 2021)

Official repository for paper Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection | 中文版

News


Block diagram of DFM-Net.

The most efficient RGB-D SOD method ⚡

  • Low model size: Model size is only 8.5Mb, being 6.7/3.1 smaller than the latest lightest models A2dele and MobileSal.
  • High accuracy: SOTA performance on 9 datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP, DUT-RGBD, RedWeb-S, COME).
  • High Speed: Cost 50ms on CPU (Core i7-8700 CPU), being 2.9/2.4 faster than the latest fastest models A2dele and MobileSal.


Performance visualization. Performance visualization. The vertical axis indicates the average S-measure over six widely used datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP). The horizontal axis indicates CPU speed. The circle area is proportional to the model size.

Extension 🔥

Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection

  • More comprehensive comparison:
    • Benchmark results on DUT-RGBD, RedWeb-S, COME are updated.
    • Metric of maximum-batch inference speed is added.
    • We re-test the inference speed of ours and compared methods on Ubuntu 16.04.
  • Working mechanism explanation
    • Further analyses verify the ability of DQFM in distinguishing depth maps of various qualities without any quality labels.
  • Application on efficient VSOD
    • One of the lightest VSOD methods!
    • Joint training strategy is proposed.

Easy-to-use to boost your RGB-D SOD network

If you use a depth branch as an affiliate to the RGB branch:

  • Use DQW/DHA to boost performance with extra 0.007/0.042Mb model size increased
  • Use our light-weight depth backbone to improve efficiency

if you adopt parallel encoders for RGB and depth:

Test

Directly run test.py

The test maps will be saved to './resutls/'.

data preparation

Results

  • We provide testing results of 9 datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP, DUT-RGBD 🆕, RedWeb-S 🆕, COME 🆕).

  • Evaluate the result maps: You can evaluate the result maps using the tool in Matlab Version or Python_GPU Version.

  • Note that the parameter file is 8.9Mb, which is 0.4Mb bigger than we report in the paper because keys denoting parameter names also occupy some space. Then put them under the following directory:

     -dataset\ 
   -RGBD_train
   -NJU2K\  
   -NLPR\
   ...
 -pretrain
   -DFMNet_300_epoch.pth
   ...

Training

  • Download training data(eb2z)
  • Modify setting in options.py and run train.py

Application on VSOD 🆕

Citation

Please cite the following paper if you use this repository in your research

@inproceedings{zhang2021depth,
title={Depth quality-inspired feature manipulation for efficient RGB-D salient object detection},
author={Zhang, Wenbo and Ji, Ge-Peng and Wang, Zhuo and Fu, Keren and Zhao, Qijun},
booktitle={Proceedings of the 29th ACM International Conference on Multimedia},
pages={731--740},
year={2021}
}

@artical{zhang2022depth,
title={Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection},
author={Zhang, Wenbo and  Fu, Keren and  Wang, Zhuo and Ji, Ge-Peng and Zhao, Qijun},
booktitle={arXiv:2208.03918},
year={2022}
}