Skip to content

Latest commit

 

History

History
80 lines (58 loc) · 3.16 KB

README.md

File metadata and controls

80 lines (58 loc) · 3.16 KB
Raw

BTS-Net (ICME 2021)

BTS-NET: BI-DIRECTIONAL TRANSFER-AND-SELECTION NETWORK FOR RGB-D SALIENT OBJECT DETECTION


Block diagram of the proposed BTS-Net.

1. Introduction

Features

  • Achieves a new state-of-the-art on 6 public datasets by the time the paper was accepted (Leaderboard in paper-with-code).
  • First RGB-D SOD methods to introduce bi-directional interactions across RGB and depth during the encoder stage.
  • Bi-directional Transfer-and-Selection (BTS) module is designed to achieve this idea.
  • An effective light-weight group decoder is designed to improve efficiency.

Easy-to-use to boost your methods

if you adopt parallel encoders for RGB and depth:

  • In encdoer stage, adopt proposed bi-direcitonal interaction stategy to boost your methods (Naive fusion e.g., pixel-wise addition rather than BTS module may also improve). Or otherwise you adopt uni-directional interaction, I strongly recommend not to use D=>R , even R=>D is consistently better with the same space/time consumption.
  • In decoder stage, adopt our group decoder to replace naive U-Net like deocder to boost efficiency.

If you use a depth branch as an affiliate to RGB branch:

  • refer to our another work DFM-Net

2. Requirements

  • Python 3.7, Pytorch 1.7, Cuda 10.1
  • Test on Win10 and Ubuntu 16.04

3. Data Preparation

  • Download the test data (containing NJU2K, NLPR, STERE, RGBD135, LFSD, SIP) from Here [code: 940i], trained model (epoch_100.pth) from Here [code: 2j99], training data from Here [code: eb2z]. Then put them under the following directory:

     -dataset\ 
   -RGBD_train\  
   -NJU2K\
   -NLPR\
   ...
 -pretrain
   -epoch_100.pth\
   ...

4. Testing & Training

  • Testing

    Directly run test.py, the test maps will be saved to './resutls/'.

  • Evaluate the result maps:

    You can evaluate the result maps using the tool in Matlab Version or Python_GPU Version.

  • Training

    Modilfy setting in options.py and run tarin.py

5. Results


Quantitative comparison with 16 SOTA over 4 metrics (S-measure, max F-measure, max E-measure and MAE) on 6 datasets.

Download

  • Test results of the above datasets can be download from here [code: cujh].

6. Citation

Please cite the following paper if you use this repository in your reseach

@inproceedings{Zhang2021BTSNet,
 title={BTS-Net: Bi-directional Transfer-and-Selection Network for RGB-D Salient Object Detection},
  author={Wenbo Zhang and Yao Jiang and Keren Fu and Qijun Zhao},
  booktitle={ICME},
  year={2021}
}