Okyanus Oral and Figen S. Oktem.
This repository contains the official codes for the paper "Plug-and-Play Regularization on Magnitude with Deep Priors for 3D Near-Field MIMO Imaging". (to appear in IEEE Transactions on Computational Imaging)
Youtube: 3D Rotating views (Experimental Results) |
Youtube: 3D Rotating views (Simulation Results) |
You can download the synthetically generated dataset from here. If you use this dataset, you should cite [1].
You should refer to (and cite) [2,3] for the experimental data.
Check format-exp-data.ipynb for the instructions on formatting the experimental data provided at [3].
[1] I. Manisali, O. Oral, and F. S. Oktem, ”Efficient physics-based learned reconstruction methods for real-time 3D near-field MIMO radar imaging”, Digital Signal Processing, vol. 144, p. 104274, 2024, doi: 10.1016/j.dsp.2023.104274.
[2] J. Wang, P. Aubry and A. Yarovoy, "3-D Short-Range Imaging With Irregular MIMO Arrays Using NUFFT-Based Range Migration Algorithm," in IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 7, pp. 4730-4742, July 2020, doi: 10.1109/TGRS.2020.2966368.
[3] Jianping Wang, January 10, 2020, "EM data acquired with irregular planar MIMO arrays", IEEE Dataport, doi: https://dx.doi.org/10.21227/src2-0y50.
- For simulations, follow the instructions provided in main-simulated.ipynb.
- For the experimental reconstructions, follow the instructions provided in main-experimental.ipynb.
- Codes for the reconstruction algorithms: You can find them in ../src/optimization.py (apart from the comments present in optimization.py, further explanations are provided in main-simulated.ipynb and main-experimental.ipynb).
- Deep denoiser: Check ../src/nn_models/unet3D.py for the network class. The parameters of the utilized model architecture can be found in ../trained_model/info.json. For the parameter dictionary of the trained model, check ../trained_model/base_model.pt`.
- Observation model: Check ../src/forward_models/bornapprox.py for the forward model class. The base model is MVprod(), but if you have memory problems, you may prefer to use MVProd_iterK(), which computes the Born approximated measurements iteratively.
- Parallel processing: All solvers can work on CPUs and GPUs. You can process the measurements in parallel and also asynchronously (see main-simulated.ipynb for further info). If you plan on parallel processing batch of measurements, then I suggest you to enable 'asynchronous' argument of the solvers as this allows to asynchronously stop the image updates and speeds up the execution. On the other hand, if set to False, the solver will continue the optimization even after some images in the batch converge, which can potentially create numerical instability.
- Scale invariance: For the deep neural network, you should use the scale_invariance_wrapper() class if the measurements have a different scale than the training data.
- Using your DNN: The NNCG-CSALSA solver allows you to use both blind and non-blind denoisers. Please check the notebooks for further information.
Please cite the following when using this code:
@ARTICLE{oral2024plug,
author={Oral, Okyanus and Oktem, Figen S.},
journal={IEEE Transactions on Computational Imaging},
title={Plug-and-Play Regularization on Magnitude With Deep Priors for 3D Near-Field MIMO Imaging},
year={2024},
volume={10},
number={},
pages={762-773},
doi={10.1109/TCI.2024.3396388}}
If you have any questions or need help, please feel free to contact me (Okyanus Oral, email: [email protected]).