Help understanding prepare_batches and prepare_subfunctions

[ChrisJWest]

Hi! First of all, I am a massive fan of MIScnn. I've been playing around with it for several months now and it has been incredibly useful. Huge props to the developers and those who help maintain the code-base. I have a quick question regarding the prepare_* functions in preprocessor.py.

Let's say we have several data augmentations, example:

cycles=2, scaling=True, rotations=True, elastic_deform=True, mirror=True

and also several subfunctions, example:

subfunctions = [sf_resample, sf_clipping, sf_normalize]

What is actually happening behind the scenes when we prepare either one of these to save on disk? Is there a specific order that happens with data augmentations, subfunctions, and saving? It seems like these functions should save time since we are saving recompute at the expense of disk space.

Since subfunctions are sequential and the same always, it seems like we should always prepare_subfunctions when we have the space. Would you agree that this is the case?

For prepare_batches, I have a quick question. When we say prepare_batches=True, does that randomly take a training batch and apply a random cycle of the above data augmentations and save it to disk? And then all future epochs re-use that training batch with the same augmentation? Because then it seems like each epoch we will be training on the same data aug for that batch, whereas we may wish to take new random data aug cycles per batch per epoch (which seems like something we might compute in an online environment per-epoch). Or is it just saying that we are saving the raw image batch on disk on pickle format but then we apply the data augmentation after loading?

Thanks!

[muellerdo]

Hey @ChrisJWest,

First of all, I am a massive fan of MIScnn. I've been playing around with it for several months now and it has been incredibly useful. Huge props to the developers and those who help maintain the code-base.

Thank you! Always happy to hear that MIScnn is useful for the community :)
And also nice to see that the new Discussion Forum is useful ;)

I have a quick question regarding the prepare_* functions in preprocessor.py.

Let's say we have several data augmentations, example:

cycles=2, scaling=True, rotations=True, elastic_deform=True, mirror=True

and also several subfunctions, example:

subfunctions = [sf_resample, sf_clipping, sf_normalize]

What is actually happening behind the scenes when we prepare either one of these to save on disk? Is there a specific order that happens with data augmentations, subfunctions, and saving? It seems like these functions should save time since we are saving recompute at the expense of disk space.

---

Sure, the general process behind the scene is the following:

1. Image loading
2. Subfunction Application
3. Analysis Strategy -> taking just the full image or slice it in patches (via grid slicing or random cropping)
4. Image Augmentation
5. Combine samples into a batch

So now we have the two parameter: prepare_batches and prepare_subfunctions

prepare_batches:
Before training starts, this will only ONCE compute batches for the whole dataset (all steps from 1-5). These will be stored to disk and randomly loaded during training.

What does this mean?
These batches are always the same -> everything with a random factor will be done only once -> each sample has only one specific random cropping & image augmentation

Why should we ever use this? I want that my images are randomly altered in each epoch!
Understandable, but if you want some traditional machine learning with beforehand image augmentation application or no image augmentation at all because your dataset size is already in the thousands then here you go. Why waste time computing the same batch again if everything is the same and already computed an epoch ago?

If you save the batches to disk and reload the batches, doesn't that mean that the sample grouping is always the same?
Yes it is. Simply, it is a sacrifice of randomness (which can influences training efficiency and thus maybe performance) for speed.

prepare_subfcuntions:
Before training starts, this will compute Subfunctions and store them to disk (step 1-2). During training, the subfunction-preprocessed samples will be loaded instead of the original ones and then only step 3-5 will be applied.

What does this mean?
The important steps with random factors (e.g. random cropping, image augmentation and batch packing) are still random and new in each epoch. However, normally the most time-consuming steps are the subfunctions like Resizing/Resampling of high-resolution images or even 3D volumes, which will be only done once at the start.

Sounds good! Any adverse effects?
Disk space. Simply, you just copy the whole dataset. If you have enough disk space and CPU intensive / time-consuming subfunctions like Resampling, then do it! If you are working on limiting resources or your evil colleagues downloaded half of Kaggle on the workstation, you can also keep it deactivated for the cost of speed.

Hope that this is a bit helpful :)

---

Since subfunctions are sequential and the same always, it seems like we should always prepare_subfunctions when we have the space. Would you agree that this is the case?

Yes, correct!

For prepare_batches, I have a quick question. When we say prepare_batches=True, does that randomly take a training batch and apply a random cycle of the above data augmentations and save it to disk? And then all future epochs re-use that training batch with the same augmentation? Because then it seems like each epoch we will be training on the same data aug for that batch, whereas we may wish to take new random data aug cycles per batch per epoch (which seems like something we might compute in an online environment per-epoch).

Yes, absolutely correct! In my opinion, this is only useful if you have enough images and don't want to apply image augmentation at all or just very marginal. But for my experiments, I always use and recommend online augmentation (so I always disable prepare_batches).

Or is it just saying that we are saving the raw image batch on disk on pickle format but then we apply the data augmentation after loading?

No, this is only done for the prepare_subfunction parameter.

Cheers,
Dominik

[ChrisJWest]

Amazing!! This is a fantastic reply, it answered all my questions completely and very thoroughly! Thanks again so much, this codebase is really excellent; I will recommend it to anyone who is looking at running some experiments in this domain. Let me know if I can help with anything in the future! Cheers ~ Chris