Stratified sampling

[alishibli97]

No description provided.

[KerekesDavid]

Good job, seems to be working with MADOS on my end.

One nitpick, since the stratification itself takes quite long, and one of the usecases for limiting dataset size is to have a quick test run on a few samples, could you add a toggle to have the old behavior of just random sampling?

[KerekesDavid]

You can remove these, that's why we have git :)

[VMarsocci]

Hi before merging, please consider the following changes (already discussed in private- but I report them here for everyone else).

1. add the possibility to choose the stratified sampling on a single set (either train or val)
2. check the last version of HLSBurnScars (in the one you want to merge it is not updated)
3. replace the "calculate class distribution" function with just reading this distribution in the dataset config
   Thanks a lot :)

[yurujaja]

To address the comments, some modifications are made:

1. configure sampling for both train and val
2. configure the limited label strategy: random or stratified
3. the unused function in hlsburn scar is removed
4. the "calculate class distribution" function is still needed to calculate image-wise distribution to enable the selection of images

[alishibli97]

Added Regression stratification.
In train.yaml, one can specify the stratification method using the param:
limited_label_strategy: stratified_classification # or stratified_regression, or random

@RituYadav92 is it possible to validate that it works on your side with biomastters?

[RituYadav92]

I made two modifications to the code:

1. Updated the code to select a fraction of labels from each bin.
2. Updated variable names "labeled_idx, unlabeled_idx" to "selected_idx, other_idx"

Please update the same for classification if it fits well.

[RituYadav92]

The code looks fine now except line 68 in subset_sampler.py. It should be "if bin_id in indices_per_bin:" instead of "if bin_id not in indices_per_bin:"

Please check.

[alishibli97]

I think it is fine, what do you suspect?
It is only to initialize the dict key if not present

[RituYadav92]

I see now, you didn't adapt the initialization from regression but did it other way. Np problem. Resolved.