[LayerSkip] Per-Layer Dropout Rate Configuration

[mostafaelhoushi]

Describe the solution you would like:
Would like to enable configuration of a different layer dropout rate for each layer.

Describe the alternatives you have considered:
Currently, layer dropout is implemented in fairseq2 as a scalar probability for all layers (check here).
We can follow an implementation similar to this PR in torchtune to support linear, exponential, or step configurations for increasing dropout rate acorss layers.

Additional Context:
This will enable implementing:

• Progressive Layer Dropping: that claims to increase accuracy and speed of training if dropout rate increases across layers linearly
• LayerSkip: that claims to increase accuracy of early exit layers if dropout rate increaes linearly or exponentially across layers

[mostafaelhoushi]

The feature (along with other features) was implemented on top of an old fork here: YJYJLee#2
What we can do is:

• The code changes can be ported to the latest fairseq2 main branch.
• Add a couple of unit test cases (similar to testcases here)

[HimanshuJanbandhu]

Hi @mostafaelhoushi,
I would like to work on this.