[Feature]: Implementation of DRY sampler.

[Abdulhanan535]

🚀 The feature, motivation and pitch

Majority of the bots repeat sentences and due to that, it's annoying.

Implementing DRY sampler for Aphrodite Engine.

DRY is a sampler that forces the model to not repeat while mantaining the quality or even improving it, some of the backends have already support for this.

• Text Gen webui
• koboldai
• llamaccp

Alternatives

No response

Additional context

No response

[81549361]

+1
LostRuins/koboldcpp#982

[AlpinDale]

Might be interesting to add. I currently don't have the bandwidth to tackle this. I may have the time after the 0.5.4 release, but if someone else wants to take over for now, it should be quite easy:

1. Add the parameters related to the DRY sampler here: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/common/sampling_params.py

2. Define the sampler's metadata here. e.g. the conditions for triggering it (could be a related sampler param being set to higher than 0, etc) and create the necessary tensors. Plenty of examples here so you won't be lost: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/modeling/sampling_metadata.py

3. Finally, implement the actual sampler logic here: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/modeling/layers/sampler.py

Should be plenty of examples at each step from other samplers. Please make sure you contribute to the rc_054 branch, unless you're reading this in a few weeks and that branch is merged into main.

[Abdulhanan535]

any chance for this?

[Abdulhanan535]

like this
``

[selalipop]

I've opened an draft PR but it definitely has some issues around the sequence breakers: #827

I'm not familiar enough with how the sampling metadata works to correctly add a new value that's a List[int] as opposed to a scalar value like most of the existing parameters, added some notes