Why is there no support for top-p / top-k ?

[DanJbk]

It seems to me like an important feature, but i see that that feature has not been implemented yet, I see the sampling logic at https://github.com/guidance-ai/guidance/blob/d36601b62096311988fbba1ba15ae4126fb695df/guidance/models/_model.py#L860C1-L860C30 is mostly complete, is there anything preventing adding this feature?

[slundberg]

This is on the near-term todo list. The tricky part is that we want to allow all these params to be controlled in the grammar, so we have to introduce some support there before we can pass that parameter to the get_logits function. If this is a blocker for anyone and they want to help, speak up :)

[wiiiktor]

1// Is my understanding right that I can get_logits instead of tokens, from the called model? Is there a manual for this?
2// I would like to be sure - these logits are before or after Softmax?