Blog post on regex constraint decoding #653
vivien000
started this conversation in
Show and tell
Replies: 1 comment
-
|
Very cool, thank you for sharing! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi! You might be interested in this new blog post in which I describe a new constrained decoding method which ensures both compliance with a regex constraint and "proper" tokenization (all generated token sequences correspond to tokenized strings). The aim of guaranteeing proper tokenization is to increase the decoding speed and reduce the divergence from the probability distribution of the unconstrained LLM. Here is a Twitter thread with a TL;DR.
Beta Was this translation helpful? Give feedback.
All reactions