You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
and realised I am getting the following warning, mainly coming from the transformers library:
\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
Following the suggestions by the huggingface developers in the opened issue, if a parameter clean_up_tokenization_spaces=True is added to the tokenizer definition here:
Dear
lambeq
developers,I was playing around with the package and testing the parsing example in the bobcat tutorial by simply running
and realised I am getting the following warning, mainly coming from the
transformers
library:Following the suggestions by the huggingface developers in the opened issue, if a parameter
clean_up_tokenization_spaces=True
is added to the tokenizer definition here:lambeq/lambeq/text2diagram/bobcat_parser.py
Line 197 in e012aa9
the warning will be resolved.
I can confirm this worked for me locally.
Many thanks!
The text was updated successfully, but these errors were encountered: