You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The string: "2 st 25 öre 1949,1950.\r\n3 st 1 kr 1943.\r\n4 st 1 kr 1944" will generate an Math::DomainError. It comes from the punkt-segmenter gem and it's a known issue.
A possible fix/workaround for this is to use the fork harrisj/punkt-segmenter and use a trained tokenizer. A code doodle on how that could look can be found here: here.
The text was updated successfully, but these errors were encountered:
In production we've seen this happen:
The string:
"2 st 25 öre 1949,1950.\r\n3 st 1 kr 1943.\r\n4 st 1 kr 1944"
will generate anMath::DomainError
. It comes from thepunkt-segmenter
gem and it's a known issue.A possible fix/workaround for this is to use the fork harrisj/punkt-segmenter and use a trained tokenizer. A code doodle on how that could look can be found here: here.
The text was updated successfully, but these errors were encountered: