-
-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error with overlapping token definitions #420
Comments
After looking at this a bit more, I'm guessing I'm running into the "no backtracking" limitation. Any workaround suggestions are welcome. |
Hello! Can you please run in debugging mode and printout the corresponding graph? |
|
Hum, so I think you are right about the fact that the error might come from no backtracking issue, but a perfect Logos implementation shouldn't have that issue. First question: are you using Logos Second: did you try not setting any priority? Here, you set the priority to Last, it is often a source of issues to have patterns embedded in others, like |
Yes, using 0.14.1.
I had to set the priorities > 3 because of the skip expression,
#[logos(skip r".|[\r\n]")]
The "." can match anything, so there was a conflict. Other than that,
the priorities can all be the same.
I'm not sure I understand the third point. How else would you do it?
…On Sun, Sep 15, 2024 at 4:20 AM Jérome Eertmans ***@***.***> wrote:
Hum, so I think you are right about the fact that the error might come
from *no backtracking issue*, but a perfect Logos implementation
shouldn't have that issue.
First question: are you using Logos >=0.14.0? As it may have fixed some
issues.
Second: did you try not setting any priority? Here, you set the priority
to 3 to all tokens, it doesn't make much sense as the priority is only
used when two or more patterns match the same slice, and they are
differentiated based on their priority. But, if the number is the same, it
doesn't help. So please try without any priority, and only edit one
priority at a time.
Last, it is often a source of issues to have patterns embedded in others,
like TermWithZ containing both Word and Number, causing backtracking
issues.
—
Reply to this email directly, view it on GitHub
<#420 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAITHKRFDXP4BS3WI3SJVGDZWVGNFAVCNFSM6AAAAABOEMBCYWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJRGQ4DOOBRGM>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Ok perfect.
Ok seems legit, wasn't aware of that.
Usually, you can break down your logic in unique, non-overlapping, tokens, and then use callbacks and extras to handle more complex logic. Unfortunately, I don't have enough time to dig into this problem and understand really the root causes of why it doesn't work :-/ |
I'm getting a strange error when a regex could match the prefix of another regex. Maybe. I just don't know what the problem is. Here's a simplified case:
This generates:
If I replace the regex over
TermWithZ
with#[regex(r"Z", priority = 3)]
, I get:The "42world" is getting recognized correctly as a number and word.
What I don't understand is, why does the first TermWithZ regex mess up the recognition of "42world"? It doesn't contain a Z, so TermWithZ should ignore it completely and let the first two variants do their job.
The text was updated successfully, but these errors were encountered: