Expand ASI using the Parse Table. #626
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This sharing of states causes the resolution of the inconsistencies to take a long time, and also causes issues in lower_reduce_action. This change remove the sharing and explicit each tokens.
When we have conditionals which are checking whether the previous character is on a new line or not, such as `[no LineTerminator here]`, it is safe to wrap any non-garded edge with both the condition and its negated version. This way, the resolution of lookahead or reduce-reduce and shift-reduce conflict, does not have to push the check for new lines further than necessary.
This change adds `mark_sweep_states` function to replace unreachable states by `None`, and prevent them from being reused. This removes the need for tracking inconsistent states which are potentially unreachable, but we are to re-scan the whole parse table in order to refill the todo list in cased of missed inconsistencies.
When restore_edges adds a consistent action in front of shifted terminals and non-terminals, we want to record this as we want to annotate the target destination while adding the fact that we are lacking a shifted token.
Lower Reduce Action lookup for the state which would be used to shift the non-terminal, and then encode the Replay action based on the the state where we are reducing the nonterminal. However, this strategy alone does not work when reduce actions are shared across, as having different lookahead with the same reduce actions, we generate multiple Replay paths for the same state which shift the reduced nonterminal. Adding the Lookahead terms to the reduce action, ensure that we would not add sharing if we did not have sharing in LR0 before.
The Rust backend expect that we have a FunCall which produces a "value", to be wrapped. This modification adds the FunCall as part of the AstBuilder to produce a value.
Previously, the condition was only checking in order to error at runtime. Now that the errors are handled at compile time, we can remove all code path handling automatic semicolon insertion as an error case.
nbp
added
the
parser generator
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ASI is quite complex as it relates to "offensive tokens" and "restricted tokens", and it would verbose and complex to convert these to grammar rules.
These changes aim at fixing #623 by expanding the ASI rules, such that
lower_reduce_actioncan reason about the fact that ASI move the current failing token to the replay list before reducing anErrorSymbol.By adding these rules to the parse table, we would be able of making optimization which would be crossing the presence of
CheckNotOnNewLine(renamed toCheckLineTerminator), which would imply adding arguments to states which have aCheckLineTerminatoras an out-going edges, and thus should be able to avoid usage of the replay list completely.(This is a draft, as the generate file phase takes ~10 minutes)