Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strange lexing behavior #33

Open
databus23 opened this issue Oct 16, 2015 · 2 comments
Open

strange lexing behavior #33

databus23 opened this issue Oct 16, 2015 · 2 comments

Comments

@databus23
Copy link
Contributor

I'm having a hard time understanding the behavior of the lexer in the following case:

/\(/   { fmt.Printf("-> %q\n", yylex.Text()) }
/\)/   { fmt.Printf("-> %q\n", yylex.Text()) }
/[^( ][^ ]*[^ )]/ { fmt.Printf("-> %q\n", yylex.Text()) }
//

package main
import ("fmt")
func main() {
  fmt.Printf("lexing %q:\n", "(rule)")
    NN_FUN(NewLexer(strings.NewReader("(rule)")))
  fmt.Printf("lexing %q:\n", "( rule  )")
    NN_FUN(NewLexer(strings.NewReader("( rule )")))
}

Output of nex -r -s huh.nex:

lexing "(rule)":
-> "("
-> "rule"
lexing "( rule  )":
-> "("
-> "rule"
-> ")"

Why is the lexer swallowing the trailing bracket when there is no space between the content and the surrounding brackets?. This looks like a bug to me. It has something to do with second character class in the content regex ([^ ]*), when I change that to also not match ) it works.

@databus23 databus23 changed the title strange flexing behavior strange lexing behavior Oct 16, 2015
@drewwells
Copy link
Contributor

#16 mentions requiring a newline at the end of the file. This looks to apply here too. Adding even a space at the end matched the last bracket correctly "(rule) "

@databus23
Copy link
Contributor Author

@drewwells not for me:

lexing "(rule) ":
-> "("
-> "rule"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants