match until/til?

[hh9527]

for example, how to lex rust like raw string: r###"..."###

rule Init {
  'r' '#'* '"' => |lexer| {
    *lexer.state() = lexer.match_().len() - 1; // set the number of end mark “#...
    lexer.switch_and_return(LexerRule::RawText, Token::RawTextStart)
  },
}

rule RawText {
  // here, I dont know how to constraint to number of '#'
  _* '"' '#'* => |lexer| {
    len n = *lexer.state();
    let cont = lexer.match_();
    let body = &cont[..cont.len() - n];
    lexer.switch_and_return(LexerRule::Init, Token::RawTextCont(body, n));
  },
}

[osa1]

Counting needs to be done using states. Here are two examples:

• Lexing Rust r###... syntax: https://github.com/osa1/lexgen_rust/blob/9e42b50572a2be12509d458e9be4d663c5fc4fe8/crates/lexgen_rust/src/lib.rs#L475-L479
• Lexing Lua [==[... syntax:

  lexgen/crates/lexgen/tests/lua_5_1.rs

  Lines 189 to 198 in 271ba43

  	"[" => |lexer| {
  	match lexer.peek() {
  	Some('[') | Some('=') => {
  	lexer.state().long_string_opening_eqs = 0;
  	lexer.state().in_comment = false;
  	lexer.switch(LexerRule::LongStringBracketLeft)
  	}
  	_ => lexer.return_(Token::LBracket),
  	}
  	},

  a simpler version of this is in the README: https://github.com/osa1/lexgen/#stateful-lexer-example