toki Regexp tokenizer/scanner in Go. The scanner takes in a list of token definitions and string input. it is general and easy to use. token definition a token is a pair of constant and regexp string.