Cache input_ids on CPU memory.

[pmfirestone]

_get_partial_codes now caches input_ids greedily whenever it's given new ones. The caller is responsible for maintaining the validity of the cache. In particular, is_valid has to remove the last input_id from the cache where the last input_id isn't valid.

When benchmarking on the following settings, we see an 80% reduction in the time spent in syncode.

• model: "microsoft/phi-2"
• Grammar: "python"
• mode: "grammar_mask"
• parser: "lr"
• prompt: '''def print_prime(n):
  """
  Print all primes between 1 and n
  """
  '''
• max_new_tokens: 50.
• Using the huggingface LogitsProcessor API.
• Computer is a Vultr A16, 1 GPU.