Built-in search in mandarin config throw error

[riccox]

To Reproduce

1. use latest auto start template to create a docs project.
2. config mandarin search by this steps in docs.
3. start dev server
4. enter some words input search input.
5. see dev server output.

Current vs. Expected behavior

throw orama tokenizer type error, and without any search results.

node_modules/fumadocs-core/node_modules/@orama/orama/dist/esm/components/index.js (130:1) @ eval
TypeError: tokens is not iterable
    at eval (./app/api/search/route.ts:17:94)
  128 |                 const tokens = tokenizer.tokenize(value, language, prop, false);
  129 |                 implementation.insertDocumentScoreParameters(index, prop, internalId, tokens, docsCount);
> 130 |                 for (const token of tokens) {
      | ^
  131 |                     implementation.insertTokenScoreParameters(index, prop, internalId, tokens, token);
  132 |                     node.insert(token, internalId);
  133 |                 }

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 24.1.0: Thu Oct 10 21:02:45 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T8112
  Available memory (MB): 24576
  Available CPU cores: 8
Binaries:
  Node: 20.17.0
  npm: 10.8.2
  Yarn: N/A
  pnpm: 9.12.3
Relevant Packages:
  next: 15.0.3 // Latest available version is detected (15.0.3).
  eslint-config-next: 15.0.3
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.7.2

Which area(s) are affected? (Select all that apply)

Core

Additional context

No response

[fuma-nama]

I found that it's a upstream problem caused by invoking tokenizer.tokenize without await, they made insert methods sync but it's not compatible with tokenizer, will report it to Orama later.