Seemingly arbitrary maximum limit of entries using local git repository

[xelivous]

• Sveltia Version: 0.40.1
• Browser: Chromium 128.0.6613.137 (Official Build) Arch Linux (64-bit)

---

I'm noticing some strange behaviour with the local git repository. I currently have 6 collections with the following quantities (161, 35, 1, 2, 2, 157) for a total of 358 items.

If I create a new item and bring the total up to 359, I can browse through the site and edit any information as normal. However if I refresh the page (or simply add in another collection with more than 1 entry), I will get an "unexpected error". Looking at the console log provides the following stack trace:

As soon as I remove any item from any of my collections (or comment out a collection) to bring the total down to 358 or lower I am able to enter the CMS once more.

[kyoshino]

Thanks for the heads-up. The CMS itself doesn’t set any limitation, but the error message comes from Chrome. I’ll figure out what’s going on.

[kyoshino]

I have duplicated a collection on my client instance, bringing the total number of entries to 1,000+, but it still works 🤔

Do you have many images, or some large files?

Perhaps it’s a Linux-specific issue. I’m on macOS. Will create a Linux VM to test.

[xelivous]

The 161 collection have 1x image, average around 250kb size. The 35 collection all have 1x small image each (<30kb). The other collections don't have any images. "All assets" reports 224 images.

I'll try to create some random test configs of my own to see if i can replicate it some other way.

---

After doing some testing, it might be some kind of random memory limit? I had 4GB of memory left on my computer but maybe chrome was running into some limit before that based off of the testing below.

1. My indexed DB was fine at 1.1MB
2. I browsed some thumbnails in the asset library and the indexedDB went up to 1.7MB
3. refreshed the page and it failed to load
4. Cleared the indexedDB, disabled cache
5. Refreshed, Loaded fine again, refreshed, loaded fine
6. Browsed some thumbnails to raise it above the limit, and it failed to load again.
7. Fully closed chrome and everything else, to free up as much ram as possible on my system
8. Reloaded up sveltia, browse through thumbnails, indexedDB up to 7MB, refresh, works fine
9. Wait a few minutes, refresh, fails once (same tracelog as original error), refresh again, works fine
10. The more that I refresh now, the more likely it is to fail. After about 20 refresh cycles it's now failing 100% of the time. If I restart chrome it will likely start working again.

Could be a weird bug with the local storage api.

[kyoshino]

Thanks for your investigation! I’ll create a bunch of images and try again.

[kyoshino]

I have generated 500 images in the asset library but these are still loaded (though It takes a sec; it’s a separate issue)

I think URL.createObjectURL() used for displaying thumbnails is causing a memory leak because Sveltia CMS is not revoking these URLs properly. I’ll fix the issue.

[kyoshino]

I’ve made the changes in v0.40.2. Do you see any difference?

[xelivous]

Yes, still the same issue after ensuring to pin it to 0.40.2

[kyoshino]

Sorry to hear that. I’ll set up a Linux VM with Parallels.

[kyoshino]

I have tested with Linux Mint but still can’t reproduce the problem. Will do more research.

[kyoshino]

Sorry it took so long. Not sure if you’re still seeing the problem but I’ve realized that the problem is probably not with the thumbnails, but with the entry file reading part, so have added some try...catch to prevent crashes. Let me know if the latest version solves your problem.

[xelivous]

Haven't had too many issues with my "medium"-sized repository for a while now since i've constantly restarted the OS/browser relatively often enough. However I am still having issues with my "massive" repo (1084 pages reported by hugo + 824 images) with v0.46.5 unfortunately.

I tried digging through with the debugger to see where exactly it's failing but i just can't figure it out from stepping through.

It does fail within this line though, and seems to be in the areas that you've changed; if i put a breakpoint on the line below that it will never reach it. More specifically it manages to _iterate() properly in that function and then fails when resolving the final promises here. You've already changed the hash function to be async so i'm not sure what else would be failing in there.

I'll look into making a local dev copy of sveltia this weekend to see if i can see where/how exactly it's failing instead of trying to work around with local overrides in the debugger.

Thanks

[kyoshino]

I will retry to reproduce the issue locally.