Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Support for deserializing 5.1+ maps with WorldPartition #95

Open
Kein opened this issue Jun 17, 2023 · 3 comments
Open

[FR] Support for deserializing 5.1+ maps with WorldPartition #95

Kein opened this issue Jun 17, 2023 · 3 comments

Comments

@Kein
Copy link
Contributor

Kein commented Jun 17, 2023

UE version: 5.1.1
Example map: https://pomf2.lain.la/f/ddhmwfxw.7z

Currently it fails at IOEntry reader:

OverflowException: Array dimensions exceeded supported range.
This exception was originally thrown at this call stack:
    CUE4Parse.UE4.IO.IoStoreReader.Read(long, long) in IoStoreReader.cs
    CUE4Parse.UE4.IO.IoStoreReader.Extract(CUE4Parse.UE4.VirtualFileSystem.VfsEntry) in IoStoreReader.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.Read() in FIoStoreEntry.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.CreateReader() in FIoStoreEntry.cs
    ```
@Kein
Copy link
Contributor Author

Kein commented Nov 21, 2023

Small update: the generated tiles-maps are deserialized sucessfully.
But the root persistent map/WP map itself fails.
Case of commercial games: Talos Principle 2

@cozzbp
Copy link

cozzbp commented Jan 27, 2024

This is also happening with Palworld.

@yretenai
Copy link
Contributor

yretenai commented Feb 6, 2024

The reason for this is because the world ubulk file is more than 2 GB. The asset and export data is usually around 300 MB. The ubulk file is responsible for a lot of the lighting and terrain data.

There needs to be significant refactors to how deserialization is done for large files. Unreal loads these on a chunk-by-chunk basis as they are accessed, CUE4Parse (and by extension every unreal asset processor) loads the entire file into memory.

A quick workaround is to lazy-load the ubulk until needed, or just not load large ubulks at all and silently fail things that need it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants