[Bug]: .rar file decompression failed. Error code is E_OUTOFMEMORY.

[yuzhengjun1204]

bit7z version

4.0.x

Compilation options

BIT7Z_AUTO_PREFIX_LONG_PATHS, BIT7Z_REGEX_MATCHING, BIT7Z_USE_NATIVE_STRING

7-zip version

v23.01

7-zip shared library used

7z.dll / 7z.so

Compilers

MSVC

Compiler versions

No response

Architecture

x86

Operating system

Windows

Operating system versions

windows 11

Bug description

When decompressing a large(about 2GB), password-protected RAR file, an "insufficient memory resources" error occurs, returning the error code E_OUTOFMEMORY. However, 7-Zip is able to decompress the same file without any issues.

Steps to reproduce

No response

Expected behavior

No response

Relevant compilation output

No response

Code of Conduct

• By submitting this issue, I agree to follow bit7z's Code of Conduct

[yuzhengjun1204]

I checked the 7-zip code and it seems like I need to add the IArchiveRequestMemoryUseCallback callback to the ExtractCallback class. I'm not sure if my understanding is correct.

[rikyoz]

Hi!
What version of the 7-Zip DLL are you using?
The IArchiveRequestMemoryUseCallback interface was added in 7-Zip v24.05.
If you're using this or a later version, could you try testing your code with the 7z.dll from v23.01 and see if it works?
Anyway, I'll have to add support for such an interface in a new version of the library, probably the next v4.1-beta.

[yuzhengjun1204]

Thank you!
Yes, It is v24.05.

[yuzhengjun1204]

I tested it with version 23.01, and it works fine.

[rikyoz]

I tested it with version 23.01, and it works fine.

Ok good, at least there's a possible workaround for the time being.
Thanks for the test!

I did some tests myself and I could not entirely replicate the issue.
In the 7-Zip code you already mentioned, the E_OUTOFMEMORY is returned only when you set the memory limit for the extraction operation, e.g.:

reader.useFormatProperty( L"memx", L"1g" ); // Setting the memory limit to 1 GB
reader.extractTo( BIT7Z_STRING( "test_rar" ) ); // Extracting an archive with a 2 GB dictionary.

In this case, I can replicate your issue. Is this also your case?

[yuzhengjun1204]

I did not modify any parameters during the extraction process. This issue occurs occasionally, and the compressed file is in RAR format with an encrypted filename.

[yuzhengjun1204]

It seems that the reason for insufficient memory resources could be due to memory fragmentation, which causes the failure to allocate 1GB of virtual memory. Are there any good solutions to address this issue?

[rikyoz]

Thanks for the further details!

Architecture

x86

It seems that the reason for insufficient memory resources could be due to memory fragmentation, which causes the failure to allocate 1GB of virtual memory. Are there any good solutions to address this issue?

I've done some further testing and I think I've replicated the problem: it seems to be due to the fact that you're building for x86 rather than x86_64 and therefore have a limited memory address space.
I hadn't replicated the problem before because I was testing for x86_64.

However, 7-Zip is able to decompress the same file without any issues.

Have you tested the decompression with the x86 version of 7-Zip?
I created an encrypted RAR archive similar in size to yours and was unable to extract it using x86 7-Zip:

However, when I use the x86_64 version, I don't have any problem.
The same happens when using bit7z with 7-Zip's DLLs: on x86, it fails with E_OUTOFMEMORY, while on x86_64 there's no issue.

Unfortunately, if this is your case too, it seems to be a problem with 7-Zip and the architecture you're targeting, rather than bit7z.

[yuzhengjun1204]

The 7-zip x86 runs normally, and it should be the LARGEADDRESSAWARE parameter causing the issue. After I added the LARGEADDRESSAWARE parameter, it worked fine.

[rikyoz]

This is interesting: I tested using LARGEADDRESSAWARE, but it didn't work in my case, so I didn't suggest it as a solution. I guess its effectiveness depends on the specific characteristics of the archive being extracted, although I used a similar RAR archive.
In any case, the important thing is that this is not a problem with bit7z.
I'll evaluate whether to add something like target_link_options(... INTERFACE /LARGEADDRESSAWARE) to bit7z's CMakeLists.txt, so that projects that include bit7z via CMake will automatically use this option when building for x86.