Too many open files

[gbook]

I'm experiencing an intermittent error. When using the following function:

bool squirrel::CompressDirectoryToArchive(QString dir, QString archivePath, QString &m) {
    utils::Print("Inside CompressDirectoryToArchive()");

    try {
        using namespace bit7z;
#ifdef Q_OS_WINDOWS
        Bit7zLibrary lib("C:/Program Files/7-Zip/7z.dll");
#else
        Bit7zLibrary lib("/usr/libexec/p7zip/7z.so");
#endif
        if (archivePath.endsWith(".zip", Qt::CaseInsensitive)) {
            BitArchiveWriter archive(lib, BitFormat::Zip);
            archive.setUpdateMode(UpdateMode::Update);
            archive.addDirectory(dir.toStdString());
            archive.compressTo(archivePath.toStdString());
        }
        else {
            BitArchiveWriter archive(lib, BitFormat::SevenZip);
            archive.setUpdateMode(UpdateMode::Update);
            archive.addDirectory(dir.toStdString());
            archive.compressTo(archivePath.toStdString());
        }
        m = "Successfully compressed directory [" + dir + "] to archive [" + archivePath + "]";
        return true;
    }
    catch ( const bit7z::BitException& ex ) {
        /* Do something with ex.what()...*/
        m = "Unable to compress directory into archive using bit7z library [" + QString(ex.what()) + "]";
        return false;
    }
}

... sometimes it works correctly and writes out the 7zip file. Sometimes it fails with bit7z giving this error from ex.what():

Failed to open the output file: Too many open files

What might be happening?

[rikyoz]

This is weird. What operating system is this happening on?

[gbook]

RHEL8

…

On Tue, May 7, 2024 at 3:52 PM Riccardo ***@***.***> wrote: This is weird. What operating system is this happening on? — Reply to this email directly, view it on GitHub <#209 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K5B43JMOWWQVI22Q6PX3ZBEWJPAVCNFSM6AAAAABHLHTQMKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGNBWGUYTS> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rikyoz]

As far as I remember, bit7z doesn't keep many opened files.
The only exception is when creating a multi-volume archive, but this is not your case.
Are there many files inside the directory you're compressing?
I'm not sure this an issue with bit7z, but I'll investigate it!

[gbook]

It's intermittent. I've run lsof while running my program with bit7z, and the number of open file handles is minmal. Maybe 50 or so open files, but way less than the OS limit of 1024.

[rikyoz]

Really strange. I ran some tests in my Fedora 28 VM but couldn't reproduce the problem.

In my tests, I used the strace command to trace the calls to the open, openat and close syscalls performed by the same test program I created to diagnose issue #206.
The program always closed the files it opened, and never kept too many files open at the same time.

Unfortunately, the fact that it's intermittent doesn't make it easy to replicate.

[gbook]

I'll keep trying it on my end and see what I can find.