Share basilisk's cache between the BBS and the SPB on the Linux builders

[hpages]

So I'm trying to find a satisfying solution for sharing basilisk's cache between the biocbuild (BBS) and pkgbuild (SPB) users on nebbiolo1.

For this to work, basilisk's cache and any subfolder would need to be writable by both users. A good place for such shared cache is /var/cache/basilisk (/var/cache is kind of the standard system-level cache location on Linux). This can be controlled with the BASILISK_EXTERNAL_DIR environment variable.

Now it's easy to manually set the permissions on this folder so that both biocbuild and pkgbuild can write to it, this is a one time thing to do. But we also need to make sure that any subfolder that gets automatically created after that will be writable by both users. One way to achieve this maybe is by having biocbuild and pkgbuild belong to the same group (right now they don't), and by setting the umask value appropriately. I'm going to explore that approach a little.

[hpages]

This is the setup I came up with:

1. Change the primary group of the pkgbuild user to make it the same as the primary group of the biocbuild user.

   Check their primary groups (gid) with id biocbuild and id pkgbuild.

   Assuming that biocbuild's primary group is biocbuild, make this the primary group of the pkgbuild user with:

   sudo usermod -g biocbuild pkgbuild
   

   Then check their primary groups again.

   An additional check is to check their effective group ids with id -g biocbuild and id -g pkgbuild. They should be the same.

2. Create basilisk's cache folder in /var/cache/:

   cd /var/cache
   sudo mkdir basilisk
   sudo chown biocbuild:biocbuild basilisk
   sudo chmod 775 basilisk
   
   

3. Basic testing. Log as biocbuild and do:

   cd /var/cache/basilisk
   mkdir titi
   

   The folder should be writable by any member of the group. Check this with:

   ls -ld titi
   

   This should display something like:

   drwxrwxr-x 2 biocbuild biocbuild 4096 Feb  8 01:04 titi
   

   In particular the flags must be drwxrwxr-x. If not, then set umask to 002 in ~/.profile, logout and login again for the change to take effect, and try to create another folder in /var/cache/basilisk/.

   Then log as pkgbuild and do:

   cd /var/cache/basilisk
   mkdir toto
   

   The folder should be writable by any member of the group:

   ls -ld toto
   

   This should display something like:

   drwxrwxr-x 2 pkgbuild biocbuild 4096 Feb  8 01:03 toto
   

   If not, then set umask to 002 in ~/.profile, logout and login again for the change to take effect, and try to create another folder in /var/cache/basilisk/.

   Finally clean /var/cache/basilisk/ with:

   cd /var/cache/basilisk
   rm -rf *
   

4. Set BASILISK_EXTERNAL_DIR to /var/cache/basilisk in ~/.profile for biocbuild and pkgbuild by putting the following line at the end of each file:

   export BASILISK_EXTERNAL_DIR="/var/cache/basilisk"
   

   Do NOT put this in /etc/profile!

   IMPORTANT: This setting won't be effective on the SPB until the SPB server gets restarted (even if, in an interactive R session, basilisk.utils::getExternalDir() points to the new location).

5. Next build run will repopulate /var/cache/basilisk/.

Let's see how things go in the next couple of days. If everything goes fine, then we'll be able to remove ~biocbuild/.cache/R/basilisk and ~pkgbuild/.cache/R/basilisk.

Checking again a couple of days later that ~biocbuild/.cache/R/basilisk and ~pkgbuild/.cache/R/basilisk have not been recreated will be an extra confirmation that the BBS and the SPB are now both using the new cache location. If we are curious, we could even look at the ownership of the stuff under /var/cache/basilisk, it should tell us who created what.

[jwokaty]

I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR on merida1 next, where both biocbuild and pkgbuild are members of group staff.

[martin-g]

It is important that the group is the primary one for both users! Otherwise newly created subfolders won’t be shareable.

…

On Thu, 8 Feb 2024 at 17:14, jwokaty ***@***.***> wrote: I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR on merida1 next, where both biocbuild and pkgbuild are members of group staff. — Reply to this email directly, view it on GitHub <#400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABYUQXTSKJWBKNIXJ2F3OTYSTTTZAVCNFSM6AAAAABC6ZIJPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZUGMZDSNJWGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[hpages]

It is important that the group is the primary one for both users! Otherwise newly created subfolders won’t be shareable.

@martin-g Absolutely!

I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR on merida1 next

@jwokaty That would be the next step.

I'm running into some gotchas with the new setup on nebbiolo1. The -00LOCK files and other files (e.g. basilisk/1.15.4/0/pkgs/cache/47929eba.json) under /var/cache/basilisk are created with restrictive permissions that do not let the pkgbuild user touch them. So despite the umask setting, it seems that when these files get created, their perms get explicitly restricted.

I'll investigate more.

[hpages]

Temporary hack (from a sudo account):

cd /var/cache/basilisk
sudo chmod --quiet -Rc g+rw .
sudo find . -type d -exec chmod --quiet -c go+rx {} \;

With this I can finally run R CMD build and R CMD check on sketchR as the pkgbuild user.

Of course far from being satisfying because these commands would need to be re-run every time more stuff gets added to the cache.

[hpages]

Similar setup on merida1 (but using .bash_profile instead of ~/.profile, and replacing --quiet with -f in chmod).

With this new setup sketchR passes R CMD build and R CMD check in 9m32s and 4m20s, respectively. The daily builds were not running though so it might take a little bit longer when they are.

[jwokaty]

I appreciate pointer; I did just get lucky. On Macs, you don't automatically get a group that is the same as your username, like Linux. If https://unix.stackexchange.com/questions/410367/how-to-get-the-primary-group-of-a-user#answer-410370 is correct, the primary group is staff.

merida1:~ biocbuild$ id -gn biocbuild
staff
merida1:~ biocbuild$ id -gn pkgbuild
staff

[hpages]

Yes, we are lucky that on merida1, unlike on nebbiolo1, biocbuild and pkgbuild already had the same primary group. That was one less thing to change 🍀 I don't think it would be too hard to change that if we had to though...