external-media folder use from within a virtual machine

[frecuencialibre]

I'm seeing errors upon trying to upload tracks to an instance I created within a virtual machine that attempts to store files on the base system (aka. barebone).

Contrasting my working local instance where docker is running directly within my laptop operating system, and this new instance on our radio server which runs within a virtual machine created with virt-manager, I first noticed a difference in ownership of the /external-media folder.

On my working local:

root@0c7604ccca0f:/# ls -la /external-media/
total 16
drwxrwxrwx 4 www-data www-data 4096 Sep 26 18:31 .
drwxr-xr-x 1 root     root     4096 Oct  1 18:37 ..
drwxrwxrwx 3 www-data www-data 4096 Sep 26 18:31 imported
drwxrwxrwx 2 www-data www-data 4096 Oct  2 22:04 organize

vs. on the server where the linked folder is mounted via sshfs within the VM:

root@8af2f98a491c:/var/log/airtime# ls -la /external-media/
total 12
drwxrwxrwx 1 1000 celery 4096 Oct  2 21:42 .
drwxr-xr-x 1 root root   4096 Oct  2 19:03 ..
drwxr-xr-x 1 1000 celery 4096 Oct  2 22:03 organize
-rw-r--r-- 1 1000 celery    0 Oct  1 18:45 test.txt

At first I wasn't able to change the folder owner:

root@8af2f98a491c:/var/log/airtime# chown www-data:www-data /external-media/
chown: changing ownership of '/external-media/': Permission denied

And so from within the VM I then unmounted the sshfs mountpoint, but strangely didn't see changes immediately reflected within the docker container. I removed and created libretime-core and then could see changes reflected, and was able to change the folder owner.

I'm currently thinking the cleanest way to use a volume on the "barebone" would be to use a volume driver, but I'm so new to docker that I may be completely wrong 😝 . I'll try this and report back.

thoughts? thanks!

[frecuencialibre]

hello again, continuing to post my updates here from my process of learning about using docker to best deploy LibreTime in hopes that others may benefit.

I've almost been able to successfully "cut out the middle man" by directly mounting a folder from the server barebone within libretime-core. The volume is successfully created and mounted, but remains "read-only" and I'm at a stage where I think posting an update is appropriate. :) If I can't resolve this then I'll have to abandon using the VM -- not the end of the world, and actually would simplify things, but the system would be a little less robust in that I'd no longer be able to simply restore to a stable clone.

My updated docker-compose.yml adds a volume as per the syntax I found here:

volumes:
  externalmedia:
    driver: vieux/sshfs:latest
    driver_opts:
      sshcmd: '[email protected]:/home/frecuencialibre/Música'
      allow_other: ''

Note that the vieux/sshfs docker plugin must be added. This command also sets plugin configuration to enable debugging and use the public key I added to the server to which I'm connecting:

docker plugin install --grant-all-permissions vieux/sshfs DEBUG=1 sshkey.source=/home/libretime/.ssh/

I then use this volume within the libretime-core service, replacing the volume set with the LOCAL_MUSIC_MAPPING variable:

    volumes:
      # This structure has not yet been renamed in the fork of libretime from airtime.
      - ./config/airtime:/etc/airtime

      # Airtime customisations directory...
      - ./customisations:/etc/airtime-customisations

      # NB: Libretime will create some directories in this location (for user uploads, recorded podcasts etc)
      # Change the `/localmusic` part of this directive to the location of your media files..
      - 'externalmedia:/external-media:ro'

This works, but the drive is loaded as read-only, which of course won't do. I'm assuming that the "ro" at the end of the last line above, copied from the github issue, stands for "read-only," and so I need to figure out how to ditch it. Unfortunately my various attempts to do so have consistently caused permission errors upon creating libretime-core. I'll update this as I continue to experiment with sshfs options, but one thing I don't understand why these options appear to be able to be added both in the definition of the volume, and also in it's use for the service.

[ned-kelly]

This works, but the drive is loaded as read-only, which of course won't do. I'm assuming that the "ro" at the end of the last line above, copied from the github issue, stands for "read-only," and so I need to figure out how to ditch it. Unfortunately my various attempts to do so have consistently caused permission errors upon creating libretime-core. I'll update this as I continue to experiment with sshfs options, but one thing I don't understand why these options appear to be able to be added both in the definition of the volume, and also in it's use for the service.

Hi Ryan - I've already sent you a PM on Skype, however just posting a summary of comments/chat history here in the Issue so anyone else coming across the issue will also have a bit of context to the outcome...

• It's best you just use the underlying OS to mount any volumes/network shares in the context of Libretime -- Ideally you should be setting Samba/NFS mounts in /etc/fstab (on the host OS) then just specifying the path to that mount in .env file under the LOCAL_MUSIC_MAPPING directive.

• Try avoid using SSHFS in production if possible, you'll get much better performance with SMB/NFS - You shouldn't be having any read/write issues with SSHFS regardless, so perhaps again try mounting it in your host's /etc/fstab and testing it locally before booting up the containers.

• You're right about the :ro at the end of your docker compose causing that mount point to be readonly - just remove it and the mount point will be read/write accessible like a normal mount - That said, if you can use the LOCAL_MUSIC_MAPPING directive in your .env file it's going to be better than changing the docker-compose file, this way as we ship new features to the project, it won't keep over-writing your docker-compose file each time you pull down a new build of docker-multicontainer-libretime.

In summary, use /etc/fstab on the parent host if possible - It's less moving parts for Docker to worry about also.

[frecuencialibre]

many thanks ned.

OK, I'm finally getting it that I should stop clinging to using VMs. My inexperience with docker has made me hesitant to go against the advice to never deploy directly in the barebone, but I see that the ability to commit docker containers to new images gives some ability to back them up just like how i used to clone containers. ...and there's still some isolation from the barebone OS just in case i crash the container.

Today I picked Samba and got it up and running pretty easily on my debian server. it's sharing our media folder correctly in read only mode with the windows computer used during live shows in the studio. sweet! thanks again!

[ned-kelly]

Today I picked Samba and got it up and running pretty easily on my debian server. it's sharing our media folder correctly in read only mode with the windows computer used during live shows in the studio. sweet! thanks again!

Glad you got it running - Just a heads up, generally in a larger scale production env, you will still see something like ESXi used and docker running within VM's - This way you can use tooling like VMotion and/or Veeam to shift the VM's around (between your hypervisors) if you need to take down the physical infrastructure for maintenance etc... I realise in your environment it's probably overkill, so if barebones + docker gets you by fine and you're happy with your backup options, then I'd stick with it.

I'm actually running our prod setup just in a lightweight Container OS on an ESXi VM (CoreOS) rather than a traditional more "fully fledged" OS inside a VM.