2.3.x Memory and CPU

[vitoyucepi]

Summary

I would like to share a comparison of memory and CPU usage between 2.2 and 2.3.

Setup

1. compose.yml

   services:
     ls22:
       image: savonet/liquidsoap:rolling-release-v2.2.x
       command: /tmp/test/main.liq
       volumes:
         - ./:/tmp/test
     ls23:
       image: savonet/liquidsoap:rolling-release-v2.3.x
       command: /tmp/test/main.liq
       volumes:
         - ./:/tmp/test
     prometheus:
       image: prom/prometheus
       volumes:
         - ./:/etc/prometheus
       ports:
         - 8000:9090

2. prometheus.yml

   global:
     scrape_interval: 15s
     scrape_timeout: 10s
     evaluation_interval: 15s
   scrape_configs:
     - job_name: ls22
       static_configs:
         - targets:
           - ls22:9599
     - job_name: ls23
       static_configs:
         - targets:
           - ls23:9599

3. main.liq

   settings.prometheus.server := true
   settings.prometheus.server.port := 9599
   
   %include "liqme/src/index.liq"
   
   vitoyucepi.liqme.prometheus.process_metrics()
   vitoyucepi.liqme.prometheus.cpu_metrics()
   vitoyucepi.liqme.prometheus.memory_metrics()
   vitoyucepi.liqme.prometheus.gc_metrics()
   vitoyucepi.liqme.prometheus.well_known_metrics()
   
   output.dummy(blank())

Results

My guess is that the spikes in CPU usage are a result of the turbo boost being turned on.

I don't know why liquidsoap 2.3 has 450 MB of managed memory for the first 50 minutes.

Logs

Liquidsoap 2.2

2024/07/22 20:55:00 >>> LOG START
2024/07/22 20:54:54 [ffmpeg.filter.bitstream:3] No valid mode found for filter pgs_frame_merge!
2024/07/22 20:54:54 [main:3] Liquidsoap 2.2.5+git@528bd6de4
2024/07/22 20:54:54 [main:3] Using: alsa=0.3.0 angstrom=0.16.0 ao=0.2.4 asetmap=0.8.1 asn1-combinators=0.2.6 astring=0.8.5 base64=3.5.1 bigarray=[distributed with Ocaml] bigarray-compat=1.1.0 bigstringaf=0.9.1 bjack=0.1.6 bos=0.2.1 bytes=[distributed with OCaml 4.02 or above] ca-certs=v0.2.3 camlp-streams camomile.lib=2.0 cohttp=5.3.1 cohttp-lwt=5.3.0 cohttp-lwt-unix=5.3.0 conduit=6.2.3 conduit-lwt=6.2.3 conduit-lwt-unix=6.2.3 cry=1.0.3 cstruct=6.2.0 ctypes=0.22.0 ctypes-foreign=0.22.0 ctypes.stubs=0.22.0 curl=0.9.2 domain-name=0.4.0 domain_shims dssi=0.1.5 dtools=0.4.5 dune-build-info=3.16.0 dune-private-libs.dune-section=3.16.0 dune-site=3.16.0 dune-site.private=3.16.0 duppy=0.9.4 eqaf=0.9 eqaf.bigstring=0.9 eqaf.cstruct=0.9 faad=0.5.2 fdkaac=0.3.3 ffmpeg-av=1.1.10 ffmpeg-avcodec=1.1.10 ffmpeg-avdevice=1.1.10 ffmpeg-avfilter=1.1.10 ffmpeg-avutil=1.1.10 ffmpeg-swresample=1.1.10 ffmpeg-swscale=1.1.10 fileutils=0.6.4 flac=0.5.1 flac.decoder=0.5.1 flac.ogg=0.5.1 fmt=0.9.0 fpath=0.7.3 frei0r=0.1.2 gd=1.1 gen=1.1 gmap=0.3.0 hkdf=1.0.4 inotify=2.0-62-g5e58536 integers ipaddr=5.6.0 ipaddr-sexp=5.6.0 ipaddr.unix=5.6.0 irc-client irc-client-unix ladspa=0.2.2 lame=0.3.7 lastfm=0.3.4 lilv=0.2.0 liquidsoap-lang=2.2.5 liquidsoap-lang.console=2.2.5 liquidsoap_alsa=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ao=rolling-release-v2.2.x-3-g528bd6d liquidsoap_bjack=rolling-release-v2.2.x-3-g528bd6d liquidsoap_builtins=rolling-release-v2.2.x-3-g528bd6d liquidsoap_core=rolling-release-v2.2.x-3-g528bd6d liquidsoap_dssi=rolling-release-v2.2.x-3-g528bd6d liquidsoap_faad=rolling-release-v2.2.x-3-g528bd6d liquidsoap_fdkaac=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ffmpeg=rolling-release-v2.2.x-3-g528bd6d liquidsoap_flac=rolling-release-v2.2.x-3-g528bd6d liquidsoap_frei0r=rolling-release-v2.2.x-3-g528bd6d liquidsoap_gd=rolling-release-v2.2.x-3-g528bd6d liquidsoap_irc=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ladspa=rolling-release-v2.2.x-3-g528bd6d liquidsoap_lame=rolling-release-v2.2.x-3-g528bd6d liquidsoap_lastfm=rolling-release-v2.2.x-3-g528bd6d liquidsoap_lilv=rolling-release-v2.2.x-3-g528bd6d liquidsoap_lo=rolling-release-v2.2.x-3-g528bd6d liquidsoap_mad=rolling-release-v2.2.x-3-g528bd6d liquidsoap_mem_usage=rolling-release-v2.2.x-3-g528bd6d liquidsoap_memtrace=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ogg=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ogg_flac=rolling-release-v2.2.x-3-g528bd6d liquidsoap_optionals=rolling-release-v2.2.x-3-g528bd6d liquidsoap_opus=rolling-release-v2.2.x-3-g528bd6d liquidsoap_osc=rolling-release-v2.2.x-3-g528bd6d liquidsoap_oss=rolling-release-v2.2.x-3-g528bd6d liquidsoap_portaudio=rolling-release-v2.2.x-3-g528bd6d liquidsoap_posix_time=rolling-release-v2.2.x-3-g528bd6d liquidsoap_prometheus=rolling-release-v2.2.x-3-g528bd6d liquidsoap_pulseaudio=rolling-release-v2.2.x-3-g528bd6d liquidsoap_runtime=rolling-release-v2.2.x-3-g528bd6d liquidsoap_samplerate=rolling-release-v2.2.x-3-g528bd6d liquidsoap_sdl=rolling-release-v2.2.x-3-g528bd6d liquidsoap_shine=rolling-release-v2.2.x-3-g528bd6d liquidsoap_soundtouch=rolling-release-v2.2.x-3-g528bd6d liquidsoap_speex=rolling-release-v2.2.x-3-g528bd6d liquidsoap_srt=rolling-release-v2.2.x-3-g528bd6d liquidsoap_ssl=rolling-release-v2.2.x-3-g528bd6d liquidsoap_stereotool=rolling-release-v2.2.x-3-g528bd6d liquidsoap_taglib=rolling-release-v2.2.x-3-g528bd6d liquidsoap_theora=rolling-release-v2.2.x-3-g528bd6d liquidsoap_tls=rolling-release-v2.2.x-3-g528bd6d liquidsoap_vorbis=rolling-release-v2.2.x-3-g528bd6d liquidsoap_xmlplaylist=rolling-release-v2.2.x-3-g528bd6d liquidsoap_yaml=rolling-release-v2.2.x-3-g528bd6d lo=0.2.0 logs=0.7.0 logs.fmt=0.7.0 logs.lwt=0.7.0 lwt=5.7.0 lwt.unix=5.7.0 macaddr=5.6.0 mad=0.5.3 magic-mime=1.3.1 mem_usage=0.1.1 memtrace=0.2.3 menhirLib=20231231 metadata=0.3.0 mirage-crypto=0.11.3 mirage-crypto-ec=0.11.3 mirage-crypto-pk=0.11.3 mirage-crypto-rng=0.11.3 mirage-crypto-rng.unix=0.11.3 mm=0.8.5 mm.audio=0.8.5 mm.base=0.8.5 mm.image=0.8.5 mm.midi=0.8.5 mm.video=0.8.5 ocplib-endian ocplib-endian.bigstring ogg=0.7.4 ogg.decoder=0.7.4 opus=0.2.3 opus.decoder=0.2.3 osc osc-unix parsexp=v0.16.0 pbkdf pcre=7.5.0 portaudio=0.2.3 posix-base=5a7f328 posix-socket=5a7f328 posix-socket.constants=5a7f328 posix-socket.stubs=5a7f328 posix-socket.types=5a7f328 posix-time2=5a7f328 posix-time2.constants=5a7f328 posix-time2.stubs=5a7f328 posix-time2.types=5a7f328 posix-types=5a7f328 posix-types.constants=5a7f328 ppx_sexp_conv.runtime-lib=v0.16.0 prometheus=1.2 prometheus-app=1.2 ptime=1.1.0 ptime.clock.os=1.1.0 pulseaudio=0.1.6 re=1.11.0 result=1.5 rresult=0.7.0 samplerate=0.1.7 saturn_lockfree=0.4.1 sedlex=3.2 seq=[distributed with OCaml 4.07 or above] sexplib=v0.16.0 sexplib0=v0.16.0 shine=0.2.3 soundtouch=0.1.9 speex=0.4.2 speex.decoder=0.4.2 srt=0.3.1 srt.constants=0.3.1 srt.stubs=0.3.1 srt.stubs.locked=0.3.1 srt.types=0.3.1 ssl=0.7.0 stdlib-shims=0.3.0 stereotool=rolling-release-v2.2.x-3-g528bd6d str=[distributed with Ocaml] stringext=1.6.0 taglib=0.3.10 theora=0.4.1 theora.decoder=0.4.1 threads=[distributed with Ocaml] threads.posix=[internal] tls=0.17.3 tsdl=v1.0.0 tsdl-image=0.5 tsdl-ttf=0.6 unix=[distributed with Ocaml] unix-errno=52c6ecb unix-errno.errno_bindings=52c6ecb unix-errno.errno_types=52c6ecb unix-errno.errno_types_detected=52c6ecb unix-errno.unix=52c6ecb uri=4.4.0 uri-sexp=4.4.0 uri.services=4.4.0 vorbis=0.8.1 vorbis.decoder=0.8.1 x509=0.16.5 xmlm=1.4.0 xmlplaylist=0.1.5 yaml=3.2.0 yaml.bindings=3.2.0 yaml.bindings.types=3.2.0 yaml.c=3.2.0 yaml.ffi=3.2.0 yaml.types=3.2.0 zarith=1.13
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] DISCLAIMER: This version of Liquidsoap has been compiled from a snapshot of the
2024/07/22 20:54:54 [main:3] development code. As such, it should not be used in production unless you know
2024/07/22 20:54:54 [main:3] what you are doing!
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] We are, however, very interested in any feedback about our development code and
2024/07/22 20:54:54 [main:3] committed to fix issues as soon as possible.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] If you are interested in collaborating to the development of Liquidsoap, feel
2024/07/22 20:54:54 [main:3] free to drop us a mail at <[email protected]> or to join the slack chat
2024/07/22 20:54:54 [main:3] at <http://slack.liquidsoap.info>.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] Please send any bug report or feature request at
2024/07/22 20:54:54 [main:3] <https://github.com/savonet/liquidsoap/issues>.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] We hope you enjoy this snapshot build of Liquidsoap!
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [clock:3] Using native (high-precision) implementation for latency control
2024/07/22 20:55:00 [main:3] Standard library loaded in 5.36 seconds.
2024/07/22 20:55:00 [liqme.prometheus.process:3] Adding process metrics to prometheus exporter.
2024/07/22 20:55:00 [liqme.prometheus.cpu:3] Adding cpu metrics to prometheus exporter.
2024/07/22 20:55:00 [liqme.prometheus.memory:3] Adding memory metrics to prometheus exporter.
2024/07/22 20:55:00 [liqme.prometheus.gc:3] Adding garbage collector metrics to prometheus exporter.
2024/07/22 20:55:00 [liqme.prometheus.well-known:3] Adding well known metrics to prometheus exporter.
2024/07/22 20:55:00 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2024/07/22 20:55:00 [frame:3] Video frame size set to: 1280x720
2024/07/22 20:55:00 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
2024/07/22 20:55:00 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2024/07/22 20:55:00 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
2024/07/22 20:55:00 [sandbox:3] Sandboxing disabled
2024/07/22 20:55:00 [startup:3] DSSI plugins registration: 0.00s
2024/07/22 20:55:00 [startup:3] FFmpeg filters registration: 0.03s
2024/07/22 20:55:00 [startup:3] FFmpeg bitstream filters registration: 0.00s
2024/07/22 20:55:00 [startup:3] Lilv plugins registration: 0.00s
2024/07/22 20:55:00 [startup:3] Frei0r plugin registration: 0.00s
2024/07/22 20:55:00 [startup:3] LADSPA plugins registration: 0.00s
2024/07/22 20:55:00 [startup:3] Typechecking: 5.10s
2024/07/22 20:55:00 [startup:3] Evaluation: 0.02s
2024/07/22 20:55:00 [startup:3] Typechecking: 0.04s
2024/07/22 20:55:00 [startup:3] Evaluation: 0.00s
2024/07/22 20:55:00 [startup:3] Typechecking: 0.02s
2024/07/22 20:55:00 [startup:3] Evaluation: 0.00s
2024/07/22 20:55:00 [startup:3] Loaded /tmp/test/main.liq: 0.03s
2024/07/22 20:55:00 [dummy:3] Content type is {audio=pcm(stereo)}.
2024/07/22 20:55:00 [blank.2:3] Content type is {audio=pcm(stereo)}.
2024/07/22 20:55:00 [video.converter:3] Using preferred video converter: ffmpeg.
2024/07/22 20:55:00 [audio.converter:3] Using samplerate converter: libsamplerate.
2024/07/22 20:55:00 [clock.main:3] Streaming loop starts in auto-sync mode
2024/07/22 20:55:00 [clock.main:3] Delegating synchronization to CPU clock
2024/07/22 20:55:00 [prometheus.server:3] Starting prometheus server on port 9599
2024/07/22 20:55:00 [video.text:3] Using sdl implementation
2024/07/23 09:24:04 [main:3] Shutdown signal received.
2024/07/23 09:24:04 [threads:3] Main loop exited
2024/07/23 09:24:04 [main:3] Shutdown started!
2024/07/23 09:24:04 [threads:3] Waiting for main threads to terminate...
2024/07/23 09:24:04 [clock.main:3] Streaming loop stopped.
2024/07/23 09:24:04 [threads:3] Main threads terminated.
2024/07/23 09:24:04 [threads:3] Shutting down scheduler...
2024/07/23 09:24:04 [threads:3] Scheduler shut down.
2024/07/23 09:24:04 [main:3] Cleaning downloaded files...
2024/07/23 09:24:04 [main:3] Freeing memory...
2024/07/23 09:24:04 >>> LOG END

Liquidsoap 2.3

2024/07/22 20:54:55 >>> LOG START
2024/07/22 20:54:54 [ffmpeg.filter.bitstream:3] No valid mode found for filter pgs_frame_merge!
2024/07/22 20:54:54 [main:3] Liquidsoap 2.3.0+git@b505b4cc9
2024/07/22 20:54:54 [main:3] Using: alsa=0.3.0 angstrom=0.16.0 ao=0.2.4 asetmap=0.8.1 asn1-combinators=0.2.6 astring=0.8.5 base=v0.16.3 base.base_internalhash_types=v0.16.3 base.caml=v0.16.3 base.shadow_stdlib=v0.16.3 base64=3.5.1 bigarray=[distributed with Ocaml] bigarray-compat=1.1.0 bigstringaf=0.9.1 bjack=0.1.6 bos=0.2.1 bytes=[distributed with OCaml 4.02 or above] ca-certs=v0.2.3 camlp-streams camomile.lib=2.0 cohttp=5.3.1 cohttp-lwt=5.3.0 cohttp-lwt-unix=5.3.0 conduit=6.2.3 conduit-lwt=6.2.3 conduit-lwt-unix=6.2.3 cry=1.0.3 cstruct=6.2.0 ctypes=0.22.0 ctypes-foreign=0.22.0 ctypes.stubs=0.22.0 curl=0.9.2 domain-name=0.4.0 domain_shims dssi=0.1.5 dtools=0.4.5 dune-build-info=3.16.0 dune-private-libs.dune-section=3.16.0 dune-site=3.16.0 dune-site.private=3.16.0 duppy=0.9.4 eqaf=0.9 eqaf.bigstring=0.9 eqaf.cstruct=0.9 faad=0.5.2 fdkaac=0.3.3 ffmpeg-av=1.2.0 ffmpeg-avcodec=1.2.0 ffmpeg-avdevice=1.2.0 ffmpeg-avfilter=1.2.0 ffmpeg-avutil=1.2.0 ffmpeg-swresample=1.2.0 ffmpeg-swscale=1.2.0 fileutils=0.6.4 flac=0.5.1 flac.decoder=0.5.1 flac.ogg=0.5.1 fmt=0.9.0 fpath=0.7.3 frei0r=0.1.2 gd=1.1 gen=1.1 gmap=0.3.0 hkdf=1.0.4 inotify=2.0-62-g5e58536 integers ipaddr=5.6.0 ipaddr-sexp=5.6.0 ipaddr.unix=5.6.0 irc-client irc-client-unix ladspa=0.2.2 lame=0.3.7 lastfm=0.3.4 lilv=0.2.0 liquidsoap-lang=2.3.0 liquidsoap-lang.console=2.3.0 liquidsoap_alsa=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ao=rolling-release-v2.3.x-3-gb505b4c liquidsoap_bjack=rolling-release-v2.3.x-3-gb505b4c liquidsoap_builtins=rolling-release-v2.3.x-3-gb505b4c liquidsoap_core=rolling-release-v2.3.x-3-gb505b4c liquidsoap_dssi=rolling-release-v2.3.x-3-gb505b4c liquidsoap_faad=rolling-release-v2.3.x-3-gb505b4c liquidsoap_fdkaac=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ffmpeg=rolling-release-v2.3.x-3-gb505b4c liquidsoap_flac=rolling-release-v2.3.x-3-gb505b4c liquidsoap_frei0r=rolling-release-v2.3.x-3-gb505b4c liquidsoap_gd=rolling-release-v2.3.x-3-gb505b4c liquidsoap_irc=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ladspa=rolling-release-v2.3.x-3-gb505b4c liquidsoap_lame=rolling-release-v2.3.x-3-gb505b4c liquidsoap_lastfm=rolling-release-v2.3.x-3-gb505b4c liquidsoap_lilv=rolling-release-v2.3.x-3-gb505b4c liquidsoap_lo=rolling-release-v2.3.x-3-gb505b4c liquidsoap_mad=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ogg=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ogg_flac=rolling-release-v2.3.x-3-gb505b4c liquidsoap_optionals=rolling-release-v2.3.x-3-gb505b4c liquidsoap_opus=rolling-release-v2.3.x-3-gb505b4c liquidsoap_osc=rolling-release-v2.3.x-3-gb505b4c liquidsoap_oss=rolling-release-v2.3.x-3-gb505b4c liquidsoap_portaudio=rolling-release-v2.3.x-3-gb505b4c liquidsoap_posix_time=rolling-release-v2.3.x-3-gb505b4c liquidsoap_prometheus=rolling-release-v2.3.x-3-gb505b4c liquidsoap_pulseaudio=rolling-release-v2.3.x-3-gb505b4c liquidsoap_runtime=rolling-release-v2.3.x-3-gb505b4c liquidsoap_samplerate=rolling-release-v2.3.x-3-gb505b4c liquidsoap_sdl=rolling-release-v2.3.x-3-gb505b4c liquidsoap_sdl_log_level=rolling-release-v2.3.x-3-gb505b4c liquidsoap_shine=rolling-release-v2.3.x-3-gb505b4c liquidsoap_soundtouch=rolling-release-v2.3.x-3-gb505b4c liquidsoap_speex=rolling-release-v2.3.x-3-gb505b4c liquidsoap_sqlite=rolling-release-v2.3.x-3-gb505b4c liquidsoap_srt=rolling-release-v2.3.x-3-gb505b4c liquidsoap_ssl=rolling-release-v2.3.x-3-gb505b4c liquidsoap_stdlib=rolling-release-v2.3.x-3-gb505b4c liquidsoap_stereotool=rolling-release-v2.3.x-3-gb505b4c liquidsoap_taglib=rolling-release-v2.3.x-3-gb505b4c liquidsoap_theora=rolling-release-v2.3.x-3-gb505b4c liquidsoap_tls=rolling-release-v2.3.x-3-gb505b4c liquidsoap_vorbis=rolling-release-v2.3.x-3-gb505b4c liquidsoap_xmlplaylist=rolling-release-v2.3.x-3-gb505b4c liquidsoap_yaml=rolling-release-v2.3.x-3-gb505b4c lo=0.2.0 logs=0.7.0 logs.fmt=0.7.0 logs.lwt=0.7.0 lwt=5.7.0 lwt.unix=5.7.0 macaddr=5.6.0 mad=0.5.3 magic-mime=1.3.1 mem_usage=0.1.1 memtrace=0.2.3 menhirLib=20231231 metadata=0.3.0 mirage-crypto=0.11.3 mirage-crypto-ec=0.11.3 mirage-crypto-pk=0.11.3 mirage-crypto-rng=0.11.3 mirage-crypto-rng.unix=0.11.3 mm=0.8.5 mm.audio=0.8.5 mm.base=0.8.5 mm.image=0.8.5 mm.midi=0.8.5 mm.video=0.8.5 ocplib-endian ocplib-endian.bigstring ogg=0.7.4 ogg.decoder=0.7.4 opus=0.2.3 opus.decoder=0.2.3 osc osc-unix parsexp=v0.16.0 pbkdf portaudio=0.2.3 posix-base=5a7f328 posix-socket=5a7f328 posix-socket.constants=5a7f328 posix-socket.stubs=5a7f328 posix-socket.types=5a7f328 posix-time2=5a7f328 posix-time2.constants=5a7f328 posix-time2.stubs=5a7f328 posix-time2.types=5a7f328 posix-types=5a7f328 posix-types.constants=5a7f328 ppx_compare.runtime-lib=v0.16.0 ppx_hash.runtime-lib=v0.16.0 ppx_sexp_conv.runtime-lib=v0.16.0 prometheus=1.2 prometheus-app=1.2 ptime=1.1.0 ptime.clock.os=1.1.0 pulseaudio=0.1.6 re=1.11.0 result=1.5 rresult=0.7.0 samplerate=0.1.7 saturn_lockfree=0.4.1 sedlex=3.2 seq=[distributed with OCaml 4.07 or above] sexplib=v0.16.0 sexplib0=v0.16.0 shine=0.2.3 soundtouch=0.1.9 speex=0.4.2 speex.decoder=0.4.2 sqlite3=5.1.0 srt=0.3.1 srt.constants=0.3.1 srt.stubs=0.3.1 srt.stubs.locked=0.3.1 srt.types=0.3.1 ssl=0.7.0 stdlib-shims=0.3.0 stereotool=rolling-release-v2.3.x-3-gb505b4c str=[distributed with Ocaml] stringext=1.6.0 taglib=0.3.10 theora=0.4.1 theora.decoder=0.4.1 threads=[distributed with Ocaml] threads.posix=[internal] tls=0.17.4 tsdl=v1.0.0 tsdl-image=0.5 tsdl-ttf=0.6 unix=[distributed with Ocaml] unix-errno=52c6ecb unix-errno.errno_bindings=52c6ecb unix-errno.errno_types=52c6ecb unix-errno.errno_types_detected=52c6ecb unix-errno.unix=52c6ecb uri=4.4.0 uri-sexp=4.4.0 uri.services=4.4.0 vorbis=0.8.1 vorbis.decoder=0.8.1 x509=0.16.5 xmlm=1.4.0 xmlplaylist=0.1.5 yaml=3.2.0 yaml.bindings=3.2.0 yaml.bindings.types=3.2.0 yaml.c=3.2.0 yaml.ffi=3.2.0 yaml.types=3.2.0 zarith=1.13
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] DISCLAIMER: This version of Liquidsoap has been compiled from a snapshot of the
2024/07/22 20:54:54 [main:3] development code. As such, it should not be used in production unless you know
2024/07/22 20:54:54 [main:3] what you are doing!
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] We are, however, very interested in any feedback about our development code and
2024/07/22 20:54:54 [main:3] committed to fix issues as soon as possible.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] If you are interested in collaborating to the development of Liquidsoap, feel
2024/07/22 20:54:54 [main:3] free to drop us a mail at <[email protected]> or to join the slack chat
2024/07/22 20:54:54 [main:3] at <http://slack.liquidsoap.info>.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] Please send any bug report or feature request at
2024/07/22 20:54:54 [main:3] <https://github.com/savonet/liquidsoap/issues>.
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [main:3] We hope you enjoy this snapshot build of Liquidsoap!
2024/07/22 20:54:54 [main:3] 
2024/07/22 20:54:54 [clock:3] Using native (high-precision) implementation for latency control
2024/07/22 20:54:55 [liqme.prometheus.process:3] Adding process metrics to prometheus exporter.
2024/07/22 20:54:55 [liqme.prometheus.cpu:3] Adding cpu metrics to prometheus exporter.
2024/07/22 20:54:55 [liqme.prometheus.memory:3] Adding memory metrics to prometheus exporter.
2024/07/22 20:54:55 [liqme.prometheus.gc:3] Adding garbage collector metrics to prometheus exporter.
2024/07/22 20:54:55 [liqme.prometheus.well-known:3] Adding well known metrics to prometheus exporter.
2024/07/22 20:54:55 [main:3] User script loaded in 0.62 seconds.
2024/07/22 20:54:55 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2024/07/22 20:54:55 [frame:3] Video frame size set to: 1280x720
2024/07/22 20:54:55 [frame:3] Targeting 'frame.duration': 0.02s = 882 ticks.
2024/07/22 20:54:55 [sandbox:3] Sandboxing disabled
2024/07/22 20:54:55 [startup:3] DSSI plugins registration: 0.00s
2024/07/22 20:54:55 [startup:3] FFmpeg filters registration: 0.03s
2024/07/22 20:54:55 [startup:3] FFmpeg bitstream filters registration: 0.00s
2024/07/22 20:54:55 [startup:3] Lilv plugins registration: 0.00s
2024/07/22 20:54:55 [startup:3] Frei0r plugin registration: 0.00s
2024/07/22 20:54:55 [startup:3] LADSPA plugins registration: 0.00s
2024/07/22 20:54:55 [startup:3] main script hash computation: 0.04s
2024/07/22 20:54:55 [startup:3] main script cache retrieval: 0.04s
2024/07/22 20:54:55 [startup:3] stdlib hash computation: 0.04s
2024/07/22 20:54:55 [startup:3] Loading stdlib from cache!
2024/07/22 20:54:55 [startup:3] stdlib cache retrieval: 0.16s
2024/07/22 20:54:55 [startup:3] Typechecking main script: 0.01s
2024/07/22 20:54:55 [startup:3] Error while saving cache: Sys_error("/usr/share/liquidsoap/.cache: Permission denied")
2024/07/22 20:54:55 [startup:3] Evaluating main script: 0.03s
2024/07/22 20:54:55 [clock:3] Starting clock generic with 2 source(s) and sync: auto
2024/07/22 20:54:55 [blank:3] Content type is {audio=pcm(stereo)}.
2024/07/22 20:54:55 [output.dummy:3] Content type is {audio=pcm(stereo)}.
2024/07/22 20:54:55 [dummy:3] Content type is {audio=pcm(stereo)}.
2024/07/22 20:54:55 [video.converter:3] Using preferred video converter: ffmpeg.
2024/07/22 20:54:55 [audio.converter:3] Using samplerate converter: libsamplerate.
2024/07/22 20:54:55 [prometheus.server:3] Starting prometheus server on port 9599
2024/07/22 20:54:55 [video.text:3] Using sdl implementation
2024/07/23 09:24:04 [main:3] Shutdown signal received.
2024/07/23 09:24:04 [threads:3] Main loop exited
2024/07/23 09:24:04 [main:3] Shutdown started!
2024/07/23 09:24:04 [threads:3] Waiting for main threads to terminate...
2024/07/23 09:24:04 [threads:3] Main threads terminated.
2024/07/23 09:24:04 [threads:3] Shutting down scheduler...
2024/07/23 09:24:04 [threads:3] Scheduler shut down.
2024/07/23 09:24:04 [main:3] Cleaning downloaded files...
2024/07/23 09:24:04 [main:3] Freeing memory...
2024/07/23 09:24:04 >>> LOG END

Error while saving cache: Sys_error("/usr/share/liquidsoap/.cache: Permission denied")

[toots]

Thanks. The cache directory has been fixed in the last docker image.

To really get an idea of the benefit of caching on memory consumption, I would suggest to first cache the script using liquidsoap --cache-only /path/to/script.liq and then run the test against the script. This should get rid of the initial bump in memory.

I have to write a full blog post about memory usage. It is a really tricky issue. First off the bat, the discrepancy above could very likely be explained by having disabled jemalloc in 2.3.0. This has been confirmed by Radio France to eat up a lot of initial memory.

Next, you have the issue of shared memory. This is the memory that is used by shared libraries when initiated. The more features you enable, the more shared memory you will consume.

Finally, each library has their own initialization routine which also allocates private memory. The more features, the more memory you will allocate, again.

Here's a graph of memory usage at startup with each optional dependency enabled on their own:

As you can see this varies a lot!

Here is the same graph with jemalloc enabled:

And, finally on 2.2.5:

So, for memory consumption, starting with release 2.3.x, our message will be:

if this is your concern, please make sure to build your own binary with only the features that you are using so as to make sure you minimize the memory used.

Also, please read insert future blog post for more details.