Diarization Causes High System Memory Usage

[agorman]

Tested versions

pyannote.audio 3.1.1
pyannote.core 5.0.0
pyannote.database 5.1.0
pyannote.metrics 3.2.1
pyannote.pipeline 3.0.1

System information

Rocky Linux 9 - NVIDIA GeForce GTX 1070 - 16GB system RAM

Issue description

I'm seeing very high memory usage when performing diarization on a long audio file (4 hours). The process uses around 12GB of RAM at the end of the embeddings phase. Diarization detects around 50 speakers with this audio.

This isn't necessarily a bug but I'm looking for suggestions or solutions to mitigate the issue.

The colab MRE will run out of memory at the end of the embeddings phase but can be used to demonstrate the issue.

Minimal reproduction example (MRE)

https://colab.research.google.com/gist/agorman/38ef94e8b4ae7fd3fef474e49c5b212a/mre_template.ipynb