Any way to load audio from a buffer, pydub, or numpy?

[jbmaxwell]

I have a process where I'm segmenting songs in 5-second chunks and doing analysis on the chunks. The process is currently based on librosa and I'd like to try using essentia instead.

Currently, I'm loading with pydub and creating AudioSegments for each 5-second chunk. Is there a way to reproduce this behaviour in essentia? I know about FrameCutter, but I feel like that's intended for the low-level analysis, whereas this is more a high-level chunking over which the low-level analysis will be performed. Is it maybe an option to have 2 FrameCutters, at different frame/hop sizes—e.g., framerate=sample_rate * 5.0, hopsize=sample_rate * 5.0 and framerate=4096, hopsize=2048?

[dbogdanov]

Hi, @jbmaxwell
You can use the FrameCutter in the way you mention. Make sure to set the startFromZero parameter as you need. There is also the Slicer algorithm that cuts into slices given start and end time positions.

To use buffer inputs in the streaming mode of Essentia you can use VectorInput.

[jbmaxwell]

One "hitch"... It isn't clear how to use VectorInput in this context. I seem to be able to instantiate it from audio_segment.data, but I can't connect it to framecutter.signal in my streaming network: vector_input.data >> framecutter.signal fails with:

Cannot convert data from type UNDEFINED (<class 'bytes'>) to type VECTOR_REAL

Can you advise on the proper approach? Just to clarify, I'm trying to connect it to my "inner" framecutter, which is there to analyze 5-second "chunks" of a longer audio file.

Thanks.

[jbmaxwell]

Excellent!
Thanks, @dbogdanov