Main vocal vs chorus

[dwaipayansinha]

Please add the capability to split the vocals to main vocal and chorus parts.... currently it just gives one vocal track.

[RyanMetcalfeInt8]

Thanks for the recommendation!

If someone can recommend an open source AI model that is able to do this, I'd be happy to take a look at potentially porting the feature.

[RyanMetcalfeInt8]

Perhaps a good 'speaker source separation' model could be used here. And it's been a while since I've evaluated those, but I haven't been too impressed with the quality produced by the ones that I've found (that had pre-trained models available).

@MarcoRavich or others may have stumbled upon something better suited.

[MarcoRavich]

Hi there,
as you probably already know the "de facto" standards in AI audio separation field are demucs by @facebookresearch and spleeter by @deezer...
...as always there are many interesting competitors (such as @sigsep's Open-Unmix for example) listed @ HyMPS but, if you're looking for "state of the art" ones, I strongly suggest you to check recent international challenges results such as @sdx-workshop's one:

• Sound Demixing Challenge 2023 - Build A System To Perform Audio Source Separation
• The Sound Demixing Challenge 2023 (summary paper) – Music Demixing Track or Cinematic Demixing Track

Some GH-available projects/authors that participated in it:

• BS-RoFormer by @lucidrains;
• MVSEP-MDX23-music-separation-model by @ZFTurbo;
• kmdx-net Music Source Separation by @KimberleyJensen;
• kuielab by @kuielab (@rlaalstjr47 & @junicusmaximus);
• AIMLESS by @aim-qmul (@yoyololicon & c).

Last but not least, there are interesting issue/discussion in @Anjok07's Ultimate Vocal Remover GUI git where other interesting resources has been shared (such as @TRvlvr's repositories for example).

Hope that helps.

[Brodvd]

Small clarification:
I took advantage of this discussion to discuss a model that has capacity for separation of sound events wider, such as Audio Sep, perhaps in the specific case of @dwaipayansinha there is a need for a model on purpose, I do not know.
However we can choose more models.

[Brodvd]

I'm not sure but maybe Audio Sep can funtion? https://github.com/Audio-AGI/AudioSep#readme

[RyanMetcalfeInt8]

I'm not sure but maybe Audio Sep can funtion? https://github.com/Audio-AGI/AudioSep#readme

Whoa, nice -- that is super cool -- thanks for pointing it out. I haven't stumbled on this one yet.

[Brodvd]

You can tried It here: https://huggingface.co/spaces/Suniilkumaar/AudioSep

[RyanMetcalfeInt8]

Although reading through some of the issues filed in that GitHub project, it seems like many are not too impressed with the results in 'real world' use cases. So the concept for this one is great, but if the pre-trained model doesn't work very well, I'm going to just avoid it for now.

[RyanMetcalfeInt8]

You can tried It here: https://huggingface.co/spaces/Suniilkumaar/AudioSep

Thanks -- maybe I'll also give it a shot when I get a chance.

[Brodvd]

@RyanMetcalfeInt8 I think that Is better that you try. Also only for use It for voice and instruments Is good

[Brodvd]

@RyanMetcalfeInt8 Sorry if I fill you with notifications. I ask for a moment of patience. I share a little test I did with Audio Sep:

below the output of Audio Sep:

351649683-58dd06c5-31d9-4796-9347-54324b90059f.mp4

As you said the idea of this model is fantastic and (I think) it’s the only one that does this.

Obviously it is impossible that a model like this which must associate text with audio characteristics (with separation) totally covers all the sounds, just think of the thousands of ambient sounds, but if you use it consciously this model (as in this case) is absolutely great performance.

So I ask you not to discard Audio Sep at all but keep it very much in mind because if you use Audio Sep to separate musical instruments or voices, you don’t have to resort to stems.

Think about it

[RyanMetcalfeInt8]

wow, very cool, thanks for sharing -- I'll definitely give this a try. It would be a very cool feature to include, and if it works this well (as you said, for specific cases) I think it would come in very handy!

[Brodvd]

Thanks a lot :)
Sorry if I haven't others examples. I haven't Anaconda.

[Brodvd]

Update : I tried to run Audio Sep but there are conflict with the libreries, the same said here.
Anyway Audio Sep can be run with pinokio, like said here , but I don't know.
I used Miniconda 3.12 and python 3.9, maybe it can be funtion with Miniconda 3.9.