yata[WIP]

Yet Another Tools for Audio deep learning(for myself).

pip install libyata

Usage

(NEW!)As command line batch running tools

• usage

  yata glob_pattern --command command_pattern --replace_from foo --replace_to bar --j NUM_PROCESSES --dry_run True/False
  

  args:
  • --command: set command pattern.
    with placeholder of {i} as input file and {o} as output file
  • --j number of process to run
  • --dry_run: only show command to be run but not actual running it.
    for example: "mv 1.wav 1_backup.wav"
• examples

  • batch rename example:
    On terminal, just type this to see guide:

    yata_guide rename
    

    you will get a guide:

    yata './**/*' --command 'mv {i} {o}' --replace_from .aac --replace_to .wav --j 4 --dry_run True
    

    You can run it directly and will show you the dry running command to be run:

    yata date:20210418_161624, total_files:56, dry_run=True
    mv ./testdir/1.wav ./testdir/1.aac
    mv ./testdir/2i/2.wav ./testdir/2i/2.aac
    mv ./testdir_16k/1.wav ./testdir_16k/1.aac
    mv ./testdir_16k/2i/2.wav ./testdir_16k/2i/2.aac
    mv ./yata/61-70968-0002.wav ./yata/61-70968-0002.aac
    100%|██████████████████████████████████| 55/55 [00:00<00:00, 37724.73it/s]
    yata done:20210418_161705, total_files:56, dry_run=True
    

    You can modify it as your will and run it, remember to remove "--dry_run True":

    yata './**/*' --command   'mv {i} {o}' --replace_from .wav --replace_to _backup.wav --j 4
    

  • batch resample example:
    Type

    yata_guide resample
    

    to get this guide:

    yata './**/*' --command 'ffmpeg -hide_banner -loglevel quiet  -i {i} -f wav -ar 16000 -acodec pcm_s16le -ac 1 {o}  -y' --replace_from .aac --replace_to .wav --j 10 --dry_run True
    

• with pythonic support
  • tqdm
  • multiprocessing
  • auto new_dir

As python package

import yata

• handy tools

  • yata.utils.run():
    No more ArgumentParser!!
    you can pass and update any parameter with:

    python test.py --a 2 --lr 0.01
    

    with code like:

    import yata
    
    default_hp = {"a":1,"b":2}
    args = yata.util.run(default_hp)
    

    you can acess params like HParams:

    print(args.a, args.b) # acess default params
    print(args.lr) # acess newly add params from CLI
    

  • yata.utils.new_dir:
    Make directory like this ./file_a/tag/1/ with:

    new_dir("file_a", "tag", 1)
    

  • yata.utils.backup_code:
    Backup all your *.py(optional) to a zip file, eg. backup code for every experiments before running.
  • yata.utils.get_current_date: Get date as string
  • yata.utils.merge_parts: merge csv files
    • merge_parts(pattern, output_name="", sep=",")

• Tensorflow alternatives

  • yata.utils.HParams:
    An alternative to tf.contrib.training.HParams without Tensorflow dependency
  • yata.utils.to_categorical:
    An alternative to tf.keras.utils.to_categorical without Tensorflow & keras dependency

• data augmentation

  • mixup: paper code
  • SpecAugment: paper code
  • mp3 as augumentation，用MP3编码后去掉的不可听噪声，把这种生成不可听噪声作为数据增强的手段（做法：加高斯，把MP3当成一个mask去编码，把MP3mask挖掉的区域的高斯留下来加到频谱上面，形成不可听噪声）
  • phase putertubation

• feature extraction

  • PASE: paper code
  • Multi scale MelSpectrogram