Support of whisper params

[pminev]

All params for the inference can be checked here:

Whisper params needed for the MVP:

• translate -> translate the audio - seems like a reasonable feature
• no_context - do not use past transcription as initial prompt for the decoder
• no_timestamps - no timestamps generation
• single_segment - force single segment
• initial prompt
• language auto-detection
• new_segment_callback -> called for every newly generated text segment. We'll need this if we want to stream the result and not to wait until it's completed.
• progress_callback -> called on each progress update

Whisper params not needed for the MVP:

NOTE: All of the following will have valid default values, but will be not exposed to the clients

• params to enable/disable prints on stdout of the internal library during decoding. I don't think they will be useful by any means for the clients and we should hide them

  • print_special; // print special tokens (e.g. , , , etc.)
  • print_progress; // print progress information
  • print_realtime; // print results from within whisper.cpp (avoid it, use callback instead)
  • print_timestamps; // print timestamps for each text segment when printing realtime

• suppress_regex - desc

• common decoding params:

  • suppress_blank - suppresses blank outputs
  • suppress_non_speech_tokens
  • temperature - initial decoding temp
  • max_initial_ts - desc
  • length_penalty - desc

• fallback parameters - seems like they are for fine tuning

  • temperature_inc
  • entropy_thold
  • logprob_thold
  • no_speech_thold - not implemented In whisper either

• grammar -> GBNF grammar to guide decoding. At some point we might need it.

  • grammar_rules
  • n_grammar_rules
  • i_start_rule
  • grammar_penalty

• encoder_begin_callback -> called each time before the encoder starts

Experimental in whisper

• token level timestamps - check all of them here
  • token_timestamps
  • thold_pt
  • thold_ptsum
  • max_len
  • split_on_word
  • max_tokens
• debug_mode - extra info
• audio_ctx
• tdrz_enable

[pminev]

IMO experimental parameters are totally out of context for now.

[iboB]

Since the code for tokenization used for initial prompts is largely a copy-paste of llama.cpp, I'm wondering whether we can reuse it. I think that introducing a dependency to whisper.cpp on llama.cpp is not a terrible pain.

[pminev]

I need to investigate it a little bit further but isn't that too small chunk of code in order to introduce a dependency between the two?

[iboB]

new_segment_callback
progress_callback

Callbacks can be useful for the local implementation but we have no way of transferring them through the API as params. The functionality that callbacks cover will have to pass through the progress callback of sessions.