add Qwen2-VL static generation #1512
Conversation
Spycsh
requested review from
ssarkar2,
bhargaveede,
vivekgoe and
regisss
as code owners
|
The pipeline test will not pass until optimum-habana matches the latest changes in transformers huggingface/transformers#34769, namely the task name |
Are you saying we need transformer4.47 for this to work or can you update the examples/images-to-text/run_pipeline to support both cases? |
| @@ -974,14 +979,16 @@ def generate( | |||
| # 1. Handle `generation_config` and kwargs that might update it, and validate the `.generate()` call | |||
| self._validate_model_class() | |||
| tokenizer = kwargs.pop("tokenizer", None) # Pull this out first, we only use it for stopping criteria | |||
| #assistant_tokenizer = kwargs.pop("assistant_tokenizer", None) # only used for assisted generation | |||
There was a problem hiding this comment.
please remove all this commented out codes.
There was a problem hiding this comment.
I think it is also affected by transformers 4.47. Will remove or uncomment it when transformers has its v4.47.0 release & optimum habana update to match 4.47.
| if hpu_graphs and not lazy_mode: | ||
| raise ValueError( | ||
| "`hpu_graphs` is True but `lazy_mode` is False. HPU graphs require `lazy_mode` to be set to True." | ||
| ) | ||
| num_virtual_tokens = kwargs.pop("num_virtual_tokens", 0) | ||
| generation_config, model_kwargs = self._prepare_generation_config(generation_config, **kwargs) | ||
| self._validate_model_kwargs(model_kwargs.copy()) | ||
| self._validate_assistant(assistant_model) | ||
| #self._validate_assistant(assistant_model, tokenizer, assistant_tokenizer) | ||
| self._validate_assistant(assistant_model,) | ||
|
|
There was a problem hiding this comment.
Is this file(utils.py) change needed? please remove if it's not needed.
|
@tthakkal Could you also review this please. THanks. |
Yes. An update to latest transformers is needed here. |
What does this PR do?
Add Qwen2-VL static generation.
Before submitting