generation utils update (minor)

[yafshar]

What does this PR do?

• Fix import path for streamers module from transformers.streamers -> transformers.generation.streamers
• Fix the _prepare_decoder_attention_mask interface interface
  • return x.index_fill(1, torch.tensor(0), 1) uses the wrong index of torch.tensor(0), it is fixed to the correct index on the correct device index = torch.tensor(0, device=device)
• Improve the _pad_past_key_values function

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[yafshar]

The text-generation CI has been executed and will be compared with the main branch once the run is complete.

[emascarenhas]

@yafshar , Just a couple of comments below.
Please post results of CI, before and after change.

[emascarenhas]

@yafshar , Makes sense.
Please post the CI results when available and we can move it further along.

[emascarenhas]

@yafshar , Could you post CI results. Thanks.

[regisss]

LGTM!

Aligned with @emascarenhas, let's make sure there is no regression in generation tests and then I'll merge it 🙂

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yafshar]

I am doing slow CI tests python -m pytest tests/test_text_generation_example.py tests/test_encoder_decoder.py -v -s Till now all have been matched exactly. I will finish all the tests by tomorrow

[emascarenhas]

I am doing slow CI tests python -m pytest tests/test_text_generation_example.py tests/test_encoder_decoder.py -v -s Till now all have been matched exactly. I will finish all the tests by tomorrow

@yafshar , Can you post results here from the CI tests. Thanks.

[yafshar]

I just finished the CI tests

on both main and this PR on the same machine

>>> python -m pytest tests/test_text_generation_example.py tests/test_encoder_decoder.py -v -s
4 failed, 59 passed

I checked the failures

-> test_text_generation_bf16_1x[token0-EleutherAI/gpt-j-6b-1-False-160.5823842101192-False]
pr:   FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-google/gemma-7b-1-False-109.70751574382221-True] - AssertionError: assert 'DeepSpeed is...be efficient,' == 'DeepSpeed is... PyTorch, and'
main: FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-google/gemma-7b-1-False-109.70751574382221-True] - AssertionError: assert 'DeepSpeed is...be efficient,' == 'DeepSpeed is... PyTorch, and'

-> test_text_generation_bf16_1x[token0-state-spaces/mamba-130m-hf-1536-False-5385.511100161605-False]
pr:   FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-state-spaces/mamba-130m-hf-1536-False-5385.511100161605-False] - assert 4895.173518373703 >= ((2 - 1.05) * 5385.511100161605)
main: FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-state-spaces/mamba-130m-hf-1536-False-5385.511100161605-False] - assert 4895.212904578489 >= ((2 - 1.05) * 5385.511100161605)

-> test_text_generation_bf16_1x[token0-Deci/DeciLM-7B-1-False-120-False]
pr:   FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-Deci/DeciLM-7B-1-False-120-False] - assert 107.58924903315328 >= ((2 - 1.05) * 120)
main: FAILED tests/test_text_generation_example.py::test_text_generation_bf16_1x[token0-Deci/DeciLM-7B-1-False-120-False] - assert 107.56332773820075 >= ((2 - 1.05) * 120)

-> test_text_generation_fp8[token0-tiiuae/falcon-180B-4-950-True-128-128-2506.68]
pr:   FAILED tests/test_text_generation_example.py::test_text_generation_fp8[token0-tiiuae/falcon-180B-4-950-True-128-128-2506.68] - AssertionError: The following command failed:
main: FAILED tests/test_text_generation_example.py::test_text_generation_fp8[token0-tiiuae/falcon-180B-4-950-True-128-128-2506.68] - AssertionError: The following command failed:

The failures are exactly the same.

[yafshar]

@regisss @emascarenhas I do not see any regression. The behavior is the same as far as I tested