build_waitk_mask does not exist

[mako2sake]

Thank you so much for your work and open access.
I followed Quick Start in the readme.

While training, the program runs throuh this below line.
But maybe build_waitk_mask function does not exist.

When I commented out this else scope, the train went without stopping.
But I cannot know whether way to fix is correct.

DiSeg/fairseq/modules/waitseg_multihead_attention.py

Line 337 in feff06a

waitk_mask = self.build_waitk_mask(attn_weights, training_lagging_seg)

I searched ictnlp's repo, and I found the similar code in the MoE-Waitk repo.

https://github.com/ictnlp/MoE-Waitk/blob/6f8ca9834c2ab77785ebd93fd569f73c3819340b/fairseq/modules/moe_waitk_multihead_attention.py#L400

I copied this build_waitk_mask function. But i got effor masked_fill only supports boolean masks, but got dtype Float in following procedure.

Please kindly tell me how to fix.

Thank you.

[zhangshaolei1998]

Sorry, I just saw your issue. This function was deleted by mistake when standardizing the code. I've corrected it in the new commit.
Refer to: waitseg_multihead_attention.py

[mako2sake]

@zhangshaolei1998 Thanks a lot for your reply. I noticed it soon. This problem is solved. Train works well.

But, there are some troble wihr evaluation.
I should comment by another issue?

I comment here some.

  File "~/DiSeg/examples/simultaneous_translation/models/convtransformer_simul_trans.py", line 13, in <module>
    from fairseq.models.speech_to_text import (
ImportError: cannot import name 'convtransformer_espnet_wav2vec_encwav' from 'fairseq.models.speech_to_text' (~/DiSeg/fairseq/models/speech_to_text/__init__.py)

=> should it comment out like this?

from fairseq.models.speech_to_text import (
    ConvTransformerModel,
    convtransformer_espnet,
    convtransformer_espnet_wav2vec,
    # convtransformer_espnet_wav2vec_encwav,
    # ImportError: cannot import name 'convtransformer_espnet_wav2vec_encwav' from 'fairseq.models.speech_to_text' (/home/s2210411/DiSeg/fairseq/models/speech_to_text/__init__.py)
    ConvTransformerModelWac2VecSeg,
    ConvTransformerEncoder,
    ConvTransformerModelWac2Vec,
    # ConvTransformerCIFModel,
    # ConvTransformerModelWac2VecCIF,
    # convtransformer_espnet_cif,
    # convtransformer_espnet_wav2vec_cif,
)

  File "~/DiSeg/fairseq/models/__init__.py", line 121, in register_model_cls
    raise ValueError("Cannot register duplicate model ({})".format(name))
ValueError: Cannot register duplicate model (convtransformer_simul_trans)

=> I don't know how to solve. It seems like this error exists in fairseq repo's issue.