Transfer Learning using Save_and_Load.py

[Mahmedturk]

Hi @nreimers

I have tried to use Save_and_Load.py for pre-training on a bigger dataset and after reloading training on a smaller set. But it seems that Save_and_Load.py only works for the same dataset. Like you train for few epoch on a dataset, then save the model and then continue training the new loaded model. Because if i use new dataset after loading pre-existing model it throws a Key-Error. Is this how it is supposed to be working?

[nreimers]

Are the labels / tags the identical between the two datasets?

If not, continuing training is much more challenging, as the CRF no longer fits. There, you would need to replace the CRF layer with a new layer for your dataset. This is not support right now in this framework.

[Mahmedturk]

Yes the labels are exactly the same (BIO tagged). I even have tried continuing training by making a copy of first dataset, it still produces the key-error.