MultiWOZ 2.2, 2.3 데이터셋을 사용할 수 있도록 코드 수정. #4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
som-dst모델에서 기본적으로 MultiWOZ 2.1을 사용하고 있는데요.
최근에 2.2와 2.3이 릴리즈되어서 최신 데이터셋도 사용할 수 있도록 해보았습니다.
데이터셋은 2.1 포맷으로 컨버팅하여 JJinIT/som-dst에 wget으로 다운로드 가능하게 해두었구요.
개인적으로 테스트해보니 2.2일떄 조금 더 성능이 잘 나오는것 같네요.
2.3에서는 성능이 현저하게 떨어지는데, 저희쪽 실험 환경이 잘못되었을 수 있습니다.
아래는 Google Colab에서
batch_size=16
으로 30 epochs 돌리고 얻은 evaluation 결과들입니다.2.1 (baseline)
2.2 (https://github.com/budzianowski/multiwoz/tree/master/data/MultiWOZ_2.2)
2.3 (https://github.com/lexmen318/MultiWOZ-coref)
cc. @d4rk6un