New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

汉语中有做分词吗？还是直接在字级别做了 #10

Open

MrCuri0sity opened this issue Dec 19, 2018 · 2 comments

MrCuri0sity commented Dec 19, 2018

No description provided.

Owner

NLPScott commented Dec 20, 2018

这个不做分词，投入预训练transformer后，会进行切词，切词的依据是谷歌bert提供的词表

JepsonWong commented Dec 27, 2018

汉语预训练模型作者是在字级别做的，不是词级别。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment