Skip to content

Hashformers v2.0.0: Enhanced Compatibility with Broad Range of Transformer Models including Large Language Models (LLMs)

Latest
Latest
Compare
Choose a tag to compare
@ruanchaves ruanchaves released this 03 Jun 15:52
· 3 commits to master since this release
v2.0.0
144b8f8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

The new hashformers v2.0.0 release marks a significant upgrade to our hashtag segmentation library. Earlier versions were compatible only with BERT and GPT-2 models. Now, hashformers v2.0.0 allows you to segment hashtags with nearly any transformer model, including Large Language Models (LLMs) like Dolly, GPT-J, and Alpaca-LoRA. This will enable us to achieve unprecedented state-of-the-art results that surpass our earlier benchmarks.

Key improvements in this release include:

  • Expanded Support: Accommodates various Seq2Seq models, Masked Language Models, and autoregressive Transformer models available on the Hugging Face Model Hub. This includes but is not limited to FLAN-T5, DeBERTa, XLNet.

  • Greater Language Coverage: Thanks to the increase in supported models, we can now segment hashtags in a broader range of languages. This includes improved support for large models pretrained in low-resource languages.

  • Enhanced Documentation: We have updated our documentation and wiki, providing in-depth explanations of each library feature. This will enable users to customize their usage of our library more effectively according to their specific needs.

Assets 2
Loading