Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress and decompress misc json file with snappy compression #102

Open
ntedgi opened this issue Oct 11, 2022 · 9 comments
Open

Compress and decompress misc json file with snappy compression #102

ntedgi opened this issue Oct 11, 2022 · 9 comments
Labels
dependencies Pull requests that update a dependency file first-timers-only Please contribute! All are welcome. hacktoberfest Hacktoberfest 2022 help wanted Extra attention is needed

Comments

@ntedgi
Copy link
Owner

ntedgi commented Oct 11, 2022

After adding a lot of translations files the package size increase to ~600KB .

We need to compress to json file and to decompress them at runtime i prefer using snappy compression

then when loading a specific language provider we only decompress is language misc file

@ntedgi ntedgi added help wanted Extra attention is needed first-timers-only Please contribute! All are welcome. hacktoberfest Hacktoberfest 2022 dependencies Pull requests that update a dependency file labels Oct 11, 2022
@rokaicker
Copy link

Hey @ntedgi, if nobody is working on this, is it okay if I give it a shot? (I am not particalurly well-versed in machine learning, but this seems like something I might be able to help with)

But just to confirm - this is to compress the language files found here.

Then decompress the specific language that is specified by the "locale" key in the result object?

@ntedgi
Copy link
Owner Author

ntedgi commented Oct 17, 2022

Go for it thanks

@PranavBhatS
Copy link

@ntedgi anyone currently working on this, if not can i pick up this issue?

@ntedgi
Copy link
Owner Author

ntedgi commented Jan 29, 2023

Go for it

@SananditaDas
Copy link

Hey @ntedgi Can I pick up this issue if its not resolved yet?
Also I wanted to know, are you aiming for static and compressed translation files before uncompressing them at runtime?

@SananditaDas
Copy link

Hey @ntedgi Can I pick up this issue if its not resolved yet? Also I wanted to know, are you aiming for static and compressed translation files before uncompressing them at runtime?

I have replaced the json translation files with compressed binary-s (using snappy), and the code is working fine while decompressing the lang file at runtime.
But some of the testcases are failing, which are invoking the remote model.
Could you help?

@ntedgi
Copy link
Owner Author

ntedgi commented Mar 6, 2023

  • Update remote files links to the relative path at your working branch
  • fix the falling test
  • after test pass and mearge to master
  • Change test files to relative path at 'main' branch

@SananditaDas
Copy link

  • Update remote files links to the relative path at your working branch
  • fix the falling test
  • after test pass and mearge to master
  • Change test files to relative path at 'main' branch

Means, I need to change the remote model url to localModelRootDirectory , right? Then after merging with main branch, change it back to the original url?

@ntedgi
Copy link
Owner Author

ntedgi commented Mar 6, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file first-timers-only Please contribute! All are welcome. hacktoberfest Hacktoberfest 2022 help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants