v3.0.0: LLM support, Improved UI, Aggregator Refactor

Changes summary

Machine Learning and collaborative learning

• Add LLM support based on gpt-tfjs, for collaborative pre-training and fine-tuning #643 #644
• Split the collaborative training logic into the Aggregator and Client abstractions #578.
• Support text datasets and tokenization via Transformers.js #577 #651
• Benchmark gpt-tfjs #659
• Add browser support for LLM (tokenization, memory requirements, IndexedDB storage) #675
• Improve all base ML tasks #616 #660

Technical

• Update and extend documentation #614 #636, a table of content can be found in DEV.md
• Greatly simplify repository management via NPM workspaces #617
• Extend ESLint coverage #625
• Upgrade codebase to ES2022 modules and typescript #652
• Upgrade Node.js from v16 to v20 #653
• Update dependencies and fix security issues #655
• discojs is now a standalone package #664
  • discojs-node & discojs-web provide platform specific helpers
• Publish a container image to easily run the server #664
• Rolling releases for NPM packages and container image, each push to develop builds and deploys latest version #664

User Interface

• Fix bugs #608 #624
• Add user feedback during error handling #624
• Improve navigation #668
• UI design improvements #547 #686 #692

Downgrades

• The Byzantine-Robust aggregator support is temporarily inactive, due to the aggregator refactoring #578

Additional PRs merged

• v2.1.1 by @s314cy in #544
• Download and infer from remote models by @s314cy in #545
• clarify TASK docs by @morganridel in #550
• Fixing up assess(...) function by @francesco98 in #558
• more task doc update by @morganridel in #553
• adds generic training functions for trainer by @morganridel in #548
• Inference feature in the validation UI of the web client by @francesco98 in #543
• Updating GeoTags model by @francesco98 in #570
• Validation Metrics & UI Updates by @s314cy in #547
• Fix Geotags & Web Client by @s314cy in #574
• Add converter script to keras by @walidabn in #567
• Implementing end-to-end test with Byzantine-Robust Aggregator for cifar10 task by @francesco98 in #557
• aggregator and client refactor by @LucasTrg in #578
• Adding the skin_mnist task to the default tasks by @walidabn in #579
• Add support for text data & tokenization by @s314cy in #577
• Add API docs for disco by @s314cy in #580
• Reduce GCloud usage by @s314cy in #592
• Reduce GCloud usage by @s314cy in #593
• Reduce GCloud usage by @s314cy in #594
• reduce gcloud usage by @s314cy in #595
• Task creation form privacy parameters default by @walidabn in #596
• Remove stale package by @s314cy in #598
• CLI: Add placeholder for custom tasks and data by @s314cy in #599
• Refactoring created mismatched function calls by @JulienVig in #608
• Fix training related failures by @JulienVig in #616
• Update DISCO documentation by @JulienVig in #614
• Fix flaky titanic validator test by @JulienVig in #630
• *: use npm workspaces by @tharvik in #617
• Solve UI bugs and add UI error feedback by @JulienVig in #624
• full eslint by @tharvik in #625
• Update doc according npm workspace by @JulienVig in #636
• don't reexport tfjs/discojs-core by @tharvik in #640
• add fondations for LLM by @tharvik in #643
• #643 fixes by @tharvik in #645
• discojs-core/models: add gpt by @tharvik in #644
• bump node by @tharvik in #653
• *: upgrade to es2022 by @tharvik in #652
• Add tokenization and prompting API to GPT models by @JulienVig in #651
• updates dependencies by @tharvik in #655
• Fix gpt-tfjs bugs, add tests and refactor code by @JulienVig in #658
• small lus covid fixes by @tharvik in #661
• Benchmark GPT-tfjs by @JulienVig in #659
• LUS COVID demo by @JulienVig in #660
• Add benchmark results summary in cli/README.md by @JulienVig in #663
• docker: freeze node version by @tharvik in #672
• Improves and fixes the web-client UI by @JulienVig in #668
• rolling deploy by @tharvik in #664
• discojs/aggregator: rework tests by @tharvik in #676
• Create a Skin Condition classification task by @JulienVig in #678
• webapp/ModelLibrary: add some tests by @tharvik in #681
• Integrate the wikitext task in the webapp by @JulienVig in #675
• Fix DEV.md style by @JulienVig in #685
• UI polishing by @JulienVig in #686
• *: prerelease fixes by @tharvik in #688
• UI tweaks by @JulienVig in #692
• add auto-release by @tharvik in #693
• dependabot: fix for monorepo by @tharvik in #698
• webapp/TrainingInformation: fix graph jumps by @tharvik in #696
• docker: bump base image by @tharvik in #706
• Fix preprocessing memory leaks by @JulienVig in #711
• Update README.md by @JulienVig in #709
• Fix decentralized learning fail by @JulienVig in #708
• readd some privacy by @tharvik in #712
• Display landing page before server boots by @JulienVig in #720
• small UI changes by @tharvik in #725
• release v3 by @tharvik in #724

New Contributors

• @LucasTrg made their first contribution in #578
• @JulienVig made their first contribution in #608

Full Changelog: v2.1.1...v3.0.0