-
Notifications
You must be signed in to change notification settings - Fork 276
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BREAKING: v2.0.0 #1433
base: main
Are you sure you want to change the base?
BREAKING: v2.0.0 #1433
Commits on Nov 11, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e2520df - Browse repository at this point
Copy the full SHA e2520dfView commit details
Commits on Nov 13, 2024
-
Consolidate Retrieval/Reranking/Instruction Variants (#1359)
* update * merged retrieval; working * update tasks; working multilingual * everything working except instructions * working instructions; just need cleanup * add metadata for all but MindSmall * faster evaluation; mindsmall can compute in reasonable time * fix bad merge of docs * lint * fix test * qa * updated mindsmall * lint * fix debug * Update mteb/abstasks/dataloaders.py Co-authored-by: Roman Solomatin <[email protected]> * lint --------- Co-authored-by: Roman Solomatin <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9c58518 - Browse repository at this point
Copy the full SHA 9c58518View commit details
Commits on Nov 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2a8a370 - Browse repository at this point
Copy the full SHA 2a8a370View commit details -
Configuration menu - View commit details
-
Copy full SHA for dea2b77 - Browse repository at this point
Copy the full SHA dea2b77View commit details -
Configuration menu - View commit details
-
Copy full SHA for 23d6cb2 - Browse repository at this point
Copy the full SHA 23d6cb2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8868cd4 - Browse repository at this point
Copy the full SHA 8868cd4View commit details -
fix: Merge main into v2 (#1454)
* fix: Count unique texts, data leaks in calculate metrics (#1438) * add more stat * add more stat * update statistics * fix: update task metadata to allow for null (#1448) * Update tasks table * 1.19.5 Automatically generated by python-semantic-release * base * sync with main --------- Co-authored-by: Kenneth Enevoldsen <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5470c88 - Browse repository at this point
Copy the full SHA 5470c88View commit details
Commits on Nov 15, 2024
-
feat: enable codecarbon by default (#1428)
* enable codecarbon by default * lint * update flag * add allow_multiple_runs param * make lint * add warning * lint * negate the flag --------- Co-authored-by: Isaac Chung <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 70a3ff2 - Browse repository at this point
Copy the full SHA 70a3ff2View commit details
Commits on Nov 18, 2024
-
Add decriptive stat almost to all datasets (#1466)
* run tasks * remove test script * lint * remove cache * fix sickbrsts * fix tests * add datasets
Configuration menu - View commit details
-
Copy full SHA for 0e9b6fd - Browse repository at this point
Copy the full SHA 0e9b6fdView commit details
Commits on Nov 19, 2024
-
fix: Fix test for empty descriptive tasks (#1413)
* fix test * skip mock * add message to assert * fix test * lint * fix tests * upd tests * update descriptive stats files * add stat to speed
Configuration menu - View commit details
-
Copy full SHA for 0a5bedb - Browse repository at this point
Copy the full SHA 0a5bedbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6da2a1a - Browse repository at this point
Copy the full SHA 6da2a1aView commit details -
feat: Multilingual retrieval loader (#1473)
* multilingual loader * lint
Configuration menu - View commit details
-
Copy full SHA for a27de33 - Browse repository at this point
Copy the full SHA a27de33View commit details
Commits on Nov 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0df0210 - Browse repository at this point
Copy the full SHA 0df0210View commit details -
Add descriptive stats to mising tasks and add number of qrels (#1476)
* add code for comupting number of qrels * add stats fever hotpotqa msmarco topiocqa * miracl mrtidy * multilongdoc miracl reranking * add multi eurlex * fix tests for descriptive stats * fix tests --------- Co-authored-by: Roman Solomatin <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 0abe1a0 - Browse repository at this point
Copy the full SHA 0abe1a0View commit details
Commits on Nov 23, 2024
-
1475 add descriptive stats to all tasks v2 (#1482)
* add code for comupting number of qrels * BibleNLPBitextMining descriptive stats added * SwissJudgementClassification descriptive stats added * VoyageMMarcoReranking descriptive stats added * WebLINXCandidatesReranking descriptive stats added * MultiEURLEXMultilabelClassification descriptive stats added * MIRACLReranking descriptive stats added * MindSmallReranking descriptive stats added * updated test_TaskMetadata * fix test --------- Co-authored-by: Imene Kerboua <[email protected]> Co-authored-by: Imene Kerboua <[email protected]> Co-authored-by: Roman Solomatin <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for a7a5214 - Browse repository at this point
Copy the full SHA a7a5214View commit details -
fix: Fix
BrightRetrieval
calculate stats (#1484)* fix bright loader * lint * fix comment
Configuration menu - View commit details
-
Copy full SHA for 99247b2 - Browse repository at this point
Copy the full SHA 99247b2View commit details