Skip to content

Issues: THUDM/LongBench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

34 Open 58 Closed
34 Open 58 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Error code: 400 - {'object': 'error', 'message': "This model's maximum context length is 131072 tokens. However, you requested 351430 tokens. Please reduce the length of the messages or completion.", 'type': 'BadRequestError', 'param': None, 'code': 400
#104 opened Jan 18, 2025 by guanzy2012
Token indices sequence length is longer than the specified maximum sequence length for this model (1113927 > 128000). Running this sequence through the model will result in indexing errors
#103 opened Jan 18, 2025 by guanzy2012
Error code: 404 - {'object': 'error', 'message': 'The model /dev/shm/glm-4-9b-chat/ does not exist.', 'type': 'NotFoundError', 'param': None, 'code': 404}
#102 opened Jan 18, 2025 by guanzy2012
Any Implementation of new models like Meta-Llama-3.1-8B , Qwen2.5-7B?
#101 opened Jan 17, 2025 by WeiweiZhang1
Why are empty responses ignored in LongBench v2?
#100 opened Jan 15, 2025 by ZeonfaiHo
[LongBench] How was Edit Sim for code tasks calculated?
#96 opened Jan 9, 2025 by cornzz
1
如何使vllm下测评结果更接近hf？
#95 opened Jan 7, 2025 by miaoyuxun
2
关于Llama-3.1-8B-Instruct在Longbench v2 测试结果和排行榜有出入的问题
#94 opened Jan 2, 2025 by chaochen99
3
针对 Paper中 Long数据集的评测配置
#93 opened Jan 2, 2025 by MaiziXiao
3
关于Claude和gemini的token处理
#92 opened Jan 1, 2025 by Violettttee
1
关于cot
#90 opened Dec 26, 2024 by Violettttee
7
DeepSeek V3 Result enhancement New feature or request
#89 opened Dec 26, 2024 by zmwang03
@bys0318
1
results.py bug
#88 opened Dec 24, 2024 by cizhenshi
1
安装完longbench后，运行报错
#86 opened Dec 14, 2024 by fxnie
Request for Mixtral-8*7B model
#84 opened Nov 18, 2024 by Aaronhuang-778
请问用GPT-3.5-turbo-16K测试一遍完整的数据集需要花费多少？
#83 opened Nov 4, 2024 by Ethereal-sakura
1
Evaluation mechanism update
#82 opened Oct 30, 2024 by cizhenshi
1
可以测试基于OpenAI接口的模型管理框架吗，比如ollama, xinference
#74 opened Aug 29, 2024 by jiusi9
1
No initialization for the process group
#73 opened Aug 28, 2024 by Mugariya
1
How to evaluate on llama3-8b-instruct? enhancement New feature or request
#71 opened Aug 11, 2024 by txchen-USTC
@bys0318
1
关于提升数据集测试有效性的建议
#70 opened Jul 25, 2024 by wsn555
7
Code for evaluation with GPT-3.5?
#69 opened Jul 22, 2024 by RuskinManku
3
Load dataset from hf failed
#68 opened Jul 16, 2024 by murphypei
4
The "anwser" for some examples in "qasper.jsonl" is strange
#67 opened Jul 9, 2024 by Zcchill
6
Loading local datasets with split=‘test’
#65 opened May 30, 2024 by yichen0104
1
ProTip! Find all open issues with in progress development work with linked:pr.