Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
102 Open 1,275 Closed
102 Open 1,275 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fix gguf loading via Transformers
#2596 opened Dec 25, 2024 by CL-ModelCloud Loading…
2
change to single process for bootstrap_stderr
#2593 opened Dec 23, 2024 by zhuyuhua-v Loading…
1
Fix the format of mgsm zh and ja.
#2587 opened Dec 20, 2024 by timturing Loading…
2
Update dependencies in pyproject.toml: remove duplicated 'evaluate', add 'anthropic' and 'openai' to optional dependencies
#2582 opened Dec 19, 2024 by Technolog796 Loading…
5
fix multiple input chat tempalte
#2576 opened Dec 17, 2024 by baberabb Loading…
1
Added caseHOLD task
#2570 opened Dec 16, 2024 by zolastro Loading…
2
add llama3 tasks
#2556 opened Dec 10, 2024 by baberabb Loading…
add option to add an assistant_prefix
#2545 opened Dec 5, 2024 by baberabb • Draft
[MM] Chartqa
#2544 opened Dec 5, 2024 by baberabb • Draft
[MM] Ai2d
#2542 opened Dec 5, 2024 by baberabb • Draft
New arabicmmlu
#2541 opened Dec 5, 2024 by bodasadallah Loading…
1
Update KorMedMCQA: ver 2.0
#2540 opened Dec 5, 2024 by GyoukChu Loading…
8
add Darija (Moroccan dialects) tasks including darijammlu. darijahellaswag and darija_bench
#2521 opened Nov 28, 2024 by hadi-abdine Loading…
2
Add --examples Argument for Fine-Grained Task Evaluation in lm-evaluation-harness. This feature is the first step towards efficient multi-prompt evaluation with PromptEval [1,2]
#2520 opened Nov 26, 2024 by felipemaiapolo Loading…
1
max_length not used
#2515 opened Nov 25, 2024 by lintangsutawika Loading…
3
Added small fix to split by eos_token_id before decoding
#2512 opened Nov 24, 2024 by EtashGuha Loading…
1
fixed mmlu generative response extraction
#2503 opened Nov 18, 2024 by RawthiL Loading…
3
Added regex filter for bbh fewshot
#2502 opened Nov 18, 2024 by RawthiL Loading…
1
Add GigaChat API
#2495 opened Nov 15, 2024 by seldereyy Loading…
Yaml crowspairs tasks
#2488 opened Nov 14, 2024 by NAM00 Loading…
1
Biology ds
#2486 opened Nov 13, 2024 by deema-A Loading…
2
MILU dataset from AI4Bharat for Indic LLM eval
#2482 opened Nov 12, 2024 by abhinand5 Loading…
1
Update citation
#2474 opened Nov 8, 2024 by Sypherd Loading…
1
Use global filter alias
#2473 opened Nov 8, 2024 by Sypherd Loading…
allow fewshots for multimodal tasks
#2450 opened Nov 1, 2024 by artemorloff Loading…
2
ProTip! Adding no:label will show everything without a label.