train, val-test split: How to select the right split on demand

[sorobedio]

Hello, i would like to be able to select the split i wish to evaluate my model on. but i am not seen such an option .
how do i do that for dataset such as mmnlu, mmlu-pro, hellaswag with llaba3.2-1B-instruct

[baberabb]

Hi! We check test_split> validation_split > training_split to determine which evaluation split to use. You can change them in the task configs:

lm-evaluation-harness/lm_eval/tasks/gsm8k/gsm8k.yaml

Lines 7 to 9 in 8558b8d

	training_split: train
	fewshot_split: train
	test_split: test