We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opencompass/opencompass/datasets/agieval/agieval.py 文件显示,目前AGIEval评测仅支持zero-shot设置:
def load(path: str, name: str, setting_name: str): from .dataset_loader import load_dataset, load_dataset_as_result_schema assert setting_name in 'zero-shot', 'only support zero-shot setting' dataset_wo_label = load_dataset(name, setting_name, path) dataset_with_label = load_dataset_as_result_schema(name, path) dataset = [] for d1, d2 in zip(dataset_wo_label, dataset_with_label): dataset.append({ 'id': d2.index, 'problem_input': d1['context'], 'label': d2.label, }) dataset = Dataset.from_list(dataset) return dataset
原生AGIEval支持few-shot设置,是否可以追加few-shot评测设置?
The text was updated successfully, but these errors were encountered:
bittersweet1999
No branches or pull requests
描述该功能
opencompass/opencompass/datasets/agieval/agieval.py 文件显示,目前AGIEval评测仅支持zero-shot设置:
原生AGIEval支持few-shot设置,是否可以追加few-shot评测设置?
是否希望自己实现该功能?
The text was updated successfully, but these errors were encountered: