BibTex format
@inproceedings{Sukpanichnant:2024,
author = {Sukpanichnant, P and Rapberger, A and Toni, F},
title = {PeerArg: argumentative peer review with LLMs},
url = {http://hdl.handle.net/10044/1/114327},
year = {2024}
}
Citation
RIS format (EndNote, RefMan)
TY - CPAPER
AB - Peer review is an essential process to determine the quality of papers submitted to scientific conferences or journals. However, it is subjective and prone to biases. Several studies have been conducted to apply techniques from NLP to support peer review, but they are based on black-box techniques and their outputs are difficult to interpret and trust. In this paper, we propose a novel pipeline to support and understand the reviewing and decision-making processes of peer review: the PeerArg system combining LLMs with methods from knowledge representation. PeerArg takes in input a set of reviews for a paper and outputs the paper acceptance prediction. We evaluate the performance of the PeerArg pipeline on three different datasets, in comparison with a novel end-2-end LLM that uses few-shot learning to predict paper acceptance given reviews. The results indicate that the end-2-end LLM is capable of predicting paper acceptance from reviews, but a variantof the PeerArg pipeline outperforms this LLM.
AU - Sukpanichnant,P
AU - Rapberger,A
AU - Toni,F
PY - 2024///
TI - PeerArg: argumentative peer review with LLMs
UR - http://hdl.handle.net/10044/1/114327
ER -
Contact us
Artificial Intelligence Network
South Kensington Campus
Imperial College London
SW7 2AZ
To reach the elected speaker of the network, Dr Rossella Arcucci, please contact:
To reach the network manager, Diana O'Malley - including to join the network - please contact: