?
Разрешение споров при помощи абстрактных аргументационных структур и его автоматизация на примере научного рецензирования
The process of reviewing scientific publications is hampered by the constant increase in the volume of submissions and the bias of reviewers. Artificial intelligence can speed up the scientific review process, but it is also capable of reproducing the biases embedded in the training data. We propose a formal method for representing and analyzing arguments, which contributes to reducing the duration and different kinds of bias in scientific reviewing. We consider scientific peer review as a single mixed argumentative dispute between authors of a manuscript submitted for publication and reviewers evaluating it according to certain criteria, and apply a formalism based on abstract argumentation frameworks with credulous extension semantics, which allows to find resolutions of the disputes. Our method for dispute resolution consists of 1) marking up a review, 2) converting the markup to JSON, 3) generating its OWL representation, and 3) identifying the dispute solution by finding its preferred extension, i.e. the maximum subset of acceptable arguments put forward by authors and reviewers. We developed a markup scheme for scientific reviews and an algorithm for generating their OWL representations, which provide a formalization of argumentation disputes during scientific peer review with the help of abstract argumentation frameworks and allow us to find solutions to the disputes by means of the automated logical inference. We validated the proposed method by annotating a corpus of peer reviews and applying the proof-of-concept algorithm to it. We evaluated the proposed annotation scheme with respect to the inter annotator agreement, which showed that the proposed scheme provides unambiguous markup of auth or arguments, whereas the markup of reviewers’ arguments depends on the interpretation of the annotators.