Results
- Showing results for:
- Reset all filters
Search results
-
Conference paperFreedman G, Toni F, 2025,
Exploring the potential for large language models to demonstrate rational probabilistic beliefs
, 38th International FLAIRS Conference, Publisher: LibraryPress@UF, ISSN: 2334-0762Advances in the general capabilities of large language models (LLMs) have led to their use for information retrieval, and as components in automated decision systems. A faithful representation of probabilistic reasoning in these models may be essential to ensure trustworthy, explainable and effective performance in these tasks. Despite previous work suggesting that LLMs can perform complex reasoning and well-calibrated uncertainty quantification, we find that current versions of this class of model lack the ability to provide rational and coherent representations of probabilistic beliefs. To demonstrate this, we introduce a novel dataset of claims with indeterminate truth values and apply a number of well-established techniques for uncertainty quantification to measure the ability of LLM's to adhere to fundamental properties of probabilistic reasoning.
-
Conference paperAyoobi H, Potyka N, Toni F, 2025,
ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation
, AAAI Conference on Artificial Intelligence -
Conference paperChen L, Dejl A, Toni F, 2025,
Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts
, The 39th Annual AAAI Conference on Artificial Intelligence -
Conference paperFreedman G, Dejl A, Gorur D, et al., 2025,
Argumentative large language models for explainable and contestable claim verification
, AAAI Conference on Artificial Intelligence, Publisher: Association for the Advancement of Artificial Intelligence, Pages: 14930-14939, ISSN: 2159-5399The profusion of knowledge encoded in large language models (LLMs) and their ability to apply this knowledge zero-shot in a range of settings makes them promising candidates for use in decision-making. However, they are currently limited by their inability to provide outputs which can be faithfully explained and effectively contested to correct mistakes. In this paper, we attempt to reconcile these strengths and weaknesses by introducing argumentative LLMs (ArgLLMs), a method for augmenting LLMs with argumentative reasoning. Concretely, ArgLLMs construct argumentation frameworks, which then serve as the basis for formal reasoning in support of decision-making. The interpretable nature of these argumentation frameworks and formal reasoning means that any decision made by ArgLLMs may be explained and contested. We evaluate ArgLLMs’ performance experimentally in comparison with state-of-the-art techniques, in the context of the decision-making task of claim verification. We also define novel properties to characterise contestability and assess ArgLLMs formally in terms of these properties.
-
Conference paperRusso F, Toni F, 2025,
Shapley-PC: constraint-based causal structure learning with a Shapley inspired framework
, 4th Conference on Causal Learning and Reasoning (CLeaR 2025)Causal Structure Learning (CSL), also referred to as causal discovery, amounts to extracting causal relations among variables in data. CSL enables the estimation of causal effects from observational data alone, avoiding the need to perform real life experiments. Constraint-based CSL leverages conditional independence tests to perform causal discovery. We propose Shapley-PC, a novel method to improve constraint-based CSL algorithms by using Shapley values over the possible conditioning sets, to decide which variables are responsible for the observed conditional (in)dependences. We prove soundness, completeness and asymptotic consistency of Shapley-PC and run a simulationstudy showing that our proposed algorithm is superior to existing versions of PC.
-
Conference paperRapberger A, Ulbricht M, Toni F, 2024,
On the correspondence of non-flat assumption-based argumentation and logic programming with negation as failure in the head
, 22nd International Workshop on Nonmonotonic Reasoning NMR 24), Publisher: CEUR Workshop Proceedings, Pages: 122-121, ISSN: 1613-0073The relation between (a fragment of) assumption-based argumentation (ABA) and logic programs (LPs) under stable model semantics is well-studied. However, for obtaining this relation, the ABA framework needs to be restricted to being flat, i.e., a fragment where the (defeasible) assumptions can never be entailed, only assumed to be true or false. Here, we remove this restriction and show a correspondence between non-flat ABA and LPs with negation as failure in their head. We then extend this result to so-called set-stable ABA semantics, originally defined for the fragment of non-flat ABA called bipolar ABA. We showcase how to define set-stable semantics for LPs with negation as failure in their head and show the correspondence to set-stable ABA semantics.
-
Conference paperVasileiou S, Kumar A, Yeoh W, et al., 2024,
Dialectical reconciliation via structured argumentative dialogues
, KR 2024We present a novel framework designed to extend model reconciliation approaches, commonly used in human-aware planning, for enhanced human-AI interaction. By adopting a structured argumentation-based dialogue paradigm, our framework enables dialectical reconciliation to address knowledge discrepancies between an explainer (AI agent) and an explainee (human user), where the goal is for the explainee to understand the explainer's decision. We formally describe the operational semantics of our proposed framework, providing theoretical guarantees. We then evaluate the framework's efficacy ``in the wild'' via computational and human-subject experiments. Our findings suggest that our framework offers a promising direction for fostering effective human-AI interactions in domains where explainability is important.
-
Conference paperBattaglia E, Baroni P, Rago A, et al., 2024,
Integrating user preferences into gradual bipolar argumentation for personalised decision support
, Scalable Uncertainty Management, 16th International Conference (SUM 2024), Publisher: Springer, Pages: 14-28, ISSN: 1611-3349Gradual bipolar argumentation has been shown to be aneffective means for supporting decisions across a number of domains. Individual user preferences can be integrated into the domain knowledge represented by such argumentation frameworks and should be taken into account in order to provide personalised decision support. This howeverrequires the definition of a suitable method to handle user-provided preferences in gradual bipolar argumentation, which has not been considered in previous literature. Towards filling this gap, we develop a conceptual analysis on the role of preferences in argumentation and investigate some basic principles concerning the effects they should have on the evaluation of strength in gradual argumentation semantics. We illustrate an application of our approach in the context of a review aggregation system, which has been enhanced with the ability to produce personalisedoutcomes based on user preferences.
-
Conference paperRago A, Vasileiou SL, Toni F, et al., 2024,
A Methodology for Gradual Semantics for Structured Argumentation under Incomplete Information
, ArXiv -
Journal articleKampik T, Potyka N, Yin X, et al., 2024,
Contribution functions for quantitative bipolar argumentation graphs: a principle-based analysis
, International Journal of Approximate Reasoning, Vol: 173, ISSN: 0888-613XWe present a principle-based analysis of contribution functions for quantitative bipolar argumentation graphs that quantify the contribution of one argument to another. The introduced principles formalise the intuitions underlying different contribution functions as well as expectations one would have regarding the behaviour of contribution functions in general. As none of the covered contribution functions satisfies all principles, our analysis can serve as a tool that enables the selection of the most suitable function based on the requirements of a given use case.
-
Conference paperLehtonen T, Rapberger A, Toni F, et al., 2024,
On computing admissibility in ABA
, 10th International Conference on Computational Models of Argument (COMMA 2024), Publisher: IOS Press, Inc., Pages: 121-132Most existing computational tools for assumption-based argumentation (ABA) focus on so-called flat frameworks, disregarding the more general case. Here, we study an instantiation-based approach for reasoning in possibly non-flat ABA. For complete-based semantics, an approach of this kind was recently introduced, based on a semantics-preserving translation between ABA and bipolar argumentation frameworks (BAFs). Admissible semantics, however, require us to consider an extension of BAFs which also makes use of premises of arguments (pBAFs).We explore basic properties of pBAFs which we require as a theoretical underpinning for our proposed instantiation-based solver for non-flat ABA under admissible semantics. As our empirical evaluation shows, depending on the ABA instances, the instantiation-based solver is competitive against an ASP-based approach implemented in the style of state-of-the-art solvers for hard argumentation problems.
-
Conference paperRapberger A, Toni F, 2024,
On the robustness of argumentative explanations
, 10th International Conference on Computational Models of Argument (COMMA 2024), Publisher: IOS Press, Inc., Pages: 217-228The field of explainable AI has grown exponentially in recent years.Within this landscape, argumentation frameworks have shown to be helpful ab-stractions of some AI models towards providing explanations thereof. While exist-ing work on argumentative explanations and their properties has focused on staticsettings, we focus on dynamic settings whereby the (AI models underpinning the)argumentation frameworks need to change. Specifically, for a number of notionsof explanations drawn from abstract argumentation frameworks under extension-based semantics, we address the following questions: (1) Are explanations robust toextension-preserving changes, in the sense that they are still valid when the changesdo not modify the extensions? (2) If not, are these explanations pseudo-robust inthat can be tractably updated? In this paper, we frame these questions formally. Weconsider robustness and pseudo-robustness w.r.t. ordinary and strong equivalenceand provide several results for various extension-based semantics.
-
Conference paperAyoobi H, Potyka N, Toni F, 2024,
Argumentative interpretable image classification
, 2nd International Workshop on Argumentation for eXplainable AI co-located with the 10th International Conference on Computational Models of Argument (COMMA 2024), Publisher: CEUR Workshop Proceedings, Pages: 3-15, ISSN: 1613-0073We propose ProtoSpArX, a novel interpretable deep neural architecture for image classification in the spirit of prototypical-part-learning as found, e.g. in ProtoPNet. While earlier approaches associate every class with multiple prototypical-parts, ProtoSpArX uses super-prototypes that combine prototypical-parts into single class representations. Furthermore, while earlier approaches use interpretable classification layers, e.g. logistic regression in ProtoPNet, ProtoSpArX improves accuracy with multi-layer perceptronswhile relying upon an interpretable reading thereof based on a form of argumentation. ProtoSpArX is customisable to user cognitive requirements by a process of sparsification of the multi-layer perceptron/argumentation component. Also, as opposed to other prototypical-part-learning approaches,ProtoSpArX can recognise spatial relations between different prototypical-parts that are from various regions in images, similar to how CNNs capture relations between patterns recognized in earlier layers.
-
Conference paperSukpanichnant P, Rapberger A, Toni F, 2024,
PeerArg: argumentative peer review with LLMs
, First International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2024)Peer review is an essential process to determine the quality of papers submitted to scientific conferences or journals. However, it is subjective and prone to biases. Several studies have been conducted to apply techniques from NLP to support peer review, but they are based on black-box techniques and their outputs are difficult to interpret and trust. In this paper, we propose a novel pipeline to support and understand the reviewing and decision-making processes of peer review: the PeerArg system combining LLMs with methods from knowledge representation. PeerArg takes in input a set of reviews for a paper and outputs the paper acceptance prediction. We evaluate the performance of the PeerArg pipeline on three different datasets, in comparison with a novel end-2-end LLM that uses few-shot learning to predict paper acceptance given reviews. The results indicate that the end-2-end LLM is capable of predicting paper acceptance from reviews, but a variantof the PeerArg pipeline outperforms this LLM.
-
Conference paperOluokun B, Paulino Passos G, Rago A, et al., 2024,
Predicting Human Judgement in Online Debates with Argumentation
, The 24th International Workshop on Computational Models of Natural Argument (CMNA’24) -
Conference paperYin X, Potyka N, Toni F, 2024,
Applying attribution explanations in truth-discovery quantitative bipolar argumentation frameworks
, 2nd International Workshop on Argumentation for eXplainable AI (ArgXAI) co-located with 10th International Conference on Computational Models of Argument (COMMA 2024), Publisher: CEUR Workshop Proceedings, ISSN: 1613-0073 -
Conference paperJiang J, Leofante F, Rago A, et al., 2024,
Robust counterfactual explanations in machine learning: a survey
, The 33rd International Joint Conference on Artificial Intelligence, IJCAI 2024, Publisher: International Joint Conferences on Artificial Intelligence Organization (IJCAI), Pages: 8086-8094Counterfactual explanations (CEs) are advocated as being ideally suited to providing algorithmic recourse for subjects affected by the predictions of machine learning models. While CEs can be beneficial to affected individuals, recent work has exposed severe issues related to the robustness of state-of-the-art methods for obtaining CEs. Since a lack of robustness may compromise the validity of CEs, techniques to mitigate this risk are in order. In this survey, we review works in the rapidly growing area of robust CEs and perform an in-depth analysis of the forms of robustness they consider. We also discuss existing solutions and their limitations, providing a solid foundation for future developments.
-
Conference paperYin X, Potyka N, Toni F, 2024,
Explaining arguments’ strength: unveiling the role of attacks and supports
, IJCAI 2024, the 33rd International Joint Conference on Artificial Intelligence, Publisher: International Joint Conferences on Artificial Intelligence, Pages: 3622-3630Quantitatively explaining the strength of arguments under gradual semantics has recently received increasing attention. Specifically, several works in the literature provide quantitative explanations by computing the attribution scores of arguments. These works disregard the importance of attacks and supports, even though they play an essential role when explaining arguments' strength. In this paper, we propose a novel theory of Relation Attribution Explanations (RAEs), adapting Shapley values from game theory to offer fine-grained insights into the role of attacks and supports in quantitative bipolar argumentation towards obtaining the arguments' strength. We show that RAEs satisfy several desirable properties. We also propose a probabilistic algorithm to approximate RAEs efficiently. Finally, we show the application value of RAEs in fraud detection and large language models case studies.
-
Conference paperGould A, Paulino Passos G, Dadhania S, et al., 2024,
Preference-based abstract argumentation for case-based reasoning
, International Conference on Principles of Knowledge Representation and Reasoning, Publisher: IJCAI Organization, Pages: 394-404, ISSN: 2334-1033In the pursuit of enhancing the efficacy and flexibility of interpretable, data-driven classification models, this work introduces a novel incorporation of user-defined preferences with Abstract Argumentation and Case-Based Reasoning (CBR). Specifically, we introduce Preference-Based Abstract Argumentation for Case-Based Reasoning (which we call AA-CBR-P), allowing users to define multiple approaches to compare cases with an ordering that specifies their preference over these comparison approaches. We prove that the model inherently follows these preferences when making predictions and show that previous abstract argumentation for case-based reasoning approaches are insufficient at expressing preferences over constituents of an argument. We then demonstrate how this can be applied to a real-world medical dataset sourced from a clinical trial evaluating differing assessment methods of patients with a primary brain tumour. We show empirically that our approach outperforms other interpretable machine learning models on this dataset.
-
Conference paperFreedman G, Toni F, 2024,
Detecting scientific fraud using argument mining
, ArgMining@ACL2024, Publisher: Association for Computational Linguistics, Pages: 15-28proliferation of fraudulent scientific research in recent years has precipitated a greater interest in more effective methods of detection. There are many varieties of academic fraud, but a particularly challenging type to detect is the use of paper mills and the faking of peer-review. To the best of our knowledge, there have so far been no attempts to automate this process.The complexity of this issue precludes the use of heuristic methods, like pattern-matching techniques, which are employed for other types of fraud. Our proposed method in this paper uses techniques from the Computational Argumentation literature (i.e. argument mining and argument quality evaluation). Our central hypothesis stems from the assumption that articles that have not been subject to the proper level of scrutiny will contain poorly formed and reasoned arguments, relative to legitimately published papers. We use a variety of corpora to test this approach, including a collection of abstracts taken from retracted papers. We show significant improvement compared to a number of baselines, suggesting that this approach merits further investigation.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.