Abstract
From the early developments of machines for reasoning and decision making in higher-level information fusion, there was a need for a systematic and reliable evaluation of their performance. Performance evaluation is important for comparison and assessment of alternative solutions to real-world problems. In this paper we focus on one aspect of performance assessment for reasoning under uncertainty: the accuracy of the resulting belief (prediction or estimate). We propose a framework for assessment based on the assumption that the system under investigation is uncertain only due to stochastic variability (randomness), which is partially known. In this context we formulate a distance measure between the “ground truth” and the output of an automated system for reasoning in the framework of one of the non-additive uncertainty formalisms (such as imprecise probability theory, belief function theory or possibility theory). The proposed assessment framework is demonstrated with a simple numerical example.
Original language | English |
---|---|
Pages (from-to) | 11-16 |
Number of pages | 6 |
Journal | Information Fusion |
Volume | 71 |
Early online date | 15 Jan 2021 |
DOIs | |
Publication status | Published - Jul 2021 |
Bibliographical note
Funding Information:This research was supported by DST Group, Australia under the Research Agreement “Classification decisions using uncertain knowledge-base” (2019–2020).
Publisher Copyright:
© 2021 Elsevier B.V.
Keywords
- Automated reasoning under uncertainty
- Belief function theory
- Imprecise probability theory
- Performance evaluation
- Possibility theory
ASJC Scopus subject areas
- Software
- Signal Processing
- Information Systems
- Hardware and Architecture