Estimating and evaluating the uncertainty of rating predictions and top-n recommendations in recommender systems

Thumbnail Image
3584021.pdf(948.29 KB)
Accepted Version
Coscrato, Victor
Bridge, Derek
Journal Title
Journal ISSN
Volume Title
Association for Computing Machinery
Published Version
Research Projects
Organizational Units
Journal Issue
Uncertainty is a characteristic of every data-driven application, including recommender systems. The quantification of uncertainty can be key to increasing user trust in recommendations or choosing which recommendations should be accompanied by an explanation; and uncertainty estimates can be used to accomplish recommender tasks such as active learning and co-training. Many uncertainty estimators are available but, to date, the literature has lacked a comprehensive survey and a detailed comparison. In this paper, we fulfil these needs. We review the existing methods for uncertainty estimation and metrics for evaluating uncertainty estimates, while also proposing some estimation methods and evaluation metrics of our own. Using two datasets, we compare the methods using the evaluation metrics that we describe, and we discuss their strengths and potential issues. The goal of this work is to provide a foundation to the field of uncertainty estimation in recommender systems, on which further research can be built.
Information systems , Recommender systems , Uncertainty , Computing methodologies , Uncertainty quantification , General and reference , Empirical studies , Evaluation , Estimation , Mathematics of computing , Probability and statistics , Uncertainty , Recommender systems
Coscrato, V. and Bridge, D. (2023) 'Estimating and evaluating the uncertainty of rating predictions and top-n recommendations in recommender systems', ACM Transactions on Recommender Systems. doi: 10.1145/3584021
© 2023, the Authors. Publication rights licensed to ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Recommender Systems,