\chapter[Prediction Uncertainty in \acrshort{irss}]{Prediction Uncertainty in Implicit Feedback-Based Recommender Systems}
\label{chapter:implicit}

\acrfull{irss} have reshaped the field by inferring user preferences from their item interaction behaviour. In contrast to explicit feedback, which involves direct user input such as ratings, implicit feedback focuses on actions such as clicks, downloads and purchases. This approach is valuable in scenarios where explicit feedback is limited. \acrshort{irss} use these interactions to infer user preferences, offering seamless and unobtrusive insights. This has led to advancements in machine learning techniques and collaborative filtering algorithms, enhancing personalized recommendations across many domains. In an evolving digital landscape, \acrshort{irss} continue to play a crucial role in delivering tailored content, boosting user engagement, and enriching the overall user experience.

Because of their current prominence, it is natural for the research on prediction uncertainty\footnote{As per the previous chapter, here we refer to prediction uncertainty simply as uncertainty throughout.} in \acrshort{rs} to be carried over to the implicit feedback case. The available literature in this case is even more limited. For this reason, in this chapter, we:

\begin{itemize}
    \item Generalize uncertainty estimation methods from Chapter \ref{chapter:explicit}, identifying which are directly applicable to \acrshort{irss}.
    \item Generalize uncertainty evaluation metrics from Chapter \ref{chapter:explicit}, distinguishing those that remain relevant in the case of \acrshort{irss}.
    \item Describe the existing methods for uncertainty estimation for \acrshort{irss}, and propose novel uncertainty estimation methods. In particular, due to their increasing usage in the field, we propose methods for uncertainty estimation in neural network-based \acrshort{irss}.
    \item Present an extensive comparison between all the methods surveyed in the chapter and all the novel methods proposed in the chapter.
\end{itemize}

%\derek{If you revise Section 3.1 in the way I suggest, then this Section, 4.1, needs to parallel it a bit more. It needs: a similar title to 3.1 (but not identical); a statement that rating prediction no longer applies; and a reminder of ReBR (instead of RaBR) and of UBF and PRR.}
%\victor{The following section was also remade.}

\section{Prediction and Recommendation in \acrshort{irss}}
\label{sec:setup4}

In this chapter, we will explore RS based on implicit feedback data, specifically those based on unary interaction data (e.g.\ clicking, purchasing, watching, listening) and with no explicit feedback given. Now, in the absence of ratings, we assume that our data consists of a set $R$ of user-item interaction pairs. That is, if $\langle u, i \rangle \in R$, then user $u \in U$ has interacted with item $i \in I$.

\subsection{Relevance prediction}

Essentially, \acrshort{irss} are designed to retrieve, from a large catalog, those items that are most relevant to each user. To achieve its goal, the system must learn a scoring function $F: U \times I \rightarrow \mathbb{R}$ to predict the user-item interaction relevance. Note how this differs from \acrfull{erss}: an \acrshort{ers} make rating predictions; for an \acrshort{irs} rating prediction no longer applies; scoring is based on predicted relevance. In this chapter, for relevance predictions, we confine our attention to model-based \acrfull{cf} algorithms, in particular, traditional \acrfull{mf} and \acrfull{nn} models.

In addition to relevance, we want to estimate the uncertainty associated with each relevance prediction. Again, an uncertainty estimator is a function $G: U \times I \rightarrow \mathbb{R}$, that predicts real-valued quantities associated with each relevance prediction. 

\subsection{Top-$n$ recommendation}
The task of top-$n$ recommendation in \acrshort{irss} is analogous to the explicit case, but using predicted relevance instead of predicted ratings. Hence, the conventional way to retrieve a set of recommendations $Z^n_u$ for user $u \in U$ with candidate items $C_u \subseteq I$ in an \acrshort{irs} is to select the top-$n$ candidates with highest predicted relevance. We denote this form of ranking \acrfull{rebr}. 

But, similarly to the previous chapter, there are uncertainty-aware alternative to \acrshort{rebr}. The uncertainty estimates may again assist in ranking and selection of a top-$n$.
In particular, \acrfull{ubf}, introduced in Section \ref{sec:ubf}, is applicable to \acrshort{irss} with no further adaptation. On the other hand, \acrfull{prr} is not applicable because scoring in \acrshort{irss} is by default already based on relevance.

\section{Estimating Uncertainty}
\label{sec:estimating2}

Recommendation uncertainty has several causes, including sparsity of data, modeling choices, and stochastic learning algorithms. For this reason, methods for uncertainty estimation in the field are very diverse. Therefore, similar to Section \ref{sec:estimating}, it helps to organize the uncertainty estimation methods according to their uncertainty source.

%\derek{I think it would be useful somewhere (right here?) to say that there were 5 classes in Chapter 3 but to explain that two of them no longer apply (and to explain why they no longer apply) and then to give the remaining three.} \victor{Added after the listing.}

\begin{description}
    \item [Information-based:] The amount of available data continues to be a fundamental source of uncertainty in implicit feedback data.
    \item [Stability-based:] As previously, the stability of the learning algorithms can be used to estimate uncertainty. In particular, stability becomes a serious consideration when using neural-based recommenders, due to the stochastic mechanisms involved in their learning \cite{colbrook2022difficulty, summers2021nondeterminism}.
    \item [Distribution-based:] Interaction relevance scores may be assumed to follow a statistical distribution, whose dispersion is an uncertainty estimate. In this case, uncertainty estimation is tackled as a \acrlong{ml} problem, in which the uncertainty estimator is \emph{learned} just like the relevance estimator. Due to particularities of learning uncertainty estimators in the case of \acrshort{irs}, we exclude them from this chapter. We will these particularities in depth and this class of models in Chapter \ref{chapter:implicit_learn}.
\end{description}
We note that, in Chapter \ref{chapter:explicit}, we classified the uncertainty estimators into five groups, while above we have just three of them. This is because the Error-based and Multinomial-based uncertainty estimators are not defined for \acrshort{irss}: since there are no ratings, there can be no  rating prediction errors (required for Error-based estimators) and discrete rating scales (required for Multinomial-based estimators).

We now present information-based and stability-based uncertainty estimation methods for \acrshort{irs}. In each case, we make clear which recommender algorithms can be used with each uncertainty estimation method.

\subsection{Information-based uncertainty}
\label{sec:info_based2}

One of the most notable sources of uncertainty is sparsity. For this reason, the amount of available data offers good baseline estimates of recommendation uncertainty \cite{mazurowski_estimating_2013}. Furthermore, these estimates can be used with any recommendation algorithm. In the past, they have been used for explicit feedback recommenders, but their generalization to \acrshort{irss} is straightforward. These estimates can be user-centric or item-centric. Hence, following \cite{mazurowski_estimating_2013}, we define the following uncertainty metrics,

\begin{align}
    \usup : G_{ui} = - \#R_{u \cdot} \\
    \isup : G_{ui} = - \#R_{\cdot i} \label{eq:isup}
\end{align}
\noindent
where $\#u$ and $\#i$ denote the number of observed interactions for the user and the item, respectively. The clear drawback of these uncertainty estimates is that they are either at user-level or at item-level, that is $G_{ui}$ is defined solely based on the user, or the item, but not on the user-item interaction. Nevertheless, they have the advantage of needing no additional learning and can be easily plugged into any system.

As we noted previously (Section \ref{sec:info_based}), since top-$n$ recommendation is a user-based task, user-wise uncertainty estimates, such as $\usup$, are largely useless. The same reasoning also applies to the case of \acrshort{irss}, and therefore, we will not explore $\usup$ any further in this chapter. Naturally, $\uvar$ and $\ivar$, the other forms of information-based uncertainty from Chapter \ref{chapter:explicit} are not applicable in the absence of explicit ratings.

\subsection{Stability-based uncertainty}
\label{sec:stability_based2}

Beyond the uncertainty introduced by the data, every recommender algorithm has its own uncertainty issues. Consider, for example, models that are based on representation learning, such as MF, where vector embeddings are learned as a latent representation for each user and item. For such models, the uncertainty surrounding the learning of such representations will affect the system recommendations. In fact, MF is known to suffer from learning instability \cite{gabbolini2022instability,pena2020combining}. 

In the case of explicit feedback, ensembles have been successful at estimating the uncertainty of MF rating predictions \cite{mazurowski_estimating_2013}. But, explicit feedback MF is only one of many algorithms that can benefit from ensembling. In fact, an ensemble can be used to estimate uncertainty for any model that relies on a stochastic mechanism, such as random parameter initialization or stochastic learning protocols. This is the case for implicit feedback MF and also any neural network model, and in particular the MLP model (Eq.\ \ref{eq:MLP}).\footnote{An ensemble of neural models is often referred to as a Deep-Ensemble \cite{lakshminarayanan2017simple}.}

Formally, the principle is to train several models $F^{(k)}$, for $k = 1, \ldots, n$ using a different random initialization each time, and then calculate interaction relevance and uncertainty as follows:

\begin{align}
    F(u, i) &= \frac{\sum_{k=1}^n {F^{(k)}_{\theta}(u, i)}}{n} \label{eq:ensemble} \\
    G_{ui} &= \frac{\sum_{k=1}^n \big( {F^{(k)}_{\theta}(u, i) - r_{ui} \big) ^2}}{n} \label{eq:ensemble_unc}
\end{align}

But there are alternatives to ensembling, which we can find in the \acrshort{ml} literature and apply to the case of \acrshort{irss}. We will look at two: Bayesian Neural Networks and MCDropout.

Bayesian Neural Networks (BNN) are a major tool tailored to uncertainty quantification in neural models. BNNs differ from their deterministic counterpart by treating the parameters as random variables \cite{goan2020bayesian}, which are assumed to follow some prior distribution $p(\theta)$. Given some training data $\mathcal{D}$, the posterior weight distribution, according to Bayes rule, is as follows,

\begin{align}
    p(\theta | \mathcal{D}) = \frac{p(\theta) p(D | \theta)}{p(D)}
\label{eq:bayes}
\end{align}

Calculating the posterior directly from Eq.\ \ref{eq:bayes} is generally not possible, because the data evidence, $p(D)$, is unknown. For this reason, inference methods such as \acrfull{mcmc} \cite{geyer1992practical} and \acrfull{vi} \cite{graves2011practical} are applied to approximate the exact posterior. More recently, \acrfull{bbb} has been proposed \cite{blundell2015weight}, a method that allows for the posterior weights distribution to be learned through back-propagation, just as the weights of a non-Bayesian network are learned by conventional back-propagation. Predictions can then be made using the estimated posterior. 

More precisely, the output's expected value $\mathbb{E}[F_\theta(u, i)]$ is a point prediction for the interaction relevance $F_{ui}$, and its variance $Var[F_\theta(u, i)]$ is an estimate of the relevance's uncertainty $G_{ui}$. In practice, the values are estimated using samples $\theta_k, \ldots, \theta_k$ from the posterior, as follows,

\begin{align}
    F_{ui} &= \frac{\sum_{k=1}^n {F_{\theta_k}(u, i)}}{n} \label{eq:relevance} \\
    G_{ui} &= \frac{\sum_{k=1}^n \big( {F_{\theta_k}(u, i) - r_{ui} \big) ^2}}{n} \label{eq:uncertainty}
\end{align}

Another uncertainty estimation method that is tailored to neural networks is MCDropout \cite{gal2016dropout}. The method, which can be thought of as an approximation of a Bayesian network, consists of taking multiple forward passes with dropout enabled at prediction time.\footnote{Conventionally, dropout is enabled at training time and combats overfitting. In MCDropout, it is enabled at prediction time to sample a space of predictions.} Formally, let $F^{(k)}$, for $k=1, \ldots, n$ denote $k$ predictions calculated with dropout enabled. Then, the final estimates for relevance and uncertainty follow according to Equations \ref{eq:ensemble} and \ref{eq:ensemble_unc}.

Interestingly, MCDropout has been used to estimate the uncertainty of a \acrshort{rs} in \cite{zeldes2017deep}. But, to the best of our knowledge, the usage of Deep-Ensembles and BNNs for uncertainty estimation remain unexplored in the field. In Section \ref{sec:results2}, we fill this gap by comparing all of these methods as uncertainty estimators for \acrshort{irss}.

\section{Evaluating Uncertainty Estimates}
\label{sec:evaluating2}

With several uncertainty estimators defined, we now turn our attention to their evaluation. In the previous chapter, we explored a very wide set of evaluation methods for uncertainty estimate in \acrshort{erss} (see Section \ref{sec:evaluating}). Naturally, in the absence of ratings, most of them are no longer useful. Nevertheless, all the evaluation criteria for \acrlong{rabr} (see Section \ref{sec:eval_rabr}) can be translated into \acrlong{rebr}. Furthermore, some of the expected correlations still hold. In this section, we list the uncertainty evaluation metrics that are applicable to \acrshort{irss}, highlighting any necessary modifications.

\subsection{Expected correlations}
Three of the patterns that we discussed in the previous chapter and that we expect to observe with uncertainty estimates can also be measured in the case of \acrshort{irs}: the negative correlations between uncertainty and user profile size; the negative correlations between uncertainty and item support; and the negative correlation between recommendation uncertainty and accuracy. These expected correlations follow from the same reasoning and can be measured exactly as for \acrshort{ers}.


\subsection{Top-$n$ recommendation}

%\victor{There seems to be a lot of confusion from here onwards. Several of your comments are in relation to PRR. I remind that we don't define PRR for IRSs, and therefore we only have ReBR and UBF. I changed the whole structure of this section to match chapter's 3.}

%\derek{You need to divide this into three subsections: ReBR, UBF and PRR. The URI stuff goes into the ReBR section; the stuff you have below in 4.3.2.1 will now be in 4.3.2.2. And the PRR subsection may simply remind the reader of the idea and say that it applies without change, at least to those recommenders that can predict probabilites.}

Because we no longer consider explicit ratings, all the uncertainty evaluation tailored to rating prediction (Section \ref{sec:eval_rating_prediction}) do no apply, and therefore we jump straight to the top-$n$ recommendation case. %First we will consider recommendations given with \acrshort{rebr}. In sequence, we remind the application-based evaluation through \acrshort{ubf} that generalizes to \acrshort{irss}.

\subsubsection{Relevance-based ranking}

To begin with, we generalize \acrfull{uri} to make it suitable to \acrshort{irss}. Its motivation remains the same: recommendations that have lower uncertainty are expected to be more precise. Therefore, \acrshort{uri} compares the uncertainty of each item against the average uncertainty in the recommendation list. The only adaptation we must make to \acrshort{uri} is to get rid of the relevance threshold $\omega$ (from Eq.\ \ref{eq:uri_crit}), because all the observed interactions are assumed to be equally relevant. Therefore, the \acrshort{uri} for \acrshort{irss} is defined as:

\begin{equation}
    C^{URI}_{u} = \sum_{i \in Z_u^n | i \in R_{u\cdot}} ( \overline{\rho}_u - G_{ui})
\end{equation}
where $\overline{\rho}_u$ is the mean uncertainty for the items in $Z_u^n$, i.e.\ the $n$ items that are recommended to user $u$. 

Now, the URI is defined as:
\begin{equation}
    URI = \ddfrac{\sum_{u} C^{URI}_{u}}{{\sigma_{\rho_u} \sum_{u}\#\{i \in Z_u^n | i \in R_{u\cdot}\}}}
\end{equation}
where $\sigma_{\rho_u}$ is the standard deviation of the uncertainty values for $Z_u^n$.

\subsubsection{Uncertainty-based filtering}

Of the two types of uncertainty-aware ranking (\acrlong{ubf} and \acrlong{prr}), only the former (\acrshort{ubf}) extends to \acrshort{irss}. We recall that \acrshort{ubf} is the form of ranking that discards candidate items whose estimated uncertainty is above a threshold $\tau$ (see Section \ref{sec:ubf}). Therefore, we can use it again as a form of evaluation. Identically to the description in Section \ref{sec:eval_ubf}, our proposal is to progressively increase the uncertainty threshold for filtering $\tau$ while measuring the accuracy after each cutoff. In addition, it is also important to measure the average relevance of the recommended items because, with more candidates discarded, the average relevance of recommendations will drop. We calculate Mean Predicted Relevance as,

\begin{equation}
    \text{Mean Predicted Relevance}@n = \frac{1}{\#U} \sum_{u \in U} \frac{1}{n} \sum_{i \in Z_u^n} F_\theta(u, i)
    \label{eq:avg_relevance2}
\end{equation}

Finally, we must also measure coverage, to measure the extent to which we are still able to make $n$ recommendations as we discard ever more candidates. The same coverage metric from Eq.\ \ref{eq:coverage} can be employed. 

% \subsection{Rating prediction}
% \subsubsection{Rating-based ranking}
% \subsubsection{Probability-of-relevance ranking}

\section{Empirical Study}
\label{sec:results2}

In this section, we aim to compare the uncertainty estimation methods proposed in Section \ref{sec:estimating2}. To do so, we use the evaluation methods and metrics explored in Section \ref{sec:evaluating2}. For reproducibility, we make all the code used available\footnote{https://github.com/vcoscrato/uncertain}. Our implementation is based on PyTorch \cite{paszke2017automatic}. %\derek{Do we need to say this here? Hasn't it been said, and the URL given, in Chapter 1?} \victor{It's a slightly different link (a different branch on git), and I don't think repeating it hurts much.}

\subsection{Models}

Our comparative study in this chapter include the following uncertainty estimators: $\isup$, MF-ENSEMBLE, MLP-ENSEMBLE, BayesianMLP and MCDropout. 

%With the exception of BayesianMLP, all of t
These methods rely on a separate, underlying relevance prediction model ($F$). For this model, we adopt either MF or MLP, as described in Section \ref{sec:models}. To choose the best underlying model in each case, we conduct a comparison between several versions of MF and MLP (see Section \ref{sec:baselines}). The best performing models will have two uses. First, they will be used as the underlying relevance prediction models; %, whenever applicable (e.g. for $\isup$);
and second, they will serve as a baseline for recommendation accuracy. Our goal is to assess whether the uncertainty estimators will help improve accuracy relative to the baseline or not. Table \ref{tab:baseline_models} shows the relevance prediction models that we compare.

\begin{table}[t]
    \centering
    \begin{tabular}{|c|c|c|} \hline
    \bf{Notation}  &  \bf{Prediction model}  &  \bf{Learning objective} \\ \hline
    MF-MSE  &  MF (Eq.\ \ref{eq:MF})  &  MSE (Eq.\ \ref{eq:mse_loss}) \\ \hline
    MF-BCE  &  MF (Eq.\ \ref{eq:MF})  &  BCE (Eq.\ \ref{eq:bce_loss2}) \\ \hline
    MF-BPR  &  MF (Eq.\ \ref{eq:MF})  &  BPR (Eq.\ \ref{eq:bpr}) \\ \hline
    MLP-MSE  &  MLP (Eq.\ \ref{eq:MF})  &  MSE (Eq.\ \ref{eq:mse_loss}) \\ \hline
    MLP-BCE  &  MLP (Eq.\ \ref{eq:MF})  &  BCE (Eq.\ \ref{eq:bce_loss2}) \\ \hline
    MLP-BPR  &  MLP (Eq.\ \ref{eq:MF})  &  BPR (Eq.\ \ref{eq:bpr}) \\ \hline
    \end{tabular}
    \caption{Relevance prediction models that we compare (with equation numbers). These are used for $F$, but also as baselines in accuracy comparisons.}
    \label{tab:baseline_models}
\end{table}

In Table \ref{tab:models}, we list all uncertainty estimation methods that we consider, also highlighting the used underlying relevance prediction model. Note that the objective function used to learn the underlying model will be the best one, for each dataset, according to Section \ref{sec:baselines}.

\begin{table}[t]
    \centering
    \caption{Uncertainty estimation methods that we compare.}
    \begin{tabular}{|c|c|c|} \hline
        \bf{Notation} & \bf{Uncertainty estimator} & \bf{Relevance model} \\ \hline
        NIS & ~$\isup$ (Eq.\ \ref{eq:isup}) & Best baseline \\ \hline
        MF-Ensemble & ~Ensemble (Eqs.\ \ref{eq:ensemble} -- \ref{eq:ensemble_unc}) & Best MF baseline \\ \hline
        MLP-Ensemble & ~Ensemble (Eqs.\ \ref{eq:ensemble} - \ref{eq:ensemble_unc}) & Best MLP baseline \\ \hline
        BayesianMLP & ~Bayesian inference (BBB) (Eqs.\ \ref{eq:relevance} -- \ref{eq:uncertainty}) & Best MLP baseline \\ \hline
        MCDropout & ~Monte-Carlo Dropout (Eqs.\ \ref{eq:ensemble} -- \ref{eq:ensemble_unc}) & Best MLP baseline \\ \hline
    \end{tabular}
    \label{tab:models}
\end{table}

\subsection{Datasets}
\label{sec:data4}
We evaluate our models and uncertainty estimation methods on two datasets: an implicit version of the MovieLens-1M dataset \cite{harper2015movielens}\footnote{To make this dataset implicit, we simply treat every given rating as an implicit signal (1), ignoring the numeric rating value.} and one Pinterest dataset \cite{geng2015learning}. These are very popular and openly available datasets. Table \ref{tab:data} presents some summary statistics.

\begin{table}[t]
    \centering
    \begin{tabular}{|c|c|c|} \hline
        \textbf{Dataset} & \bf{MovieLens-1M} & \bf{Pinterest} \\\hline
        \textbf{num. users}, $\#U$ & 6,040 & 55,187 \\\hline 
        \textbf{num. items}, $\#I$ & 3,416 & 9,643 \\ \hline
        \textbf{num. ratings}, $\#R$ & 1 Million & 1.5 Million \\\hline
        \textbf{sparsity} & 95.15\% & 99.72\% \\\hline
        \textbf{avg. num.\ ratings per user} & 166 & 27 \\\hline
        \textbf{avg. num.\ ratings per item} & 293 & 156 \\\hline
    \end{tabular}
    \caption{Characteristics of the datasets used in our experiments.}
    \label{tab:data}
\end{table}

\subsection{Methods}

\subsubsection{Data splitting}
For both datasets, we use a user-based chronological data splitting method: we chronologically order the interactions of each user and place the last 20\% of them in the test set. The remaining 80\%  of interactions for each user are used for training or validation. More precisely, we order these remaining ratings chronologically; for each user, we use the first 80\% of these for training, and the rest for validation.

\subsubsection{Tuning}
\label{sec:tuning4}

MF, MLP and BayesianMLP have hyper-parameters that need to be tuned. We tuned our models using a Bayesian parameter search, assisted by Optuna \cite{optuna_2019}. More precisely, we define a parametric search space and train multiple models by sampling parameter settings from the space.\footnote{In a Bayesian parameter search, the sampler is initially random and then progressively guided by the performance of the learned models. For details, see \cite{nguyen2019bayesian}.}  For each algorithm, we learned 20 models with different parameter settings and chose the one with the highest MAP@10 (see Eq.\ \ref{eq:map}).

We sample hyper-parameters from the following:

\begin{itemize}
    \item For all models, learning rate is sampled from $[0.0001, 0.01]$.
    \item For models that involve negative sampling (Point-wise objectives), the number of negative training instances $N$ per positive training instance is sampled from $\{1, 2, ..., 20\}$.
    \item MF: The L2 penalty factor applied to the user and item factors was taken from $[10^{-5}, 10^{-2}]$.
    \item MLP: We use the same three-layer MLP as in \cite{he2017neural}. We also employ dropout on the training stage. The dropout rate is tuned in $[0, 0.2]$.
    \item BayesianMLP: We use a Bayesian MLP with the same architecture as our deterministic MLP. We use the same prior and tune the hyper-parameters related to it across the exact same grid as used in \cite{blundell2015weight}.
\end{itemize}

For MF-ENSEMBLE and MLP-ENSEMBLE, we set the ensemble size to $n=5$. We experimented with larger sample sizes ($n=10$ and $n=20$), but found they all produced similar results.

The user and item latent embeddings size $d$ to 128 for all models. Setting them to the same size gives a fair comparison. While it has been shown that both MF and MLP can benefit from even higher dimensions \cite{rendle2020neural}, $d=128$ gives us reasonable computational cost. 

Finally, to avoid the need to tune the number of training iterations, we employ early-stopping to end the learning phase when the MAP@10 on the validation set does not improve for three consecutive iterations.

\section{Results}

In this section, we present the results\footnote{We note that the results presented here are not identical to the results that we published in \cite{coscrato2022}. The major difference is that we added a more complex search for optimal baselines (see Table \ref{tab:baseline_models}). As a consequence, there are small variations in the performance of the uncertainty estimators that use the baselines as their underlying models.}. We begin with the comparison between the baselines. From this comparison, we adopt the best performing ones as the underlying relevance prediction models for our uncertainty estimators. Then, we compare the uncertainty estimators using the methods and metrics from Section \ref{sec:evaluating2}.

\subsection{Baseline models}
\label{sec:baselines}

Figure \ref{fig:baselines} illustrates the recommendation accuracy for each baseline. In both datasets, we observe a clear advantage of MF over MLP. This is expected, because the same has been observed in \cite{rendle2020neural}. 

For the MovieLens dataset, the MAP and Recall of MF-BPR and MF-BCE are almost indistinguishable, while MF-MSE falls behind. Interestingly, the opposite is observed in the Pinterest dataset: MF is the most accurate method, irrespective of the loss function, according to both MAP and Recall.

Focussing on MLP, MLP-BCE is better than MLP-MSE and MLP-BPR in almost every case. The only exception is the Recall on the MovieLens data, where all MLP methods had almost identical performance.

\begin{figure}[tp]
\centering
\begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=\linewidth, height=230pt]{img/Implicit/Movielens/baselines4.pdf} 
    \caption{MovieLens dataset.}
\end{subfigure}
\begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=\linewidth, height=230pt]{img/Implicit/Pinterest/baselines4.pdf} 
    \caption{Pinterest dataset.}
\end{subfigure}
\caption{MAP@k and Recall@k, for $k = 1, 2, \ldots, 10$ for all the baseline models on the MovieLens and Pinterest datasets.}
\label{fig:baselines}
\end{figure}

According to the results obtained, our choice for underlying models are: NIS and MF-Ensemble will use MF-BPR for the MovieLens dataset and MF-MSE for the Pinterest. MLP-Ensemble, BayesianMLP and MCDropout will use MLP-BCE for both datasets.

\subsection{Expected correlations}

We begin our uncertainty evaluation by analyzing correlations between the uncertainty estimates and: the user profile size ($\#R_{u\cdot}$); and item support ($\#R_{\cdot i}$).

We follow exactly the same procedure from Section \ref{sec:results_correlations}. That is, we calculate the correlations from a randomly sampled set of 100,000 user-item pairs. Figure \ref{fig:corr4} illustrates the results. Note that the correlation between $\isup$ and $\#R_{\cdot i}$ is trivially perfect, and therefore ignored.

In the MovieLens data, we observe many of the expected correlations: the uncertainties estimated by MLP-ENSEMBLE, MCDropout and BayesianMLP are negatively correlated with both user profile size (weakly) and item support (strongly). The exception to the rule is MF-ENSEMBLE. The uncertainties estimated by the model grow with both $\#R_{u\cdot}$ and $\#R_{\cdot i}$. Interestingly, the same counter-intuitive behaviour was also observed when using a MF-ENSEMBLE for \acrshort{erss} (see Section \ref{sec:results_correlations}). This is also the case for MF-ENSEMBLE in the Pinterest dataset (\ref{fig:corr4}, b).

Furthermore, in the Pinterest data, we also observe negative correlations between MCDropout and BayesianMLP uncertainties and $\#R_{\cdot i}$ although the magnitude of the correlations are much smaller than for the MovieLens dataset. In addition, these models' uncertainties do not present any noticeable correlations with $\#R_{u\cdot}$. Finally, we do not observe correlations between MLP-Ensemble's uncertainties and either $\#R_{\cdot i}$ or $\#R_{u\cdot}$.

It is reassuring that the three neural models have similar correlation results. After all, MLP-ENSEMBLE and MCDropout are approximations of the BayesianMLP. As for the counter-intuitive results observed with MF-ENSEMBLE in both datasets, these combine with similar results from Section \ref{sec:results_correlations} to reinforce the concerns regarding this as a method for uncertaint estimation.

\begin{figure}[tp]
\centering
\begin{subfigure}{0.8\linewidth}
    \centering
    \includegraphics[width=\linewidth]{img/Implicit/Movielens/corr4.pdf}
    \caption{MovieLens dataset.}
\end{subfigure}
\begin{subfigure}{0.8\linewidth}
    \centering
    \includegraphics[width=\linewidth]{img/Implicit/Pinterest/corr4.pdf} 
    \caption{Pinterest dataset.}
\end{subfigure}
    \caption{Correlations between uncertainty estimates and dataset statistics. We expect negative correlation with $\#R_{u\cdot}$ and $\#R_{\cdot i}$.}
    \label{fig:corr4}
\end{figure}

\subsection{Top-$n$ recommendation}

We now turn our attention to the most important task of a \acrshort{rs}: top-$n$ recommendation.

\subsubsection{Relevance-based ranking}
We begin by comparing our models' accuracy for recommendations given using \acrfull{rebr}. To do so, we calculate the MAP@n and Recall@n, averaged over all users, for different recommendation list sizes $n \in 1, 2, \ldots, 10$. Figure \ref{fig:accuracy4} shows the results.

\begin{figure}[tp]
\centering
\begin{subfigure}{0.8\linewidth}
    \centering
    \includegraphics[width=\linewidth]{img/Implicit/Movielens/accuracy4.pdf}
    \caption{MovieLens dataset.}
\end{subfigure}
\begin{subfigure}{0.8\linewidth}
    \centering
    \includegraphics[width=\linewidth]{img/Implicit/Pinterest/accuracy4.pdf} 
    \caption{Pinterest dataset.}
\end{subfigure}
    \caption{\acrshort{rebr}: MAP@n and Recall@n for $n \in 1, 2, \ldots, 10$. The MF and MLP correspond to those with highest MAP@10, for each dataset, according to Figure \ref{fig:baselines}.}
    \label{fig:accuracy4}
\end{figure}

The foremost result from the graphs is the pronounced advantage of the ensemble-based models, for both datasets and according to both metrics, in comparison with the baseline models (MF and MLP). Intriguingly, we have repeatedly observed such an advantage (see also Section \ref{sec:results_rabr}), even though ensembles do not seem great in relation to our uncertainty evaluation, both for \acrshort{erss} (Section \ref{sec:eval_top_n_recommendation}) and \acrshort{irss}, as we showed in the previous section and as we will confirm later in this chapter.

Moreover, we observe that both MCDropout and BayesianMLP are superior to the corresponding MLP baseline, for both metrics and both datasets. This, together with the success of MLP-ENSEMBLE, gives support to the usage of Bayesian Neural Networks, or their approximations, in \acrshort{rss}, although, we note that all MLP-based methods used here are worse than the baseline MF, in all cases.

We now turn our attention to the uncertainty estimates. We aim to verify the validity of the uncertainty estimates provided by all our researched methods on top-$n$ recommendations. To evaluate the uncertainty estimates produced by each method quantitatively, we employ URI. The results are presented in Table \ref{tab:rebr_uncertainty}. A positive URI implies that the less uncertain a recommended item is compared to the average uncertainty in the user's recommendations, the more likely it is to be relevant.

Unfortunately, we do not observe a high URI with any of our methods. In fact, we even observe a negative URI (although one of very small magnitude) in many cases. The exception is $\isup$. In both datasets, $\isup$ was the only model that achieved a considerably positive URI. It means that, the more popular an item is in regards to the average uncertainty in the recommendation list it appears in, the more likely it is to be relevant.

%\db{Is there anything we can say about this 'negative' result. Maybe something about whether URI is a suitable metric?} \victor{Interestingly, I found a way to squeeze a reasoning about label uncertainty in here as well, see how you like it.}

%\victor{New paragraph:} 
One could argue that these negative results point to a potential problem with the URI metric, rather than with the uncertainty estimators. URI assumes that correct recommendations -- that is, the recommendation of relevant items -- should be given with lower uncertainty. Although this makes sense, we recall that relevance is inferred through historical interactions, but it is very plausible that at least some unobserved interactions are relevant\footnote{In fact, this is the premise of label uncertainty, as introduced in Chapter \ref{chapter:introduction} and explored in detail in Chapter \ref{chapter:implicit_learn}.}. In these cases, is it right to evaluate uncertainty estimates through URI? We leave for future exploration the question of whether this is explains the results we have obtained.

\begin{table}[tp]
    \centering
    \begin{tabular}{|l|r|r|} \cline{2-3}
    \multicolumn{1}{c|}{} & MovieLens & Pinterest \\ \hline
    NEG-ITEM-SUPPORT &   \textbf{0.1324}  & \textbf{0.1306} \\ \hline 
    MF-ENSEMBLE      &  -0.0360 & -0.1281 \\ \hline
    MLP-Ensemble     &  -0.0358 & -0.0100 \\ \hline
    MCDropout        &  -0.0306 & -0.0845 \\ \hline
    BayesianMLP      &   0.0238 & 0.0288 \\ \hline
    \end{tabular}
    \caption{\acrlong{rabr} uncertainty evaluation: URI on MovieLens and Pinterest datasets. The best (highest) values are highlighted in bold.}
    \label{tab:rebr_uncertainty}
\end{table}

\subsubsection{Uncertainty-based filtering}
\label{sec:results_ubf4}

Of more importance than URI is the suitability of the uncertainty estimators to their use cases. In \acrshort{ubf}, the uncertainty is used to prevent items from being recommended to users when the user-item pair has high estimated uncertainty. The filtered recommendation lists will have smaller average uncertainty, and are therefore expected to be more precise. 

The results reported here follow the exact same procedure used in Section \ref{sec:eval_ubf}. For each uncertainty estimator, we progressively exclude bigger fractions of the most uncertain candidates from the pool, according to cut-offs ($\tau$) that represent the 20\%, 40\%, 60\% and 80\% quantiles of their uncertainty distributions. 

The results are shown in Figure \ref{fig:cuts4}. For every estimator, and every filtering threshold, we present the MAP@10 (Eq.\ \ref{eq:map}), the Mean Predicted Relevance@10 (Eq.\ \ref{eq:avg_relevance2}) and the Average Coverage (Eq.\ \ref{eq:coverage}).

First, we observe that in no case does MAP increase with the use of \acrshort{ubf}. In addition, in both datasets, we observe that the MAP for some models progressively decreases as $\tau$ grows: among those, MF-ENSEMBLE is the most noticeable. Interestingly, MF-ENSEMBLE is the only method whose Mean Predicted Relevance  also decreases considerably when the cutting threshold is increased. This steep decrease in relevance is a possible explanation of why its MAP decreases so drastically. We do not observe any loss of coverage in any case.

\begin{figure}[tp]
\centering
\begin{subfigure}{\textwidth}
    \centering
    \includegraphics[width=0.9\linewidth]{img/Implicit/legend.png}
\end{subfigure}
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=0.9\linewidth]{img/Implicit/Movielens/cuts4.pdf} 
    \caption{MovieLens dataset.}
\end{subfigure}
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=0.9\linewidth]{img/Implicit/Pinterest/cuts4.pdf} 
    \caption{Pinterest dataset.}
\end{subfigure}
\caption{\acrshort{ubf}: MAP, Mean Predicted Relevance and Coverage. This is for top-10 recommendations with increasing $\tau$ to filter different quantiles.}
\label{fig:cuts4}
\end{figure}

With the exception of MF-ENSEMBLE, we note that in every other case, and in both datasets, the MAP and Mean Predicted Relevance varies very little, even when 80\% of the candidate interactions are removed. From this behaviour, we postulate that the recommendations are not changing much after filtering. That is, recommendations given using \acrshort{ubf} are similar to the traditional \acrshort{rebr} recommendations. This would happen only if the majority of the items recommended with \acrshort{rebr} have low uncertainty. 

To verify this, we observe the relationship between relevance and uncertainty. In Figure \ref{fig:relevance_uncertainty}, we illustrate the distribution of relevance and uncertainty for a randomly sampled set of interactions, for each uncertainty estimator, in the MovieLens dataset\footnote{We omit the corresponding figures for Pinterest, where we observed the same behaviour.}. Notice, in every case except MF-ENSEMBLE, the negative correlation between relevance and uncertainty. This explains why \acrshort{ubf} does not impact recommendations very strongly: uncertain interactions are not being recommended with \acrshort{rebr} in the first place. Again, MF-ENSEMBLE is an exception, but unfortunately, it also appears unsuccessful due to low URI and the loss of MAP on \acrshort{uri}.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{img/Implicit/Movielens/relevance_uncertainty.pdf}
    \caption{The distribution of relevance and uncertainty estimates on the MovieLens dataset. Each point represents a randomly sampled interaction. The black lines are least squares tendency lines.}
    \label{fig:relevance_uncertainty}
\end{figure}

Therefore, we conclude that none of the methods explored in this chapter are particularly successful at uncertainty estimation in \acrshort{irss} although we note the accuracy uplift observed with some of them.

\section{Conclusions and Limitations}
\label{sec:conclusion}

In this chapter, we explored methods for uncertainty estimation for implicit feedback-based recommender systems. In comparison to the explicit feedback case (see Chapter \ref{chapter:explicit}), uncertainty estimation for \acrshort{irss} was even less explored prior to our work. We decided to emphasize neural network-based \acrshort{rss} because of their prominence in the \acrshort{rss} field. 

The main takeaways from this chapter are:
\begin{itemize}
    \item We confirm the superiority of MF over MLP --- at least for these datasets. Even though this has already been previously shown by Rendle et al.\ \cite{rendle2020neural}, our methods (e.g.\ our data-splitting) differs from theirs. Therefore, our results give stronger support to this finding.
    \item We found that, in some cases, our systems that include uncertainty estimators have a higher recommendation accuracy then their respective MF or MLP baselines. The improvement is especially remarkable with the ensemble-based methods, but is also observed with MCDropout and BayesianMLP.
    \item In addition, MF-ENSEMBLE was also one of the top contenders when it came to correlation between accuracy and uncertainty, suggesting that this method can help to identify which users are prone to receive the most or least accurate recommendations.
    \item All the previous remarks highlight positives, but in all fairness all of our methods fail at their foremost goal of uncertainty estimation according to our metrics. We did not find evidence to support the usefulness of the uncertainty estimators in the chapter, especially in the top-$n$ recommendation task. With that said, it is true that the failure of the estimators may be partially due to the metrics' issues, as we pointed in the case of URI.
    %\db{Can we 'lift' this with something, maybe that it points to a need for future work, and/or that it led us to a different conception of uncertainty for irs that we explore in the next chapter?} \victor{I tried to lift a bit by criticizing the metrics, and I follows it up in the limitations below.}
\end{itemize}

In Chapter \ref{chapter:conclusion}, we will discuss possible future work on the estimation of prediction uncertainty in \acrshort{irss}. Our discussion there will partially build upon the following limitations that we identify in the work of this chapter.

\begin{itemize}
    \item We limited this chapter to MF and MLP models. Although we expect our findings regarding the estimation of prediction uncertainty in \acrshort{irss} to extend to other models, we can not guarantee they generalize to them all. In particular, there is a plethora of neural recommenders in the latest literature. We cannot be sure that our findings generalize to these various neural architectures.
    \item As in Chapter \ref{chapter:explicit}, we once more used a single train/test split, for reasons we already explained, and restricted our experiments to only two datasets. Therefore, although our results appear quite conclusive, it could be beneficial to extend our experiments to more datasets, particularly in different recommendation domains.
    \item We also acknowledge the limitation of our prediction uncertainty evaluation. In particular, the URI metric that we utilize is based on the relation between relevance and uncertainty. But again, we point to the disconnect between relevance and observation: not all unobserved interactions are irrelevant. Therefore, we believe the literature could strongly benefit from novel evaluation metrics, or at least  a meticulous investigation of the existing metrics.
\end{itemize}


%\dgb{(a) I'd still be keen to see a sentence that lifts the negativity of the last of the conclusions. (b) The limitations in chapter 3 were discussed more completely. Here, we seem to be rushing, and just sticking in a couple.} \victor{I think I managed to lift the results a bit and improve the discussion of the limitations.}