\chapter{Personalizing Loneliness Detection Through Behavioural Grouping of Passive Sensing Data From College Students}
\markboth{4. Personalizing Loneliness Detection}{}
\section{Overview}
As we discussed in Chapter 1, loneliness is a serious global issue specifically among younger population. While Chapter 2 highlighted the potential of passive sensing for loneliness detection in the existing literature but traditional one-size-fits-all approaches may overlook the diverse ways loneliness manifests in student populations. Current passive sensing-based methods for detecting loneliness use generalized models trained with all available data at once. In this method, the model learns broader patterns that are common among observations, which are then used for loneliness detection. Loneliness is a subjective experience, and its behavioural manifestations can vary among individuals based on their personality type and social contexts. Generalized models may overlook these subtle variations by focusing on broader patterns that are common across the entire population. However, by identifying subgroups of individuals with similar behavioural patterns, we can develop more targeted and personalized models that better reflect the unique ways loneliness is expressed in different groups.

We hypothesize that by identifying and analyzing subgroups of college students with similar behavioural patterns, we can capture more nuanced and specific behaviours related to loneliness that generalized models trained on the entire student population may overlook. Moreover, it is also important to analyze group dynamics over time to monitor changes and track the evolution of groups over time. This helps in identifying behavioural patterns and trends that are not apparent through traditional methods.

This chapter presents an empirical evaluation of the effectiveness of group-based loneliness classification models in college students using passive sensing. We used incremental clustering to identify groups of students with similar behaviours and to monitor changes in these groups over time. Subsequently, we developed tailored loneliness classification models for each group to compare group-based models with generalized models. In addition, this chapter also provides group-specific behavioural patterns within students. While clustering approaches have been utilized in various fields of studies \cite{xu2014cluster,hassan2022unsupervised}, their application to the understanding and classification of loneliness, particularly in the context of college students with passive sensing data is limited. All existing research on loneliness detection using passive sensing is based on generalized machine learning models for all available data that does not consider inter-subject differences or behavioural variations across groups. Moreover, existing studies have not analyzed the behavioural shifts of students in relation to loneliness during a study period. Our approach could lead to the development of more targeted and personalized interventions that consider the unique behaviours and experiences of different student subgroups. The key research questions we have explored in this chapter are:

\begin{enumerate}
  \item Can clustering methods identify distinct behavioural groups among college students using passive sensing data, and how do these groups evolve over time?
  \item How effective are group-based loneliness classification models compared to generalized models among college students using passive sensing data?
\end{enumerate}


\section{Methodology}

\begin{comment}
\subsection{Dataset and preprocessing}
We have thoroughly presented the datasets and preprocessing steps in Chapter 3. Here, we will just touch upon key aspects relevant to this study. The overall methodology has been presented in Fig.\ref{fig:methodsflowchart}. This study used the StudentLife dataset \cite{wang2014studentlife}.


Data collected via smartphone sensors (accelerometer, microphone, light, GPS, Bluetooth, Phone Usage) were used as these sensors have been identified in the literature as relevant to loneliness detection \cite{qirtas2022loneliness}. Note that, this study used one additional sensor (phone usage) as compared to sensors data used in chapter 4. Student responses to the UCLA Loneliness Scale \cite{russell1980revised} at the start and end of the semester provided ground truth measures of loneliness. We included 41 students data who completed the post study loneliness survey. To solve the missing values issue, we first eliminated all records containing outliers and then imputed missing data for each student using the median of a specific feature per session if its data values are continuous. For categorical data,
we used the mode of a particular feature for that session. To ensure equal feature contributions and enhance the identification of meaningful clusters, feature normalization (min-max scaling) was applied which converts each value to the range [0,1]. This step mitigates potential biases caused by varying scales in the original data, ensuring that all features have equal influence on the clustering process.

\begin{figure*}[htp] \centering{
\includegraphics[width=13cm, height=8cm]{figures/Ch4/whole_flow.pdf}
\caption{Overview of the Methodological Approach for Loneliness Detection in Students}
\label{fig:methodsflowchart}}
\end{figure*}
\end{comment}

\subsection{Finding Behavioural Groups from Student Data}

This study used the StudentLife dataset. We have mainly used an incremental clustering approach to capture the dynamic nature of student behaviours. Unlike traditional clustering methods, incremental clustering can process data as it arrives which allows for the identification of evolving behavioural patterns and timely updates to the clustering model. Since students' behaviours can vary over time due to academic pressures, social interactions, and personal circumstances, incremental clustering makes sense and can reflect real-time changes in student behaviour. To establish a baseline for comparison and to evaluate the clustering results' stability, we used multiple clustering algorithms to gain a broader understanding of the grouping in the dataset. We used weekly aggregated data as input for clustering methods. 

K-means is a classic partitioning algorithm that identifies clusters with well-defined centroids. \cite{sinaga2020unsupervised}. It is commonly used in behavioural analysis due to its simplicity and scalability. OPTICS is a density-based method used in handling datasets with varying densities \cite{ankerst1999optics}. PAM is similar to K-means that finds representative data points (medoids) as cluster centers, which helps in adding potential robustness against outliers often present in behavioural datasets \cite{van2003new}. IDBSCAN is also a density-based algorithm with an incremental version of the usual DBSCAN, and it works in identifying clusters of varying densities incrementally and handling noise within the data \cite{ester67incremental}. This selection of both partitioning (K-means and PAM) and density-based (OPTICS and IDBSCAN) algorithms was to capture the diverse nature of student behavioral patterns. K-means and PAM are effective when clusters are well-separated and spherical which may align with potential distinct groups of students based on factors like academic performance or social engagement. However, recognizing that student behaviours may not always fit into such categories, OPTICS and IDBSCAN can provide the flexibility to identify clusters with varying densities and shapes which might be overlooked by traditional methods.

Before applying the clustering techniques, we first normalized the weekly data using min-max scaling to ensure all features were on the same scale. Then, we applied Principal Component Analysis (PCA) to reduce the dimensionality of the data \cite{jolliffe2016principal}. PCA combines the original variables (behavioural features) into a smaller set of new variables called principal components. These principal components are like condensed summaries of the original data that capture the most significant variations in the behavioural patterns while using fewer dimensions. We determined the optimal number of principal components through the explained variance threshold, scree plot analysis, and empirical experimentation to evaluate how the number of components affects clustering quality results.

\subsubsection{Determining the Optimal Number of Clusters}
For K-means and PAM algorithms, we must define the number of potential clusters beforehand. We used the elbow method to make this selection \cite{joshi2013modified}. This method works by iteratively fitting the clustering model with different numbers of clusters (K values). For each K, a measure of clustering quality (within-cluster sum of squared errors) is calculated. By plotting these quality scores against the corresponding K values, the elbow point on the graph shows the K value beyond which adding more clusters does not significantly impact. The goal is to achieve a compact and well-separated clustering solution without unnecessarily complicating the model. We initially determined the optimal number of clusters using Week 1 data and then re-evaluated this on Week 5 based on cluster quality results to ensure the clusters remain relevant as more data accumulates.

\subsubsection{Evaluating Cluster Quality}
We used two evaluation metrics to assess the quality of clusters: the Silhouette score \cite{rochman2022comparison} and the Xie-Beni Index \cite{xie1991validity}. The Silhouette score evaluates the consistency of clusters. It measures how well each data point fits within its assigned cluster compared to its similarity to points in neighbouring clusters. It provides a value between -1 and 1, where a high value shows that the object is well matched to its own cluster and poorly matched to neighbouring clusters. This score validates the consistency within clusters and the differentiation between them which tells whether the data points have been grouped into meaningful clusters or not. The Silhouette score is particularly useful for confirming the appropriateness of the number of clusters chosen and it ensures that each cluster represents a distinct group with similarities.

The Xie-Beni index provides a combined assessment of cluster compactness and separation. Compactness means how closely data points are grouped within a cluster, while separation measures the distance between distinct clusters. A lower Xie-Beni Index means a better clustering structure with tight and well-separated clusters. This index is useful for confirming that the clusters are distinct and do not overlap significantly. We calculated these scores weekly to continuously assess clusters quality. 

\subsubsection{IDBSCAN Algorithm} 
After evaluating the clustering results produced by different algorithms in Figure~\ref{fig:silhoutte} and Figure~\ref{fig:xieben}, we selected IDBSCAN for further analysis. It is an incremental variation of the density-based DBSCAN algorithm that identifies clusters based on the density of data points in a region. This makes it suitable for discovering clusters of arbitrary shapes and handling outliers, unlike traditional distance-based methods like k-means \cite{xu2015comprehensive}. It is designed to adapt to evolving data distributions. This aspect of the algorithm helps to update and refine cluster assignments as new data becomes available without retraining the whole model with all the data available. This incremental approach aligns with the evolving nature of student behaviours captured through passive sensing data which provides a more accurate and up-to-date reflection of students' loneliness-related behavioural patterns. Upon the arrival of new data, the algorithm calculates the mean distance between each new item and the core objects of existing clusters. If a new data item is within the specified radius (eps) of any cluster core and the number of points in the cluster satisfies the minimum required threshold (minPts), the item is added directly to the nearest cluster. Otherwise, if the item does not meet these criteria, it is initially marked as an outlier. Outliers that accumulate to meet the minPts threshold can subsequently form new clusters.

To find the optimal values for the eps (distance threshold) and minPts (minimum points) parameters, we used a two step approach. First, we conducted pilot runs with different combinations of eps and minPts values to assess cluster quality using the Silhouette score and Xie-Beni index. These initial experiments helped us to identify suitable ranges for both parameters. Then we used GridSearchCV with cross-validation to find the best values through the ranges identified. This involved systematically testing various combinations of eps and minPts values found through pilot runs and evaluating their performance for cluster quality. The combination that resulted in the highest average silhouette score across the cross-validation folds was then selected as the optimal parameter. 

\subsection{Loneliness Detection for each Group}
After identifying behavioural subgroups through weekly incremental clustering, we implemented a personalized approach to loneliness detection. This personalized approach acknowledges that the relationship between behavioural patterns and loneliness may vary across different student subgroups. We trained four binary classifiers for each subgroup; logistic regression, random forest, support vector machines (SVMs), and XGBoost. To evaluate and select the best model for each group, we used 10-fold stratified nested cross-validation with a 3-fold inner loop for hyperparameters tuning. It is important to note that leave-one-subject-out cross-validation (LOSO) was not applied in this chapter. LOSO excludes one participant at a time for model testing which provides a more robust evaluation of model generalizability. Due to the small size of the dataset and the group-based approach used in this chapter, applying LOSO would have been challenging, as removing one participant from small groups could lead to overly sparse training sets and impact model stability. That is why, we relied on standard cross-validation for evaluating model performance.

We handled missing values, feature scaling, feature selection, applied SMOTE to address class imbalance in the inner fold of the cross-validation. We applied the same scaling parameters, imputation values and selected features derived from the training set to the corresponding validation and test sets. We selected the best model in the inner loop using F1-macro. The final performance metrics used for evaluation included accuracy, precision, sensitivity, and F1 Score.

\subsection{Loneliness Detection using all Students Data}
To evaluate the effectiveness of our personalized, group-based approach, we compared its performance to generic models trained on the combined dataset without any grouping. For the generic models, we used the same classification algorithms as in the group-based approach: logistic regression, random forest, support vector machines (SVMs), and XGBoost. We again used the same nested cross-validation and other steps with 10 outer folds and 3 inner folds for hyperparameter tuning and model selection. The final performance metrics used for evaluating both the generic and group-based models included accuracy, precision, sensitivity, and F1 Score.


\section{Results}

\subsection{Evaluating Cluster Quality}
To assess the effectiveness of each clustering algorithm, we evaluated cluster quality using the Silhouette score and the Xie-Beni Index. Figure~\ref{fig:silhoutte} shows the Silhouette scores for each of the algorithms used in a boxplot form. Based on the results, IDBSCAN has the highest Silhouette score which shows that data is properly combined in groups for this algorithm and that the inter-cluster and intra-cluster distances are excellent for those groups. Similarly, Figure~\ref{fig:xieben} shows the Xie-Beni Index scores for different algorithms in which IDBSCAN has the lowest score, which means clustering results are better for IDBSCAN as compared to other methods in terms of compactness and separation of the clusters. The idea behind the Xie-Beni Index is to minimize the distance between points within a cluster while maximizing the distance between different clusters. OPTICS and PAM also showed reasonable results which indicates a some degree of cluster structure. IDBSCAN's and OPTICS ability to find clusters of arbitrary shapes and their adaptability to varying densities within the data likely contributed to their superior performance.


\begin{figure}[H]
  \centering
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/BoxPlotSilohutee.pdf}
    \caption{Boxplot comparing Silhouette scores}
    \label{fig:silhoutte}
  \end{subfigure}
  \hfill 
   \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/xieben_index.pdf}
    \caption{Boxplot comparing the Xie-Beni Index}
    \label{fig:xieben}
  \end{subfigure}
    \caption{Clustering Algorithms Comparison}
\end{figure}

\subsection{Temporal Evolution of Student Groups and Behavioural Profiles}
To examine the dynamic nature of student behaviours over the 10-week period, we applied the clustering in an incremental way. Data from each subsequent week was integrated into the existing model. The aim was to track behavioural changes and the evolution of groups throughout the study period of 10 weeks. To monitor changes in students' group memberships over time, each student was assigned to the cluster where the majority of their weekly data points belonged for that specific week. This means that students are not fixed in one group for the entire study but can move between groups over time that reflects shifts in their behavioural patterns. By the end of Week 10, it is possible for a student's data points to belong to multiple groups across different weeks. The purpose of these weekly group assignments is to provide a more dynamic understanding of how behavioural patterns evolve over time and to show the changing nature of group membership over the 10-week period.

\begin{figure*}[htp] \centering{
\includegraphics[width=16cm, height=10cm]{figures/Ch4/groups_evolution.png}
\caption{Visualization of Students Temporal Behavioural Group Dynamics over a 10-Week Period (G1:W1:9 denotes Group 1 in Week 1 with 9 students)}
\label{fig:dynamicsstudentlife}}
\end{figure*}

Figure~\ref{fig:dynamicsstudentlife}, shows the evolution of student groups over the 10-week period. The x-axis represents each week while the y-axis shows the number of groups and number of students per group. It shows the stability of groups and transitions of students within the group dynamics. Initial analysis from weeks 1 to 6 show a relatively stable grouping with three distinct behavioural groups. These groups are labelled G1, G2, and G3 in the figure, with the number of students in each group indicated beside the group label (e.g., G1:W1:9 denotes Group 1 in Week 1 with 9 students). A change occurred from Week 7 onwards, with the model identifying four distinct groups (labelled G1, G2, G3, and G4). This shows a potential change in behavioural patterns across the student population around this time. It is important to note that the group sizes also show fluctuations throughout the ten weeks which highlights the dynamic nature of the group members.

\subsubsection{Group Behavioural Profiles Analysis}
To get a deeper understanding of the distinct behavioural patterns represented by each group, we analyzed average group features on a weekly basis. This aggregation gives a clear and interpretable view of the dominant behavioural tendencies within groups over time. We aimed to identify the main behavioural patterns that differentiate each group by focusing on key behavioural metrics (activity duration, conversation duration, phone usage duration, and location transitions). Figures~\ref{fig:activityHeatmap}, ~\ref{fig:conversationHeatmap}, ~\ref{fig:phoneusageHeatmap} and ~\ref{fig:locationHeatmap} presents a visual representation of how key behavioural features vary across the student groups over the 10-week study period. 

\begin{figure}[H]
  \centering
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/heatmap_activity_duration.pdf}
    \caption{Activity duration}
    \label{fig:activityHeatmap}
  \end{subfigure}
  \hfill 
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/heatmap_conversation_duration.pdf}
    \caption{Conversation duration}
    \label{fig:conversationHeatmap}
  \end{subfigure}
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/heatmap_phone_usage_duration.pdf}
    \caption{Phone usage duration}
    \label{fig:phoneusageHeatmap}
  \end{subfigure}
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch4/heatmap_location_transitions.pdf}
    \caption{Location Transitions}
    \label{fig:locationHeatmap}
  \end{subfigure}
    \caption{Average Weekly Behavioural Patterns (normalised) for 4 groups over a 10-week period}
\end{figure}

\subsubsection*{Activity Patterns}
Figure~\ref{fig:activityHeatmap} presents a heatmap that shows the activity duration patterns for each group throughout the study period of 10 weeks. This provides a comparison of the duration of physical activity. Groups 1 and 3 had overall shorter duration of activity as compared to Group 2. An observable pattern is the gradual increase in activity duration for Group 1 which shows a change in their activity level over a period of time. Group 4 had higher levels for weeks 7 and 8, while they decreased for the last 2 weeks.

\subsubsection*{Conversation Patterns}
Figure~\ref{fig:conversationHeatmap} shows the weekly trends of conversation duration across different groups. This highlights the subtle variations in social interactions. Group 1 continuously shows shorter conversation durations, whereas Group 2's reaches its highest point around week 6 and again in the final weeks. The conversation durations of Group 3 gradually increased. Group 4 has a less consistent pattern, which shows fluctuations in their social interactions.

\subsubsection*{Phone Usage Patterns}
The analysis of phone usage duration patterns in Figure~\ref{fig:phoneusageHeatmap} highlights Group 1 as the most engaged phone usage throughout the study, while Group 3 consistently shows lower engagement levels. Group 2's phone usage fluctuates between moderate and low, while Group 4's shows a low trend.

\subsubsection*{Location Transition Patterns}
Figure~\ref{fig:locationHeatmap} shows the weekly patterns for average location transitions for 4 groups. The number of location transitions represents the number of movements between different places. Group 1 has the lowest location changes as compared to other groups. Group 2 shows a moderate to high trend, while Groups 3 and 4 have a low trend.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Loneliness Detection Using all Students Data}
This section presents the performance of four binary classification models, specifically Logistic Regression, Random Forest, Support Vector Machine (SVM), and XGBoost, which are trained on all the data at once. This performance of the generalized models for the \textquotesingle lonely\textquotesingle \space class is summarized in Table~\ref{tab:centralised}. 

\begin{table}[ht]
\centering
\caption{Performance of centralised machine learning models classifying loneliness using a subset from the StudentLife dataset.}
\label{tab:centralised}
\begin{tabular}{|l|c|c|c|c|}
\hline
\textbf{Model} & \textbf{Accuracy,\%} & \textbf{Precision,\%} & \textbf{Sensitivity,\%} & \textbf{F1 Score,\%} \\
\hline
XGBoost & 81.11 & 91.48 & 86.07 & 88.50 \\
\hline
RF & 82.32 & 86.07 & 82.36 & 84.14 \\
\hline
SVM & 76.50 & 75.83 & 82.07 & 79.16 \\
\hline
LR & 72.04 & 67.16 & 78.47 & 72.35 \\
\hline
\end{tabular}
\end{table}

We can see that ensemble-based classifiers (XGBoost and Random Forest) performed overall well as compared to SVM and LR. Among these generic models, XGBoost achieved the highest overall performance (except accuracy) with an accuracy of 81.11\%, a precision of 91.48\%, a sensitivity of 86.07\%, and an F1 score of 88.50\%. This shows that XGBoost was particularly effective at correctly predicting loneliness when it identified it (high precision) while also successfully detecting a large proportion of all actual loneliness cases in the dataset (high sensitivity). The Random Forest models followed closely in performance, with the highest accuracy of 82.32\%. However, precision, sensitivity, and F1 scores were a bit lower, at 86.07\%, 82.36\%, and 84.14\% respectively. SVMs and logistic regression performed less effectively, with accuracies of 76.50\% and 72.04\%, respectively.

\subsection{Group Based Loneliness Detection}

To directly assess the impact of our personalized approach, we trained loneliness classification models specifically for each behavioural group identified through our clustering process. Over the 10-week period, we applied incremental clustering weekly to assign each student's data points to one of the evolving behavioural groups. Instead of assigning a single group to each student for the entire period, each data point was independently assigned to a group based on that week's clustering results. For each of the four behavioural groups identified, a distinct binary classification model was trained. Each model utilized all data points assigned to its respective group across the 10 weeks. Since a student's data points could belong to different groups in different weeks based on clustering results, their data contributed to multiple group-specific models at final week 10. This approach helps in accounting for behavioural fluidity to ensure that the models reflect the diversity of experiences across the student population.

Figure \ref{fig:group_behavior_example} shows sample training and test records for two behavioural groups identified through weekly clustering. For example, we can see that Student A, who has a consistent positive (lonely) label, appears in G1 in Week 3 alongside Student C, who also has a positive label, but switches to G2 in Week 4, where they are clustered with Student D, who has a negative (not lonely) label. For training and testing each group-specific model, we applied 10-fold cross-validation within each group.

\begin{figure*}[htp] \centering{
\includegraphics[width=10.8cm, height=6.0cm]{figures/Ch4/groups_example_table.pdf}
\caption{Sample Data Records For Two Different Groups}
\label{fig:group_behavior_example}}
\end{figure*}

There are a few limitations associated with this approach of using a single post-study loneliness label for all weekly records. Since the dataset only provides one loneliness score per student collected at the end of the study, we had to apply this final score as a label to all records for that student each week. This limitation restricts the model's ability to capture short-term changes in loneliness that might occur week-to-week. Ideally, if we had weekly loneliness responses, they would provide a more granular and accurate view of the fluctuations in loneliness over time and we could label each week's records based on weekly self-reports. Moreover, using weekly data with a final, static loneliness label may have introduced noise, especially in weeks where a student's behaviour differs from the overall loneliness state indicated by the end-of-study score. This discrepancy between weekly behaviour and final labels can potentially confuse the model leading to lower predictive performance. To address these limitations in future work, we have collected weekly loneliness survey responses in our own collected new dataset. By obtaining weekly labels, we aim to better capture the nuances and temporal dynamics of loneliness, which will help create more accurate and contextually relevant models that reflect the weekly experiences of participants.

Table~\ref{tab:groupbasedmodels} presents the performance metrics of these group-based models. For the majority of the groups (Groups 1, 2, and 3), the detection models show significantly improved performance compared to the generic models trained on the entire dataset. Across most groups, XGBoost emerges as the top-performing algorithm. In G1, XGBoost achieved an accuracy of 88.5\%, precision of 94\%, sensitivity of 91.5\%, and an F1 score of 92.5\%. Its performance slightly improved in G2, with an accuracy of 89.5\%, precision of 95.5\%, sensitivity of 90.5\%, and an F1 score of 93.7\%. For G3, XGBoost maintained strong performance with an accuracy of 86.5\%, precision of 94.3\%, sensitivity of 88\%, and an F1 score of 92.3\%. Random Forest ranked second in performance across G1, G2, and G3, with F1 scores of 89.4\%, 89.3\%, and 87\% respectively. SVM showed moderate performance, with F1 scores ranging from 83.4\% to 86\% across these groups. Logistic regression performed poorly as compared to other algorithms. All models showed a significant drop in performance for G4. XGBoost, while still outperforming other models, reduced its accuracy to 61.5\% and its F1 score to 48.5\%. The other models also showed declines in performance with F1 scores ranging from 42.5\% to 45\% for G4.

\begin{table*} [ht]
  \centering
  \caption{Performance of behavioural group based machine learning models classifying loneliness using a subset from the StudentLife dataset.}
  \label{tab:groupbasedmodels}
  \resizebox{\textwidth}{!}{%  
    \begin{tabular}{cccccccccccccccccccc} %{\columnwidth}{cccccc}
\textbf{Algo} & \multicolumn{4}{c}{\textbf{G1}} &  & \multicolumn{4}{c}{\textbf{G2}} &  &  \multicolumn{4}{c}{\textbf{G3}} &  &  \multicolumn{4}{c}{\textbf{G4}}  \\ 
\cline{2-5} \cline{7-10} \cline{12-15} \cline{17-20}
        &  \textbf{Acc.}   & \textbf{Prec.}  & \textbf{Sens.} & \textbf{F1.}                                             &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}   
 &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}
 &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}
 \\    \hline
 \textbf{XGBoost}   &  88.5   & 94   & 91.5  & 92.5                                 &
    & 89.5   & 95.5 & 90.5 & 93.7 & 
    & 86.5   & 94.3 & 88.0 & 92.3 &
    & 61.5   & 55.5 & 44.2 & 48.5        \\ 
\textbf{RF}   &  87.1   & 92.0   & 87.4  & 89.4                                 &
    & 89.0   & 93.4 & 85.5 & 89.3 & 
    & 85.5   & 92.4 & 82.1 & 87.0 &
    & 55.4   & 48.5 & 41.3 & 45.0        \\     
 \textbf{SVM}   &  81.4   & 84.5   & 88.0  & 86.0                                 &
    & 78.5   & 82.5 & 86.3 & 84.4 & 
    & 82.3   & 80.5 & 85.0 & 83.4 &
    & 40.4   & 46.5 & 42.3 & 44.5        \\   
    \textbf{LR}   &  74.5   & 70.3   & 83.7  & 76.4                                 &
    & 76.3   & 72.6 & 79.4 & 76.0 & 
    & 77.0   & 73.4 & 80.0 & 76.7 &
    & 41.5   & 40.0 & 44.2 & 42.5        \\  
 
      \bottomrule
    \end{tabular}%
  }
\begin{tabbing}
Algo:Binary Classification Algorithm; G1: Group 1; G2:Group 2; G3:Group 3 \\ G4:Group 4; Acc:Accuracy(\%); Prec:Precision(\%); Sens:Sensitivity(\%); F1:F1 Score(\%) \\ RF:Random Forest; SVM:Support Vector Machine; LR:Logistic Regression
\end{tabbing}
\end{table*} 

\subsection{Aggregated Performance of Personalized Models}
To assess the overall effectiveness of group-based approach for loneliness detection compared to the generic models, we calculated the aggregated performance metrics for the personalized models across all groups. We used a weighted average based on the number of data points in each group to ensure that the aggregated metrics accurately reflect the contributions of each group. Table~\ref{tab:aggregated_metricsss} presents the performance metrics for aggregated personalised models.

\begin{table}[ht]
\centering
\caption{Aggregated Performance Metrics of Personalized Models Across All Groups}
\label{tab:aggregated_metricsss}
\begin{tabular}{|l|c|c|c|c|}
\hline
\textbf{Model} & \textbf{Accuracy,\%} & \textbf{Precision,\%} & \textbf{Sensitivity,\%} & \textbf{F1 Score,\%} \\
\hline
XGBoost & 85.72 & 90.83 & 88.69 & 89.78 \\
\hline
RF & 84.33 & 88.33 & 82.96 & 85.58 \\
\hline
SVM & 76.62 & 79.13 & 82.24 & 80.77 \\
\hline
LR & 72.49 & 68.90 & 77.46 & 73.02 \\
\hline
\end{tabular}
\end{table}

The personalized XGBoost model outperformed the generic model in terms of accuracy (85.72\% vs. 81.11\%) and F1 Score (88.57\% vs. 89.78\%). Similarly, the personalized Random Forest model showed higher accuracy (84.33\% vs. 82.32\%) and F1 Score (85.58\% vs. 84.14\%), while sensitivity was almost similar. The performance for SVM and logistic regression was almost same.

\section{Discussion}

\subsection{Key Findings}
This chapter investigated the potential of using unsupervised learning to create a personalized approach for student loneliness detection. The clustering approach effectively identified various behavioural groups among the student population. This shows that loneliness is not a uniform experience and that it is subjective in nature. Students show diverse behavioural patterns that may indicate different underlying causes or expressions of loneliness. This finding aligns with research highlighting the subjective nature of loneliness and its varying manifestations across individuals \cite{qualter2015loneliness, cacioppo2008loneliness}. Furthermore, the personalized detection models achieved significantly improved performance compared to generic models trained on the entire dataset. This finding highlights the limitations of the one-size-fits-all approach. Generic models may struggle to capture the nuances of how loneliness manifests within specific behavioural subgroups. By tailoring detection models to the unique patterns of each group, we were able to achieve more accurate identification of students experiencing loneliness. Our dynamic temporal analysis shows the potential for tracking changes in student behaviour and group participants over time. This finding suggests that loneliness may not be a static state and students' behavioural patterns may shift throughout the semester. These shifts could be linked to external factors like academic pressure, social connections, or changes in mood \cite{cacioppo2003social}. Identifying such transitions provides significant opportunities for early loneliness detection and timely intervention. If a student's behavioural patterns indicate a move towards a group associated with higher loneliness risk, proactive support could be provided before loneliness significantly impacts their well-being.

\subsection{Observations and Implications}
A close examination of the behavioural profiles across the identified groups shows important differences in how students' behaviours varied across groups. Groups 1 and 3 show relatively low activity levels and low location changes. This pattern aligns with existing literature on loneliness, which suggests that reduced physical engagement with the outside world and potential reliance on technology to fulfill social needs \cite{cacioppo2009perceived, primack2017social}. Reduced physical activity could indicate in-person social withdrawal which is reported as a common symptom of loneliness in literature \cite{cacioppo2009perceived}. Furthermore, the high phone usage in G1 might indicate a substitution of digital interactions for in-person social engagement which is a phenomenon observed in previous studies linking loneliness to increased technology use \cite{primack2017social}. However, it is important to acknowledge that the relationship between technology use and loneliness is complex, with technology potentially serving as both a tool for social connectivity and a contributor to isolation depending on the quality and nature of the interactions \cite{nowland2018loneliness}. 

In contrast, Group 2 shows a significantly different profile, with high activity levels, frequent conversations, and moderate phone usage. This pattern shows a moderate degree of physical and social engagement, likely reflecting an active social life with diverse interactions. Students within this group may be less vulnerable to experiencing loneliness, as individuals with diverse and meaningful social connections tend to have better emotional regulation and coping mechanisms, reducing their susceptibility to loneliness \cite{masi2011meta}. However, it is important to acknowledge that these are speculative interpretations, and further research is needed to establish a definitive causal relationship between these behavioural patterns and loneliness. It is possible that other factors not captured in the dataset could influence both behaviour and loneliness experiences.

The emergence of a new group during Week 7 is an interesting observation that shows the importance of dynamic analysis. Existing literature emphasizes the fluctuating nature of loneliness and its susceptibility to various influences over time \cite{vanhalst2013low}. Several factors are likely to have had an influence on this finding. Expanding the sample by including more weeks of data might have uncovered a wider and more nuanced range of behavioural patterns. The increased complexity probably caused the model to identify a distinct group that was not as evident in the first set of data. Additionally, it is possible that underlying shifts in student behaviours occurred around Week 7. These shifts could be due to many reasons like evolving social dynamics, changes in academic workload, or other external events. This aligns with research highlighting the impact of life events and social dynamics impact on evolving loneliness experiences \cite{cacioppo2003social}. These shifts could have created newly identifiable differences between student subgroups. Furthermore, the model itself have evolved over time, becoming more sensitive to subtle behavioural variations and better able to distinguish distinct groups within the dataset.

The ML model results have shown the superior performance of ensemble methods (XGBoost and Random Forest) over traditional methods like SVM and Logistic Regression across all behavioural groups. This performance difference is likely due to the inherent characteristics of ensemble methods that make them well-suited for complex, non-linear relationships which are often present in behavioural data. While our personalized detection models performed well for most groups, the results for Group 4 raise important concerns. This group shows the lowest classification performance for loneliness detection which highlights challenges of effectively classifying students within this subgroup. The relatively small data size within Group 4 and due to its late emergence (week 8) may have made it challenging for the model to learn the subtle patterns associated with loneliness for this specific group. This performance drop in Group 4 affected the aggregated metrics as well and highlighted limitations in our approach for smaller groups. This indicates that while personalization enhances model performance for most groups, it may not be as effective for groups with limited data. However, this needs further investigation to find the underlying reasons for poor performance. This shows the complexity of loneliness detection and the importance of considering group heterogeneity when developing and evaluating predictive models. Simply relying on generic models trained on the entire dataset may fail to capture these nuances and could lead to less accurate predictions and missed opportunities for timely intervention.

However, one of the important factors to consider is that the absence of leave-one-subject-out cross-validation (LOSO) may have influenced these results. Without LOSO, the models may be more prone to overfitting to the specific individuals present in the training set. This could explain why the models perform better within smaller groups, where behavioural patterns are more homogeneous and fewer students are present. The use of LOSO could help assess the generalizability of these models more effectively by ensuring that each student is tested in isolation from the training set.

\section{Conclusions and Limitations}
This chapter investigated the potential of using unsupervised learning to create a personalized approach for student loneliness detection and the findings directly address the key research questions:

\begin{itemize}
  \item Our results show that clustering methods can identify distinct behavioural groups among college students using passive sensing data. Moreover, the temporal analysis showed how these groups evolve over time. This highlights the fluid nature of student behaviour and the importance of longitudinal tracking.
  \item The superior performance of personalized, group-based loneliness classification models compared to generic models trained on the entire dataset shows the value of our approach. This finding empirically validates the hypothesis that accounting for inter-subject variability through behavioural grouping can lead to more accurate and effective loneliness detection. By identifying subgroups with similar behavioural patterns, we can develop more targeted and personalized detection models that better reflect the unique ways loneliness is expressed in different groups.
\end{itemize}

Even though these findings show improved performance for group-based models for loneliness detection, there are some limitations that need to be addressed. These limitations may lead to new research questions for exploration as mentioned in future work (see Chapter 8 for details).

%Lack of statistical tests

\begin{itemize}
  \item The study is based on a specific cohort of 41 college students which limits the generalizability of the findings to other populations or age groups. College students often share similar daily routines and other psychological pressures, which reduces the generalizability of our results.
  \item Another important limitation of our study is in the reliance on the single StudentLife dataset. While this dataset provides a good amount of behavioural data, it does not capture the full spectrum of factors that can influence an individual's experience of loneliness. Other important aspects, such as demographics data, family history, personal relationships, major life events, and mental health history are not included in this data. These factors play a significant role in shaping how loneliness manifests and could interact with behavioural patterns \cite{bell1988loneliness}.
  \item While our group-based models show improved performance, future work should aim to include leave-one-subject-out cross-validation (LOSO). Applying LOSO will provide a more rigorous assessment of the model’s ability to generalize to new individuals by isolating participants from the training set during evaluation.
  \end{itemize}