\chapter{Dynamic and Personalized Loneliness Detection Using Passive Sensing}
\markboth{5. Dynamic and Personalized Loneliness Detection}{}
\section{Overview}
Chapter 4 presented the potential of personalized, dynamic behavioural analysis for improved loneliness detection within student populations. We have effectively identified distinct behavioural subgroups showing that loneliness can manifest in diverse ways among students. Furthermore, tailored models outperformed generic models trained on the entire dataset. These results emphasize the need for a more personalized approach to loneliness detection.

However, there were a few limitations with that work, including a small sample size, reliance on a single dataset, and a more static approach for detecting loneliness. These limitations highlighted the need for a larger dataset and a refined methodology to accommodate evolving behavioural patterns within the data better to improve the robustness of our approach. In this chapter, we aim to address these limitations and improve our methodology by using the GLOBEM dataset. The novelty of our work lies in adding incremental clustering with incremental classification on a weekly basis. This approach would help our models to evolve dynamically as new data becomes available each week, which will refine behavioural groups and loneliness detection over time. This adaptability is important for accurately capturing the fluid nature of student experiences and responding to loneliness early. We have also performed a temporal group analysis.

Our approach includes two primary components: incremental clustering and classification. The clustering part aims to identify subgroups of students with similar behavioural patterns. This process adds new weekly data in batches that refine the behavioural groups over time. The incremental classification component detects lonely individuals within these evolving student groups on a weekly basis. We maintain a main generic classification model trained on all available data, which is regularly updated on a weekly basis. Additionally, when new distinct behavioural patterns emerge in the form of a cluster, we create a specialized classification model for that specific group.


%Our dynamic and incremental approach addresses a critical gap in existing loneliness detection methods, which often rely on static models that cannot adapt to the natural fluctuations in student behaviour over time. Our approach evolves by integrating new data to refine and update its understanding of student behaviours and loneliness indicators. This adaptability is crucial in capturing the transient and fluctuating nature of student lifestyles. The integration of behavioural grouping and dynamic loneliness detection within these groups presents a substantial methodological improvement in loneliness detection.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Methodology}

\subsection{Evolving Student Behavioural Profiles via Incremental Clustering}

We used the GLOBEM dataset for this study. To find behavioural groups of students on a weekly basis we used the incremental clustering approach with IDBSCAN as we did in Chapter 4. Unlike traditional clustering, which analyzes a static dataset, incremental clustering continuously adapts to new data without retraining the entire model. The incremental clustering process (illustrated in Figure~\ref{fig:clustering}) begins with evaluating the new data against the existing clusters. It assesses whether the data closely aligns with the established behavioural pattern of a group (within the specified distance threshold eps) and if the group's density (number of points within eps) meets the minimum requirement (minPts). If the data fits with some existing group, it gets merged in it while increasing the cluster size. In case new data doesn't match with some existing cluster, it may indicate some new emerging behavioural trends or temporary fluctuations within the student population. When multiple data points show a distinct, consistent pattern, they form a new cluster. To determine the optimal values for eps (epsilon) and MinPts (minimum points), we used GridSearchCV combined with cross-validation and silhouette score as the evaluation metric. A set of candidate parameter values was initially selected based on experiments, as we did in Chapter 4 as well. The grid search explored these values for the eps and MinPts by evaluating each combination using the silhouette score as the performance metric. The best parameter values were chosen that maximized the average silhouette score across the cross-validation folds.

\subsection{Dynamic Loneliness Detection through Incremental Classification}
In our analysis of the StudentLife dataset in Chapter 4, we found the dynamic and evolving nature of students' behaviours. As discussed in Chapter 4, we implemented incremental clustering on a weekly basis to effectively monitor these behavioural changes in students and identified behavioural clusters that showed shifts in students' behaviours over time. Although we could track the formation and evolution of behavioural groups by using this approach but there was no timely loneliness detection within these groups. We were training classification models at the end of the study after week 10. To address this limitation, we are using the concept of incremental classification in this chapter, in which we can detect loneliness dynamically on an early basis based on the changes in students' behaviours over time. This approach can lead to a responsive detection that would detect loneliness in a timely and dynamic way.

Starting from the initial stage, we create a generic classification model trained on the initial data from week 1. This generic model is then updated incrementally with each week's incoming data from all students that allows it to adapt to general trends and evolving behaviours over time. As we presented in Chapter 4, a single generic model may struggle to capture the unique behavioural patterns associated with loneliness. To address this, we develop \textquotesingle specialized\textquotesingle \space classification models tailored to each group identified through clustering which were trained exclusively on data from students within that subgroup. Fig.\ref{fig:classification} shows the process of refining the data that leads to the generic model and then to models G1, G2, and G3 which are each specifically made for a different group.

\begin{figure}[H]
  \centering
  % Left figure (Clustering)
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch5/IncrementalCLustering.pdf}
    \caption{Visualization of Dynamic Data Clustering: Segregation into Noise, Existing, and Emergent Clusters}
    \label{fig:clustering}
  \end{subfigure}
  \hfill % Spacing between the figures
  % Right figure (Classification)
  \begin{subfigure}[b]{0.45\textwidth}
    \includegraphics[width=0.9\textwidth]{figures/Ch5/classification.pdf}
    \caption{Personalized Loneliness Detection Using Behavioural Groups}
    \label{fig:classification}
  \end{subfigure}
    \caption{Clustering and Personalized Classification}
\end{figure}

In our study, we used incremental classification for four ML algorithms: logistic regression, support vector machines (SVM), random forest, and XGBoost. For logistic regression and SVM, we used the Stochastic Gradient Descent (SGDClassifier) from scikit-learn due to its computational efficiency and it supports partial fitting for incremental learning \cite{datacamp2023incremental,wu2016introduction}. The SGDClassifier also includes L2 regularization by default which helps prevent overfitting by penalizing large coefficients. For logistic regression, we used log loss, while for SVM, we used the hinge loss function. For random forest, we used an Adaptive Random Forest (ARF) classifier from the river library \cite{datt2022incremental}. ARF is designed for incremental learning and incorporates drift detection mechanisms to identify and respond to changes in data patterns. For XGBoost, we used an incremental learning approach where the model was initially trained on the initial data, and then updated weekly with new data by using the train function with the previous model passed through the $xgbmodel$ parameter \cite{vichaar2023incremental}.


We implemented a weekly iterative approach to evaluate and refine our models over time. Each week starts with incremental clustering on the week's data to identify and update behavioural groups within the student population. These groups then lead to the training and updating of two types of classification models. First one is a single generic model trained on data from all students in that week which is trained to capture overall trends in data. The second are specialized models trained exclusively on data from students within each identified behavioural group. We applied the same data processing steps as we performed in Chapter 4. This included feature scaling, principal component analysis (PCA) for dimensionality reduction, handling of missing values and feature selection for classification models.

We start by stratified splitting the data into 95\% for training/testing and 5\% for validation. For the first six weeks, this validation set was accumulated for later hyperparameter tuning. In Week 1, we established initial models through stratified nested cross-validation with hyperparameter tuning using GridSearchCV. To address class imbalance, we used SMOTE within each cross-validation fold. For group-specific models, this process was repeated on the data specific to each cluster.

From Weeks 2 to 5, we continued incremental clustering on the data each week to update existing groups. Similarly, both generic model and group-specific models were updated incrementally with the new week's data. In parallel, the group-specific models were trained for any newly identified clusters. We evaluated generic models using 10-fold while group-specific models using 5-fold cross-validation. As we explained in Chapter 4, we have also not applied LOSO CV in this chapter due to the small sample sizes in each week's groups.

In Week 6, due to observed performance drops, we re-tuned the hyperparameters for all models using the accumulated validation data (30\%) and nested cross-validation. This adjustment ensured that the new hyperparameters generalized well across weeks.

From Weeks 7 to 10, we continued the weekly update and evaluation process for all models. In the final week, we conducted a comprehensive analysis of the performance trends across the entire 10-week study period which we will present in the results section. We evaluated models using the average performance metrics from the cross-validation process. The key performance metrics included accuracy, precision, sensitivity, and F1-score.


%%%%%%%%%%%%%

%Before evaluation, we assigned groups using the clustering model trained on that week's training data. 


\section{Results}

\subsection{Temporal Evolution of Student Groups and Behavioural Profiles}
Fig. \ref{fig:dynamics} shows the dynamic shifts in student behavioural groups over a 10-week study period. This figure shows the temporal evolution of these groups with each group denoted by a unique identifier (G1, G2, G3, G4). The y-axis represents the number of students in each group, while the x-axis represents each week. To track group membership changes over time, each student was assigned to the cluster with the highest count of their data points within a week, which classified them into the group where their behaviour most frequently aligned. 

Over the ten weeks, students switched between groups as their behavioural patterns changed. This means that group membership was not fixed, and students could shift from one group to another, which reflects their evolving behaviours. For example, a student who showed highly social and active behaviours in the early weeks may belong to G2, but if their behaviour becomes more isolated and less active, they may move to G3 or G4 in later weeks. These transitions capture the fluid nature of student behaviour, showing how their engagement levels and social interactions vary over time, which is essential for understanding how loneliness manifests differently across individuals based on their own subjective experiences.

\begin{figure}[H]
  \centering
  \includegraphics[width=0.9\textwidth]{figures/Ch5/Sankey_AISoLa.png}
  \caption{Visualization of Students Temporal Behavioural Group Dynamics over a 10-Week Period (G1:W1:84 denotes Group 1 in Week 1 with 84 students)}
  \label{fig:dynamics}
\end{figure}

\subsubsection{Behavioural Profiles}
We categorized the weekly behavioural patterns of students within each group using globally normalized feature values using min-max scaling across the entire 10-week period. For each feature, values below the 25th percentile were categorized as \textquotesingle Low\textquotesingle \space values between the 25th and 75th percentiles as \textquotesingle Moderate\textquotesingle \space and values above the 75th percentile as \textquotesingle High\textquotesingle. This approach ensures consistent interpretation of categories across all weeks. This leads to the identification of potential trends in loneliness scores associated with different behavioural profiles. Table~\ref{table:behavioural_profiles_loneliness} summarizes these observations and provides detailed information on the behavioural tendencies and associated loneliness scores for each group. Behavioural patterns are summarized for the week intervals which shows the overall dominant patterns during those weeks. However, in the cases where some variations occurred, we have added symbols with the week number to show those variations. For example, the entry \textquotesingle High calls \& SMS (=W2)\textquotesingle \space for Group 1 during weeks 1-4 indicates that call and SMS activity was generally high during this period, with a moderate level specifically in week 2.


\begin{longtable}{|p{1.5cm}|p{1.2cm}|p{8.9cm}|p{1.9cm}|}
\caption{Summary of Behavioural Patterns and Associated Loneliness Scores by Group Over 10 Weeks ((↑ = High, = = Moderate, ↓ = Low))}
\label{table:behavioural_profiles_loneliness} \\ 
\hline
\textbf{Week Interval} & \textbf{Group} & \textbf{Behavioural Patterns} & \textbf{Loneliness Score Range} \\
\hline
\endfirsthead

\multicolumn{4}{c}{Table continued...}\\ 
\hline
\textbf{Week Interval} & \textbf{Group} & \textbf{Behavioural Patterns} & \textbf{Loneliness Score Range} \\
\hline 
\endhead

\hline \multicolumn{4}{r}{Continued on next page...} \\ \endfoot

\hline \endlastfoot

% Table content (using existing rows)
Weeks 1-4     & G1    & High physical activity, High calls \& SMS(=W2), High unique Bluetooth encounters, Moderate location changes, Moderate phone usage duration(↑W1), High sleep duration(=W3) & 10-18  \\
\hline
              & G2    & High physical activity(=W1), Moderate calls \& SMS, Moderate unique Bluetooth encounters(↓W2), Moderate phone usage(↓W4), Moderate sleep duration(↓W3) & 15-26       \\
\hline
              & G3    & Low physical activity, Low calls \& SMS, Low unique Bluetooth encounters, Moderate location changes(↓W1), Moderate phone usage(↑W3), Low sleep duration & 16-30   \\
\hline
Weeks 5-7     & G1    & High physical Activity(=W7), Moderate calls \& SMS, High unique Bluetooth encounters(=W5), Moderate location changes, Moderate phone usage(↓W6), High sleep duration(=W7) & 10-22   \\
\hline
              & G2    & Low physical activity, Low calls \& SMS, Moderate unique Bluetooth encounters, Moderate location changes, Moderate phone usage(↑W7), Low sleep duration(=W5) & 15-32   \\
\hline
              & G3    & Low physical activity, Low calls, Moderate unique Bluetooth encounters, Low location changes(=W7), High phone usage, Moderate Sleep duration(↑W5) & 18-36    \\
\hline
              & G4    & Moderate physical activity(↓W6), Moderate calls \& SMS, High unique Bluetooth encounters, Moderate location changes(↓W7), Moderate phone usage, Moderate sleep duration(↓W6) & 14-28    \\
\hline
Week 8        & G1    & Moderate physical activity, High calls \& SMS, High unique Bluetooth encounters, Moderate location changes, Moderate phone usage, Moderate sleep duration & 10-20        \\
\hline
              & G2    & High physical activity, Moderate calls \& SMS, Moderate unique Bluetooth encounters, Moderate phone usage, Moderate sleep duration & 15-25          \\
\hline
              & G3    & Low physical activity, Moderate calls \& SMS, Low unique Bluetooth encounters, Moderate location changes, Moderate phone usage, Low sleep duration & 18-36        \\
\hline
Weeks 9-10  & G1    & High physical activity, High calls \& SMS, High unique Bluetooth encounters, Moderate location changes, Moderate phone usage(↓W9), High sleep duration & 10-20         \\
\hline
              & G2    & Moderate physical activity(↓W9), Moderate calls \& SMS, Moderate unique Bluetooth encounters, Moderate phone usage, Moderate sleep duration & 16-32     \\
\hline
              & G3    & Low physical activity(=W10), Moderate calls \& SMS, Low unique Bluetooth encounters, Moderate location changes, High phone usage(=W10), Low sleep duration & 18-36     \\
\hline
              & G4    & Low physical activity, Moderate calls \& SMS, Moderate unique Bluetooth encounters(↓W9), Moderate location changes, Moderate phone usage, Moderate sleep duration            & 14-35     \\
\hline
\end{longtable}

\subsubsection*{Initial Phase (1-4)}
During the initial weeks, three groups emerged. G1 students showed overall high physical activity which is calculated by steps count, high number of calls/SMS, and diverse social interactions (Bluetooth encounters), which shows good engagement with a balanced, active schedule and high sleep. Their loneliness scores ranged from 10 to 18, which shows that their active lifestyle may correlate with lower feelings of loneliness. Similarly, G2 was physically active but with moderate social interaction and phone use. This points to an active lifestyle, but perhaps less socially intensive than G1. In contrast, G3 showed the lowest physical activity, limited calls/texts, and fewer Bluetooth encounters, alongside heavy phone use and shorter sleep. Their loneliness scores ranged from 16 to 30, which is higher in comparison with other 2 groups.

\subsubsection*{Middle Phase (Weeks 5-7)}
By the middle phase of the study, behavioural patterns changed a little bit. G1 maintained their patterns which shows stability within this group. However, G2 showed a decline in activity, social interaction, and sleep. This group showed a shift in score, with increased loneliness scores (15-32). G3 remained largely unchanged, with persistently low activity, short sleep, and heavy phone usage. The emergence of G4 introduced further subtlety; this group showed moderate activity, moderate communication patterns, and moderate sleep.

\subsubsection*{Final Phase (8-10)}
In the final weeks, G1 and G2 showed moderate to high physical activity and communication patterns and moderate sleep. However, G3 remained low physically active with moderate phone usage, while G4's physical activity levels decreased.

G4 disappeared at week 8 but reappeared at week 9. To determine if the G4 that emerged in Week 9 shared similar characteristics with the earlier G4 from Weeks 5-7, we conducted a feature average analysis. We calculated the mean and standard deviation for each feature in both periods. Our analysis found that the majority of feature averages from Week 9 fell within one standard deviation of their corresponding averages from Weeks 5-7.

\subsection{Loneliness Detection Using Generic Models}
Table~\ref{tab:generic_model_performance} presents the performance of four generic classification models at the end of the study, which have been trained on a weekly incremental basis. The table shows the performance of XGBoost, Random Forest, SVM, and Logistic Regression models for the lonely class. As the table shows, the XGBoost model achieved the highest accuracy (76.52\%) followed by the Random Forest with 74.64\% and then SVM with 63.50\% and then the Logistic Regression model with an accuracy of 57.32\%.

\begin{table}[H]
\centering
\caption{Performance of Generic Models for Loneliness Detection}
\label{tab:generic_model_performance}
\begin{tabular}{lcccc}
\toprule
\textbf{Model} & \textbf{Accuracy} & \textbf{Precision} & \textbf{sensitivity} & \textbf{F1 Score} \\
\midrule
XGBoost & 76.52\% & 79.00\% & 76.83\% & 77.50\% \\
Random Forest & 74.64\% & 71.50\% & 66.72\% & 69.58\% \\
SVM & 63.50\% & 65.00\% & 55.50\% & 60.12\% \\
Logistic Regression & 57.32\% & 43.50\% & 54.65\% & 48.53\% \\
\bottomrule
\end{tabular}
\end{table}

Figure~\ref{fig:all_images_classification} shows a more nuanced perspective on how these models performed over the 10-week study period. We can see that each model showed a trajectory of performance in the prediction of loneliness. Although there were initial fluctuations but the final week’s results show that the models capabilities improved gradually. The XGBoost model showed gradual improvement, starting at 63\% accuracy in week 1 and reaching 76.5\% by week 10, with corresponding increases in precision, sensitivity, and F1 score to 79\%, 76.83\%, and 77.50\%, respectively. The model performance displayed a minor dip around weeks 5 and 6, which was subsequently improved. Random Forest accuracy improved from 59\% to 74.64\%, with precision at 71.5\% and sensitivity at 66.72\%. The F1 score for this model was 69.58\%. Similar to other models, Random Forest shows weekly variations specifically around weeks 5 and 6. The SVM model’s accuracy improved from an initial 52\% to 63.5\%. The logistic regression model shows a performance increase over a 10-week period, with accuracy starting below 40\% and rising to over 57.32\%.

\begin{figure}[H]
    \centering
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/XGBoost_performance.pdf}
        \caption{XGBoost Model}
        \label{fig:image1}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/SVM_performance.pdf}
        \caption{Support Vector Machine Model}
        \label{fig:image2}
    \end{subfigure}

    \vspace{0.5cm} % Adjust spacing between rows

    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/randomforest_performance.pdf}
        \caption{Random Forest Model}
        \label{fig:image3}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/logistic_regression_performance.pdf}
        \caption{Logistic Regression Model}
        \label{fig:image4}
    \end{subfigure}
    \caption{Temporal Performance of Generic Classification Models for Loneliness Detection Across a 10-Week Period} 
    \label{fig:all_images_classification} 
\end{figure}

\subsection{Loneliness Detection Using Group Based Models}

In this chapter, we trained loneliness classification models in a weekly incremental manner meaning that models were updated and trained each week based on the data from the dynamically evolving behavioural groups. This approach provides a more real-time and adaptive detection of loneliness compared to the static and retrospective classification approach used in Chapter 4. By incorporating weekly behavioural data, the models can better reflect the current state of a student's behaviour that allows for timely detection of loneliness as students shift between groups. This weekly training ensures that the model adapts to emerging trends in behaviour and can capture short-term changes that may indicate increasing or decreasing loneliness. Instead of relying on cumulative behaviour across the entire study as in Chapter 4, the weekly approach ensures that predictions are responsive to the latest behavioural patterns which provides a more precise and personalized detection approach. This incremental method allows the model to dynamically adjust to new behaviours as they emerge which improves its ability to detect loneliness early and more accurately compared to a static model trained only once at the end of the study.

The final results from the group-based models for loneliness detection at week 10 are presented in Table~\ref{tab:tablegroupbased} and Figure~\ref{fig:all_images}. XGBoost performed best with the highest accuracy in Groups 1 and 2, with scores of 78.64\% and 80.15\%, respectively. Its precision, sensitivity, and F1 scores were also high. Group 3 show a slight drop in accuracy for XGBoost to 76.55\%. However, in Group 4, the performance was poor, with accuracy falling to 41.38\%. The Random Forest algorithm showed almost similar accuracy as of XGBoost for G1 (78.35\%) but all other performance metrics were low. For Group 4, the Random Forest algorithm outperformed XGBoost, achieving a higher accuracy of 62.42\% and an F1 score of 64\%. The SVM model's performance was relatively lower, with the highest accuracy of 66.43\% for G1. The logistic regression model showed the lowest performance across all groups, with accuracy scores ranging from 42.48\% to 64.29\%.

Adding error bars to the accuracy helped show that some of the drops in performance during the middle weeks may not be significant since the performance ranges overlap with nearby weeks. This means the model’s performance is likely improving as more data is added, but there are some ups and downs due to the natural variability in the data and the way the model is trained. These changes are more noticeable in the earlier weeks, where there is less data available for training.



\begin{table}[H]
  \centering
  \resizebox{\textwidth}{!}{%
  \caption{Performance of Group Based Models Classifying Loneliness at Week 10}
  \label{tab:tablegroupbased}
    \begin{tabular}{cccccccccccccccccccc}
\textbf{Algo} & \multicolumn{4}{c}{\textbf{G1}} &  & \multicolumn{4}{c}{\textbf{G2}} &  &  \multicolumn{4}{c}{\textbf{G3}} &  &  \multicolumn{4}{c}{\textbf{G4}}  \\ 
\cline{2-5} \cline{7-10} \cline{12-15} \cline{17-20}
        &  \textbf{Acc.}   & \textbf{Prec.}  & \textbf{Sens.} & \textbf{F1.}                                             &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}   
 &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}
 &             
	&  \textbf{Acc.}   & \textbf{Prec.}      & \textbf{Sens.} & \textbf{F1.}
 \\    \hline
 \textbf{XGBoost}   &  78.64   & 79.84   & 81.5  & 80.5                                 &
    & 80.15   & 78.63   & 82.44  & 80.74 & 
    & 76.55   & 78.34 & 73.74 & 75.97 &
    & 41.38   & 39.66 & 44.38 & 42.10        \\ 
\textbf{RF}   &  78.35   & 77.74   & 74.44  & 75.73                                 &
    & 79.00   & 80.45 & 78.55 & 79.34 & 
    & 75.53   & 78.43 & 80.14 & 79.55 &
    & 62.42   & 63.40 & 65.33 & 64.00        \\     
 \textbf{SVM}   &  66.43   & 71.52   & 62.45  & 66.38                                 &
    & 65.49   & 72.55 & 63.32 & 68.43 & 
    & 61.31   & 69.52 & 63.33 & 66.18 &
    & 53.41   & 55.53 & 50.29 & 52.52        \\  
    \textbf{LR}   &  64.29   & 54.32   & 63.71  & 58.45                                 &
    & 58.29   & 55.62 & 58.41 & 56.29 & 
    & 52.00   & 44.42 & 55.29 & 48.73 &
    & 42.48   & 39.84 & 44.23 & 41.96        \\  
 
      \bottomrule
    \end{tabular}
    }
\begin{tabbing}
Algo:Binary Classification Algorithm; G1: Group 1; G2:Group 2; G3:Group 3 \\ G4:Group 4; Acc:Accuracy(\%); Prec:Precision(\%); Sens:sensitivity(\%)  F1:F1 Score(\%) \\ RF:Random Forest; SVM:Support Vector Machine; LR:Logistic Regression
\end{tabbing}
\end{table} 

\begin{figure}[H]
    \centering
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/xgboost_group_based.pdf}
        \caption{XGBoost Model}
        \label{fig:image1}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/svm_group_based.pdf}
        \caption{Support Vector Machine Model}
        \label{fig:image2}
    \end{subfigure}

    \vspace{0.5cm} % Adjust spacing between rows

    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/randomforest_group_based.pdf}
        \caption{Random Forest Model}
        \label{fig:image3}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.45\textwidth}
        \includegraphics[width=\linewidth]{figures/Ch5/logisticregression_group_based.pdf}
        \caption{Logistic Regression Model}
        \label{fig:image4}
    \end{subfigure}
    \caption{Performance of the Group based Classification Models for Loneliness Detection at Week 10} 
    \label{fig:all_images} 
\end{figure}

\subsection{Aggregated Performance of Personalized Models}
To assess the overall effectiveness of group-based approach for loneliness detection compared to the generic models, we calculated the aggregated performance metrics for the personalized models as we did in chapter 4 as well. We used a weighted average based on the number of data points in each group to ensure that the aggregated metrics accurately reflect the contributions of each group. Table~\ref{tab:aggregated_metrics} presents the performance metrics for aggregated personalised models.

\begin{table}[ht]
\centering
\caption{Aggregated Performance Metrics of Personalized Models Across All Groups}
\label{tab:aggregated_metrics}
\begin{tabular}{|l|c|c|c|c|}
\hline
\textbf{Model} & \textbf{Accuracy,\%} & \textbf{Precision,\%} & \textbf{Sensitivity,\%} & \textbf{F1 Score,\%} \\
\hline
XGBoost & 76.10\% & 76.25\% & 77.79\% & 77.04\% \\
\hline
RF & 76.92\% & 77.74\% & 76.09\% & 76.72\% \\
\hline
SVM & 64.28\% & 70.38\% & 62.04\% & 65.95\% \\
\hline
LR & 58.62\% & 52.05\% & 59.08\% & 55.34\% \\
\hline
\end{tabular}
\end{table}

The aggregated personalized XGBoost model had a slightly lower accuracy compared to the generic model by 0.42\% (76.10\% vs. 76.52\%). Precision decreased by 2.75\% (76.25\% vs. 79.00\%), but sensitivity increased slightly (77.79\% vs. 76.83\%). The F1 score showed a minor decrease (77.04\% vs. 77.50\%). The aggregated personalized RF model improved accuracy by 2.28\% (76.92\% vs. 74.64\%), precision by 6.24\% (77.74\% vs. 71.50\%), sensitivity by 9.37\% (76.09\% vs. 66.72\%), and F1 score by 7.14\% (76.72\% vs. 69.58\%). The personalized SVM and Logistic Regression models both showed minor improvements in accuracy, precision, sensitivity, and F1 score compared to the generic models.

\section{Discussion}

\subsection{Temporal Dynamics of Loneliness}
While Chapter 4 presented about the dynamic nature of student behaviour and loneliness, the larger dataset in this chapter helped us to investigate a more nuanced and comprehensive understanding of these temporal patterns and how they relate to loneliness levels. Over the study period, we have seen shifts in group composition and behaviours of students in each group. This evolution highlights the importance of a dynamic approach to monitor loneliness, which may provide indicators of the temporal aspects of student lifestyles and their mental well-being as we discussed in Chapter 4 as well. Although the temporal analysis in Chapter 4 suggested the possibility of external factors affecting changes in behaviour, the longer time period analyzed in this chapter provided a more detailed investigation of these patterns. We observed periodic fluctuations in group composition and behaviour that might align with typical academic cycles. These changes could potentially be associated with periods of increased academic stress, such as those that might occur during midterms or final exams. However, further studies are needed with contextual data to explore the causal relationships between these external factors and behavioural changes to understand why these patterns emerge definitively. Monitoring these changing patterns can helps us understand how stress, social interactions, and personal circumstances affect student behaviour.

Group 1 maintained consistently high physical activity and social interactions with some fluctuations throughout the study which might align with their relatively stable and low loneliness scores (10-22). This shows that a balanced lifestyle potentially contributes to lowering feelings of loneliness. However, Group 2 shows fluctuations in behaviour with decreased activity and social interactions during weeks 5-7, with increased phone usage. This pattern may indicate that academic or personal stressors negatively impact social engagement and well-being, although further research is needed to find the actual reasons. The emergence of Group 4 in week 5 showed moderate behaviours and varied loneliness scores (14-35). This shows the diverse ways students adapt to changing academic and social demands. Group 3 consistently showed low physical activity, high phone usage, and higher loneliness scores (16-36), which aligns with previous research linking reduced physical engagement and increased technology use to social withdrawal and loneliness \cite{hawkley2009loneliness,nowland2018loneliness}. The behavioural shifts in weeks 9-10, specially for Groups 2 and 3 could be due to end-of-semester pressures, which might have impacted social behaviours and loneliness levels.

\subsection{Effectiveness of Personalized Models}

The results from this chapter further highlight the effectiveness of personalized models in loneliness detection among students. The performance of group specific models over generic models, previously observed in the smaller dataset has again been validated in this chapter. Personalized models excel in performance due to their ability to detect nuanced behavioural patterns that are often masked in generic models trained on the whole dataset. Tree-based models outperformed other algorithms, likely due to the fact that they work better to handle non-linear relationships and complex interactions between features. Their ensemble nature helps them to capture a wider range of patterns in the data that potentially lead to improved predictive performance which we can see in the results. The underperformance of all models in Group 4 shows some difficulties in predicting loneliness for this group. This group might have more complex behaviour patterns that are harder for the model to learn effectively related to lonely class. There could also be data quality issues specific to this group which results in poor model performance. Despite the marginally lower performance of the aggregated personalized XGBoost model compared to its generic counterpart, the overall findings support our hypothesis that personalized models are more effective, especially when we analyze them individually for each group. However, it is important to note that the improved results, compared to the generalized approach, may be influenced by the use of traditional 10-fold cross-validation rather than leave-one-subject-out cross-validation (LOSO), which could have provided a more robust evaluation.

The choice between generic and personalized models involves a trade-off. Generic models might be less precise for individual groups, but they can be used in a broader applicability. They can be more suitable when resources are limited or where a rapid, general assessment is needed. On the other hand, personalized models work well in situations where targeted interventions are required and a deeper understanding of individual differences is important. For example, in a university setting, a generic model could be used to identify students at risk of loneliness broadly, while group-specific models could be used for designing tailored interventions for specific student populations. We will be discussing this tradeoff between personalisation and generalisation in detail in Chapter 8 (section 8.2.1).

\subsection{Comparison with Chapter 4 Findings}

The comparison of loneliness detection models between Chapter 4 (StudentLife dataset, 41 students) and Chapter 5 (GLOBEM dataset, 205 students) provides important observations about the challenges and complexities of scaling up passive sensing based loneliness detection systems.

We observe a significant decrease in performance across all metrics for generic models when moving from the smaller StudentLife dataset to the larger GLOBEM dataset. For instance, the XGBoost model's F1 score dropped from 88.50\% to 77.50\%, with similar declines in accuracy, precision, and sensitivity scores. This trend is consistent across all algorithms with more significant decreases for simpler models like SVM and logistic regression. This decline indicates that increasing sample size introduces more complexity in loneliness detection. Although this increase in sample size from 41 to 205 students provided more data for models to train but this may have introduced greater variability in behavioural patterns associated with loneliness. This increased complexity could have made it more challenging for models to learn the specific patterns associated with \textquotesingle lonely\textquotesingle \space class and generalize effectively. Secondly, the incremental training approach used in this chapter, where models were updated weekly with new data could also have contributed to reduced performance. Although this approach is more reflective of real-world scenarios but this may struggle to capture and remember consistent patterns related to loneliness as effectively as training models with all data at once.

The group-specific models also show a similar trend of decreased performance when moving from the StudentLife to the GLOBEM dataset. However, the group-specific models consistently outperform their generic counterparts in both datasets which highlights the importance of personalized approaches in loneliness detection. The observed decline in performance with the larger GLOBEM dataset aligns with findings from a recent depression detection study using passive sensing data \cite{adler2024measuring}. Similar to our findings, this study found that ML models have lower accuracy in larger, more diverse samples. This is due to the varying meanings of behavioural indicators for depression across different demographic and socioeconomic subgroups. This phenomenon likely extends to loneliness detection as well since the underlying mechanism is the same. Just like they found different associations between phone usage and mobility patterns with depression across age and socioeconomic groups, similar behavioural indicators of loneliness may vary in their predictive power across different student subpopulations. Our group-specific models' superior performance compared to generic models across both datasets supports this theory. It indicates that loneliness just like depression manifests differently across various behavioural subgroups. The more apparent performance decline in generic models when moving to the larger GLOBEM dataset further supports this idea, as a larger sample size likely introduced greater variability in how behavioural patterns relate to loneliness. We will discuss this model generalizability aspect in more detail in Chapter 8, \textquotesingle Conclusion\textquotesingle.

\subsection{Implications for Interventions and Personalized Support}

Together, the findings from Chapter 4 and Chapter 5 present practical applications for personalized loneliness detection among students. The dynamic nature of student behaviour and loneliness presented in our study highlighted the importance of continuous monitoring and adaptive interventions. As students navigate through different academic and social challenges, their risk of loneliness and its behavioural manifestations may change. By continuously tracking these changes, universities can ensure that their support services remain relevant and effective throughout the academic year. Moreover, generic interventions often lack the specificity needed to be truly effective. Personalized detection can lead to the development of targeted support systems tailored to the unique behavioural patterns of different subgroups in a broader population. For example, interventions for groups with high phone usage and low physical activity could focus on encouraging physical engagement and face-to-face interactions. This approach can improve early interventions by continually monitoring behavioural patterns and detecting shifts associated with loneliness, which would lead to timely and personalized support for students suffering from loneliness. Another important aspect to highlight is while smartphones and wearables provide passive sensing data for monitoring and detecting loneliness, they may also contribute to increased loneliness, specifically among younger populations \cite{enez2016smartphone}. This dual implication highlights the importance of balancing the benefits of using these technologies for data collection with their potential negative impacts.

\section{Conclusions and Limitations}
This chapter has expanded upon the foundations laid in Chapter 4, addressing its limitations and further validating the efficacy of personalized, dynamic approaches to loneliness detection among students. Our main findings are:
\begin{itemize}
\item The usage of incremental clustering and classification has proven to be a key improvement in our approach. This approach allowed models to adapt dynamically to evolving behavioural patterns that provide a timely representation of student experiences. This adaptability is important in capturing the fluid nature of loneliness and its behavioural indicators which ultimately will lead to early detection and intervention.
  \item The results of this study further reinforce the findings from Chapter 4 which emphasizes the effectiveness of personalized models over generic ones in detecting loneliness.
  \item The temporal weekly analysis of students' behaviour has shown the dynamic nature of students' lives and its relation with their loneliness levels. The group fluctuations and behavioural patterns highlight the need for continuous monitoring and adaptive intervention strategies.  
  \item We observed a decrease in model performances when coming from the smaller StudentLife dataset to the larger GLOBEM dataset. This shows that increasing sample size and diversity added greater complexity to loneliness detection, emphasizing the need for further research.
\end{itemize}

In Chapter 8, we will discuss possible future work related to loneliness detection systems. Our discussion there will partially build upon the following limitations that we identify in the work of this chapter.

\begin{itemize}
\item In this chapter, we performed weekly binary classification of loneliness based on the students' behavioural data. However, it is important to clarify that the ground truth used for these classifications, derived from the UCLA Scale, was collected at the end of the 10-week study period. This means that the UCLA scores provide a cumulative measure of each student's loneliness at the end of the study, rather than reflecting their loneliness on a weekly basis. The weekly classifications rely on using this post-study score as a consistent label across all weeks. While this approach is common when longitudinal ground truth data is unavailable, it introduces a limitation about capturing precise weekly fluctuations in loneliness. Future studies should focus on collecting loneliness scale data more frequently throughout the study period, to better track temporal changes and capture a more precise measurement of loneliness over time.
\item As we highlighted in Chapter 4 as well, the dataset we are using is limited to a specific student population from a single university over a fixed period. The homogeneity of this sample may not fully capture the diverse range of experiences and behavioural manifestations of loneliness present in the broader population. 
\item Our approach worked well, but there is still room for improvement, especially in predicting loneliness within groups like G3 and G4. To address this, future work needs to explore the potential reasons for reduced performance and may use advanced methods to improve performance. Also there is a need to test LOSO CV for more robust evaluation. Moreover, ensuring such detection models are fair and robust is important. This means they should work equally well for all subgroups without any bias specific to certain populations' behaviours or demographics. More contextual data related to populations will need to be incorporated to train these models effectively. Fairness in these models will make them more reliable and also help build trust in such systems' ability to support every student effectively.

\end{itemize}

