\chapter{Methodology}

\section{Smartphone Sensors for Passive Sensing}
In today's world, smartphones have become essential tools for everyday activities. Beyond their primary function as a communication device, smartphones are equipped with several sensors. These sensors are a powerful tool for passive data collection \cite{enck2014taintdroid}. Passive sensing has a significant role in psychological and behavioural research. It enables data collection in a realistic setting, which can be used to obtain data about individuals' real-world behaviours and feelings \cite{nelson2018extending}. Each sensor on a smartphone can capture different types of data, which helps in getting unique information about user behaviour and their environment. The following are the key sensors from smartphones that have been used in this thesis.

\begin{enumerate}
  \item Location (GPS): The GPS sensor provides precise geographical coordinates. Location data can be important in understanding an individual's mobility patterns and social tendencies. Frequent time spent in isolated places or reduced mobility can indicate social withdrawal which is a common symptom of loneliness \cite{watson2012rejection}.
  \item Bluetooth: Bluetooth radio can detect the presence of other nearby Bluetooth-enabled devices. Bluetooth can help to provide information about social interactions and networks by capturing data on nearby devices. It can indicate the frequency of an individual's proximity to others which can provide information about their social engagement levels. A decrease in detected devices might indicate reduced social engagement which is again an important indicator of loneliness.
  \item WIFI: Similar to Bluetooth, WiFi radio detects WiFi networks and their strength. WiFi connectivity patterns can be used to infer an individual's location within a certain area, like home or work, and their movement patterns within these environments.
  \item Calls/SMS Logs: These logs provide data on communication patterns, including call duration, frequency, and messaging activities. Communication logs are an important source of information for finding social interaction patterns. Decreased frequency or changes in communication habits can indicate changes in social behaviour, which could be linked to feelings of loneliness or isolation.
  \item Phone Usage: This involves tracking the usage patterns of smartphones. Screen time data can provide information about daily routines, sleep patterns, and potential over-reliance on digital devices for social interaction.
  \item Conversation: This uses a microphone sensor on a smartphone to record the start and end timestamps of conversation activity. Again it's an important indicator for social communications.
\end{enumerate}

\section{Fitness Bands and Wearable Technology}
The popularity of fitness bands and wearable technology has revolutionized the field of passive sensing in monitoring physical and mental health related indicators \cite{sheikh2021wearable}. These devices are widely accessible and can provide continuous, real-time data, which makes them important tool in behavioural research. Fitness bands are equipped with several sensors that can track multiple aspects of physical health which can provide important information about the user's lifestyle, habits, and even mental well-being \cite{cornet2018systematic}. Below are the fitness band sensors that have been used in this thesis.

\begin{enumerate}
  \item Physical Activity: Fitness bands typically include accelerometers and gyroscopes to measure movement and physical activity like step count. Low daily physical activity can indicate some physical or mental health condition \cite{peluso2005physical}.
  \item Sleep: Wearables use a combination of movement detection and heart rate monitoring to track sleep patterns, including duration and quality. Sleep is a critical component of overall health and well-being. Monitoring sleep patterns can give us data about disturbances or irregularities that are often associated with various psychological conditions, including stress, anxiety, and loneliness \cite{anderson2013sleep}. Consistent sleep disturbances or significant variations in sleep patterns could be indicative of underlying mental health issues.
\end{enumerate}


\section{Datasets}
%StudentLife
%GLOBEM

This thesis mainly used two existing datasets collected through smartphones and fitness band sensors. These datasets were selected based on their availability and their relevance to addressing the key research questions. The StudentLife dataset was chosen for the initial studies because it was the only available dataset at the time that provided the necessary passive sensing and survey data related to mental health. It also has been used in existing literature as reviewed in Chapter 2. Later, the GLOBEM dataset became available, which provided more recent data and a larger participant population, making it well-suited for investigating subsequent research questions.

\subsection{StudentLife Dataset}
The StudentLife dataset is a collection of data that was passively collected from undergraduate and graduate students at Dartmouth College in 2013 \cite{wang2014studentlife}. It provides data about students' daily behaviours and mental health using different smartphone sensors, multiple clinical scales, and different ecological momentary assessment (EMA) responses. The dataset collection study received ethical approval from Dartmouth College's Institutional Review Board.

We used the first dataset from the StudentLife study, which had 48 students. The dataset was collected over the period of 10 weeks during the spring semester. From the comprehensive dataset, we selected a subset of sensors based on their relevance to loneliness detection as identified in existing literature \cite{doryab2019identifying}. These sensors include activity, location, call and SMS logs, Bluetooth, WIFI, conversation and phone usage data.

For loneliness measurement, this dataset has used the University of California Los Angeles (UCLA) scale \cite{russell1980revised}. It is a 20-item questionnaire that provides the ground truth for loneliness levels by generating a continuous score that reflects participants' self-reported loneliness. While passive sensing data provides objective information about behavioural patterns, clinical scales like the UCLA Loneliness Scale provide the real ground truth data required to label the behavioural features to train and validate machine learning models as we discussed in detail in Chapter 1. The UCLA scale includes both positive and negative items, with the total scores ranging from 20 to 80. Positive items ask about feelings of connection or belonging, while negative items focus on feelings of isolation or disconnection. For example, a positive item is "I feel like I am part of a group of friends," while a negative item is "I feel left out". Participants rate how often they experience each feeling which leads to a cumulative overall score of loneliness experience. We used the threshold of 43 as the cutoff for loneliness, based on prior studies \cite{doryab2019identifying,cacioppo2008loneliness}. Participants scoring above this threshold are considered to experience significant loneliness. Within the dataset, loneliness scores varied widely among participants, ranging from as low as 25 to as high as 64 with 12 out of 41 students falling into high loneliness category. This scale was administered at two stages: pre and post-study. Table~\ref{tab:demographics_studentlife} presents the demographic and other related information for StudentLife dataset.

\begin{table}[H]
\centering
\caption{Study Information and Participant Demographics for StudentLife Dataset}
\label{tab:demographics_studentlife}
\begin{tabular}{l|l}
\toprule
\textbf{Category} & \textbf{Data} \\
\midrule
Participants & Total: 48 \\
& Undergraduate: 30 \\
& Graduate: 18 \\
Class Year & Seniors: 8, Juniors: 14, Sophomores: 6, Freshmen: 2 \\
& Ph.D.: 3, Masters (2nd year): 1, Masters (1st year): 13 \\
Gender & Female: 10, Male: 38 \\
Ethnicity & Caucasian: 23, Asian: 23, African-American: 2 \\
Ground Truth & Pre-study: 20-item UCLA scale, Post-study: 20-item UCLA scale \\
Sensor Data & Activity, Location, Call , SMS Logs \\
& Phone Usage, Conversation, Bluetooth, WIFI \\
\bottomrule
\end{tabular}
\end{table}

\subsection{GLOBEM Dataset}
The GLOBEM dataset is a collection of passively collected data from undergraduate students over a period of 10 weeks at the University of Washington during the spring semester of 2019 \cite{xu2022globem}. This 10 week period from March to June was specifically chosen to assess any potential semester effects on the students behaviours. A total of 218 undergraduate students, with a majority being female, participated in this study. These students were recruited through email and social media invitations, and detailed demographic information about the participants is provided in Table~\ref{tab:demographicsTable}.

\begin{table}[H]
\centering
\caption{GLOBEM Dataset Participant Demographics. Gender acronyms: F: Female, M: Male, NB: Non-binary. Racial acronyms: A: Asian, B: Black or African American, H: Hispanic, N: American Indian/Alaska Native, PI: Pacific Islander, W: White, NA: Did not report. The \& symbol denotes participants who identified with multiple races.}
\label{tab:demographicsTable}
\begin{tabular}{l|l}
\toprule
\textbf{Category} & \textbf{Data} \\
\midrule
Participants & Total: 218 \\
 & Gender: F 111, M 107 \\
 & Ethnicity: A 102, B 6, H 10, N 2, PI 1, W 70, \\
 & A\&B 1, A\&W 16, H\&W 2, B\&W 2, \\
 & A\&H\&W 1, B\&H\&W 1, H\&N\&W 1, NA 3 \\
Ground Truth & Pre-study 10-items UCLA scale \\
 & Post-study 10-items UCLA scale \\
Sensor Data & Bluetooth, WIFI, Call Logs\\
    &  Location, Phone Usage, Physical Activity, Sleep  \\
\bottomrule
\end{tabular}
\end{table}

The study used a smartphone app developed using AWARE framework for data collection that was designed to run continuously in the background and collect data without needing any engagement from the users. The data collection frequency is different for different sensors and details can be seen on the AWARE official site here \cite{AWAREFrameworkSensors}. A Fitbit device was used to collect data on the sleep patterns and physical activity of the participants. The dataset includes nearby logs of Bluetooth and WIFI devices encountered, location, phone usage, calls, physical activity, and sleep data. 

The University of Washington's Institutional Review Board (IRB) granted ethical approval for this data collection. All participants in the study provided informed consent. Any information which could directly identify participants was restricted to the core data team and data has been removed for any participant who chose to withdraw from the study.

Loneliness levels among the participants were assessed using the revised 10-item UCLA loneliness scale \cite{knight1988some}. This scale required participants to rate their feelings of loneliness on a scale from 1 (\textquotesingle never\textquotesingle) to 4 (\textquotesingle always\textquotesingle), with reverse scoring applied to five of the items. The scores from these items were then aggregated which gives a total score range between 10 and 40, where higher scores indicate greater feelings of loneliness. This scale was used pre- and post-study. Since the revised 10-item UCLA scale is less commonly applied in existing studies on loneliness detection, there is no widely established standard cutoff score in the literature. After reviewing the distribution of scores and considering the characteristics of the scale, we selected a cutoff score of 20. This threshold was chosen to align with the mid-point of the scale, where scores of 20 and above would indicate moderate to high loneliness based on the participants' self-reported experiences. 113 out of 218 students fall into the lonely category.


\begin{comment}
\subsection{UCC Dataset}
We also conducted our own data collection effort from 2021 to 2023 at University College Cork, Ireland. This data collection study received ethical approval from the UCC Social Science Research Ethics Committee. The initial aim was to collect data from 30 younger (18-50 years old) and 30 older (above 65 years old) participants over a 3-week period for a pilot study, though COVID-19 and other challenges led to significant delays in completing this process. We could collect data from 52 participants, details are provided in the Table~\ref{tab:demographicsTableucc}. We used the AWARE mobile application for Android devices to collect smartphone sensor data \cite{ferreira2015aware}. We collected data for location, call and SMS logs, Bluetooth and Wi-Fi logs and phone usage. Additionally, participants were provided with a Fitbit Inspire 2 device to collect sleep, step counts, and heart rate data. Smartphone data was continuously synced with our secure server at UCC. For Fitbit data, we created a daily cron job that retrieved data from the Fitbit cloud and saved it into the same UCC database with the smartphone data.

While this dataset has not been used in the main chapters of this thesis due to the delays but it was valuable in an experiment we conducted last year which focused on the generalizability of loneliness detection models. We have discussed this in detail in Chapter 8 (Section 8.2.1).

\begin{table}[H]
\centering
\caption{UCC Dataset Participant Demographics}
\label{tab:demographicsTableucc}
\begin{tabular}{l|l}
\toprule
\textbf{Category} & \textbf{Data} \\
\midrule
\multicolumn{2}{l}{\textbf{Demographics}}\\
Total Participants & 52 \\
Age Group & Young: 30, Old: 22 \\
Gender & Female: 32, Male: 20 \\
Education & Postgraduate: 27, Undergraduate: 17, Secondary: 8 \\
Health Status & Good: 51, Other: 1 \\ 
\midrule
\multicolumn{2}{l}{\textbf{Social Characteristics}} \\
Marital Status & Single: 29, Married: 14, Separated/Divorced: 5, Widowed: 4 \\
Employment & Full-time: 18, Part-time: 5, Unemployed: 3, Retired: 19, \\
   & Undergraduate Student: 5, Postgraduate Student: 2 \\
\multicolumn{2}{l}{\textbf{Others}} \\
Ground Truth & Weekly 6-items UCLA scale \\
Sensors & Location, Call and SMS logs, Phone Usage, Bluetooth \\
& WIFI, Sleep, Step count, Heart Rate\\
\bottomrule
\end{tabular}
\end{table}
\end{comment}

\section{Behavioural Features Extraction}
The main goal of feature extraction in the context of loneliness detection using passive sensing data is to transform raw sensor data into a meaningful set of features that could be used for further analysis and for training models. We extracted behavioural features using the Reproducible Analysis Pipeline for Data Streams (RAPIDS) \cite{vega2020rapids}. It is a framework that has been developed specifically for extracting digital behavioural features through passively collected datasets. A detailed overview of these features is provided in RAPIDS documentation \cite{vega2020rapids}. For a summarized form of the extracted features, please refer to Table~\ref{tab:featuresTable}. A sample of the feature matrix schema is provided in Fig.\ref{fig:samplefeature}. Each row in the matrix represents a daily data sample for a participant, identified by a unique \textquotesingle pid\textquotesingle \space (Participant ID) and date. The columns represent various features extracted from different sensors. Since people engage in different activities throughout the day, the features were extracted for different time epochs (morning, day, evening, night) alongside the day-level (24-hour) features. Sleep related features were extracted over the entire sleep period rather than divided into epochs.

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth, height=7.5cm, keepaspectratio]{figures/Ch3/sample_feature.pdf}
\caption{The sample schema of the feature matrix (each column is a feature and each row is a daily sample per participant)}
\label{fig:samplefeature}
\end{figure}

\begin{table}[t]
\centering
\caption{Summary of Sensor-Derived Features; StudentLife (SL), GLOBEM (GB)}
\label{tab:featuresTable}
\begin{tabular}{|l|p{10cm}|}
\hline
\textbf{Category}         & \textbf{Derived Sensed-behaviours} \\ \hline
Bluetooth  (SL, GB)      & Number of scans, unique devices, mean and standard deviation of scans per device, most and least frequent device scans within segments, across segments, and across the entire dataset.
 \\ \hline
 Wi-Fi  (SL, GB)                   & Number of scans, unique devices, and scans of the most connected device measured over different time segments \\ \hline
Location  (SL, GB)                & Total distance traveled, radius of gyration, maximum diameter, maximum distance from home, significant locations visited, distance lengths and durations (mean and standard deviation), Shannon entropy, circadian routine, location variance
 \\ \hline
Phone Usage    (SL, GB)           & Total, longest, shortest, average, and standard deviation of unlock duration, number of unlock episodes, time until first unlock
 \\ \hline
Call      (SL, GB)                & Number of calls, distinct contacts, mean, sum, minimum, maximum, standard deviation, mode, and entropy of call durations, time of first and last call, calls with the most frequent contact. \\ \hline
Physical Activity  (SL, GB)       & Maximum, minimum, average, median, and standard deviation of daily steps, total steps, number, and duration (total, max, min, average, standard deviation) of sedentary and active bouts.
\\ \hline
SMS (SL) & Total number of messages, number of unique contacts, time of first and last message, number of messages from the most frequent contact.
\\ \hline
Conversation (SL) & Total, maximum, minimum, and average duration of conversations; standard deviation of conversation duration; time of first and last conversation
\\ \hline
Sleep  (GB)      & Number, total, longest, shortest, average, median, and standard deviation of awake/asleep episodes (main, nap, all), first and last wake/bedtimes, sleep efficiency, total and average duration to fall asleep, asleep, awake, in bed, and after wakeup for each sleep type.
 \\ \hline
\end{tabular}
\end{table}

\subsection{Location Features}
Location features extracted from smartphones provide information about participants' movement patterns and social behaviours, which are important indicators in loneliness detection. These features are derived from GPS data and processed to understand how participants interact with their environments. Below are some of the key features.

\begin{itemize}
  \item \textbf{Total Distance and Speed:} This computes the total distance travelled by a participant within a specific epoch. Additionally, it calculates the average speed during periods labeled as \textquotesingle Moving\textquotesingle, as well as the variance in speed, to understand the variability in their movement.
  \item \textbf{Significant Places and Transitions:} The number of significant (unique) places visited gives an insight into the participants' social habits. It identifies these locations using clustering algorithms and tracks the number of transitions between these places to capture social mobility.
  \item \textbf{Predictability About Locations:} These are features about location entropy. This uses the Shannon Entropy (measures uncertainty or randomness in data) to measure the predictability of participants' location patterns. Higher location entropy indicates more varied and less predictable movements, while lower entropy means more consistent routine and predictable location behaviours.
\end{itemize}

\subsection{Phone Usage Features}
Phone usage features extracted from passive sensing data provide data on how individuals interact with their smartphones. These features are critical for loneliness detection as they can provide information about social connectivity, mood fluctuations, and overall mental well-being. For example, frequent phone usage might indicate a search for social interaction, while irregular usage patterns could reflect emotional distress or social withdrawal \cite{lee2014mobile,gao2016smartphone}. By analyzing these features, we can better understand the relationship between phone usage behaviours and feelings of loneliness. Below are some key features.

\begin{itemize}
  \item \textbf{Total Duration of Use:} This sums up the total minutes spent during all phone unlock episodes in an epoch. A \textquotesingle phone unlock episode\textquotesingle \space means any instance when a user activates their phone screen from a locked state. This total duration helps us understand the overall time a user engages with their phone.
  \item \textbf{Unlock Episode Duration:} This looks at the longest and shortest duration of phone use in any single unlock episode. This also calculates the average duration across all episodes. These metrics provide a sense of how users interact with their phones; whether they have prolonged periods of use or just brief checks.
  \item \textbf{Variability in Usage:} The standard deviation of the duration of all unlock episodes indicates the consistency of phone use. High variability may indicate irregular usage patterns which could be associated with changes in mood or social activity.
  \item \textbf{Frequency of Use:} By counting the number of unlock episodes, this measures how frequently the user interacts with their phone. Frequent unlocking could mean a search for social connections or a response to received notifications.
\end{itemize}

\subsection{Conversation Features}
Conversation features are extracted through the passive data collected through the microphone of an individual’s smartphone. It provides the start and end timestamps of each verbal communication activity. These features can provide important information about social engagement and verbal communication patterns. Below are some key features.

\begin{itemize}
  \item \textbf{Conversation Duration and Frequency:} This calculates the total duration of conversations, the longest, shortest and average durations.
  \item \textbf{Conversation Timing:} This calculates the times of the first and last conversations in a day that can indicate a person's daily communication rhythms.
\end{itemize} 

\subsection{Physical Activity/Steps Count Features}
Physical activity and step count features provide information about an individual's physical activity levels, which is an important indicator of overall well-being. These features are particularly relevant for loneliness detection as variations in physical activity can reflect changes in mood, social interactions, and daily routines \cite{morgan2013physical}. Below are some key features.

\begin{itemize}
\item \textbf{Sum/Avg Duration:} Measures the total/Avg time spent in each activity state (active, sedentary) within a time segment.
  \item \textbf{Range of Steps:} It extracts the highest and lowest number of steps taken by an individual within a time segment.
  \item \textbf{Average Daily Steps:} It calculates the average number of steps taken each day during a time segment which provides a baseline measure of an individual’s general activity level.
  \item \textbf{Median Daily Steps:}  It calculates the median step count, which can be less affected by outliers in the data and may offer a more representative value of typical daily activity.
  \item \textbf{Variability in Daily Steps:} It measures the standard deviation of the daily step count throughout a time segment to understand the consistency of physical activity. A higher standard deviation can indicate significant variations in daily activity which could be linked to changes in routine or social patterns.
\end{itemize}

\subsection{Call Log Features}
Call log features extracted from smartphones provide important information about an individual's social interactions and communication patterns. These features are important for loneliness detection as they can tell about the frequency, diversity, and nature of social connections. Frequent and diverse communication may indicate strong social ties, while irregular or limited call patterns could indicate social isolation or emotional distress \cite{thakur2021predicting}. A low frequency of calls and limited contact diversity can indicate a smaller or less active social network which is an important indicator of loneliness \cite{schutter2021loneliness}. Below are some key features.

\begin{itemize}
  \item \textbf{Call Frequency and Contact Diversity:} Metrics include the total number of incoming and outgoing calls in an epoch and the count of unique contacts.
  \item \textbf{Call Duration:} It calculates the average, total, shortest, longest, and the range of call durations. The mode and entropy are also computed to find the regularity and diversity in call durations.
  \item \textbf{Missed Call Patterns:} Different features related to missed calls like looking at how often they occur, the variety of contacts that calls are missed from, and the timing of these missed calls.
\end{itemize}

\subsection{Messages/SMS Log Features}
Similar to call log features, messages are also an important element in finding an individual's social engagement, communication patterns, and potential signs of social withdrawal. However, it is important to note that the use of traditional SMS messaging has reduced significantly in recent years due to the more common usage of messaging apps like WhatsApp. But still in the context of the StudentLife dataset (collected in 2014), SMS data remains a relevant indicator of social interaction. Below are some of the key features.

\begin{itemize}
  \item \textbf{Message frequency and contact diversity:} This Includes the total number of text messages sent and received, as well as the count of unique contacts involved in the messaging per each time segment. These figures are important indicators of the level of social engagement of an individual's social network.
  \item \textbf{Message timing:} By recording when the first and last messages are sent each day and during specific periods, can provide a person's daily communication habits.
\end{itemize}


\subsection{Connectivity features: Bluetooth and WiFi}
Bluetooth and WiFi connectivity features can provide data about an individual's social environment and potential interactions. These features include the frequency of device encounters, the diversity of connections, and the persistence of interactions with specific devices or networks. Analyzing these features can help researchers identify patterns that may correlate with loneliness. For example, a decrease in the number of unique Bluetooth devices encountered or a decline in WiFi scans could indicate reduced social interaction or maybe a person is spending time somewhere alone \cite{boonstra2018using}. Similarly, the persistence of interactions with specific devices might reveal the strength and regularity of social ties. 

\begin{itemize}
  \item \textbf{Bluetooth or WiFi Scans:} The count of scans represents the total number of times Bluetooth devices or WiFi access points are detected during a specific time segment. A higher number of scans may indicate more frequent detection of nearby devices that may show being present in a socially active environment. However, it is important to interpret these findings cautiously, as decreased scans could also be due to other factors which could be unrelated to social interaction, such as device settings or surroundings conditions.
  \item \textbf{Unique Device Count:} For both Bluetooth and WiFi, the number of unique devices or access points sensed in a given time segment is recorded which is find through MAC addresses. This metric could tell the diversity of potential interactions and suggests the individual's movement across different social settings.
  \item \textbf{Mean and Standard Deviation of Scans:} By calculating the mean and standard deviation of scans for sensed devices can help in understanding the regularity and consistency of interactions.
\end{itemize}

\subsection{Sleep Features}
Sleep patterns are important factors to determine overall health and well-being. Disturbances and rregularities in sleep patterns are often associated with various psychological conditions, including loneliness \cite{milojevich2016sleep,griffin2020loneliness}. Sleep features are computed for a single epoch, which is per day. Below are some key features related to sleep:

\begin{itemize}
  \item \textbf{Sleep timing and duration:} The times when individuals first wake up and their last bedtimes are tracked to find their sleep schedules. Similarly, the total time spent while sleeping is recorded as sleep duration.
  \item \textbf{Sleep efficiency:} This is measured by the ratio of the actual time spent asleep compared to the total time spent in bed. A higher ratio means better sleep quality, as it means a greater proportion of time in bed is spent sleeping.
  \item \textbf{Time in bed:} This includes both sleeping and awake time while in bed.
  \end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Data Cleaning}
Data cleaning is an important step for performing a meaningful analysis. This involved applying different operations to the data to improve its quality and reliability before using this data for machine learning model training.

\begin{itemize}

\item Participants Selection: For our analysis, we included only those students who had completed the post-study loneliness survey. It was to ensure that our analysis was based on a comprehensive and reliable measure of each participant's perceived loneliness at the end of the study period. This approach resulted in a final sample of 41 students out of 48 for the StudentLife dataset and 205 out of 218 for the GLOBEM dataset. Our rationale was based on the idea that post study measures more accurately reflect the student's actual feelings of loneliness during the semester. We considered that these feelings might be influenced by academic pressures, such as exams or project deadlines, as well as the overall social relationships of students throughout the semester. This makes post-study loneliness measures a more valid representation of the student's mental state throughout the academic term. While this approach provides an end of study assessment, we acknowledge that it may not capture fluctuations in loneliness levels throughout the study period.

%\item Train/Test Split: For classification tasks, the dataset needs to be divided into test and train sets. Splitting the dataset helps to evaluate the performance of the model on unseen data. The training set will be used by the model to fit, learn and adjust the weights, while the test set evaluates the model performance on the new dataset to ensure the model’s generalizability. The splitting was performed in a stratified manner to maintain the same proportion of lonely and non-lonely students in both sets. We used different split ratios in different chapters based on requirements, which we will present in each chapter.

\item Outlier Removal: It is important to identify and remove outliers from the numerical features in the data since they can influence on different analyzes and model training. We used the Interquartile Range (IQR) method for this due to its suitability for sensor data, which often contains non-normally distributed values and sometimes extreme readings. We performed outlier removal separately per student data since sensor data can vary significantly between individuals. We chose the commonly used 1.5 * IQR threshold as it has a balance between removing extreme values that could skew analysis and retaining potentially meaningful variations in the data. The IQR method works as follows:

\begin{itemize}

\item Calculate Q1 (25th percentile) and Q3 (75th percentile) for each feature
\item Compute IQR = Q3 - Q1
\item Define outliers as any data points falling outside the range [Q1 - 1.5 * IQR, Q3 + 1.5 * IQR]
\end{itemize}

\item Feature Filtering: We applied a feature filtering step to reduce noise in data. First, we excluded features with zero variance. These are the features which provide no discriminative power for our models. As features with constant values across all samples do not contribute to the model's learning process and can potentially introduce unnecessary complexity. Then we removed features with more than 30\% missing data as this is a common threshold in passive sensing based studies. This threshold was chosen as a balance between retaining potentially valuable information and ensuring the reliability of our features. High levels of missing data can lead to biased or unreliable models, and missing imputation techniques become less reliable with excessive missing values. We also assessed multicollinearity among the features using Variance Inflation Factors (VIFs) and subsequently removed any features with a VIF exceeding 5 as this threshold indicates a high level of multicollinearity that could compromise model stability and reliability. In cases where multiple features were highly correlated, we retained the feature with the highest predictive power based on preliminary model testing. We applied these filtering criteria to the training set only and then used the resulting feature set for both training and test sets.

\item Missing Value Imputation: After filtering out features with excessive missing data, we addressed the remaining missing values in our dataset. We used the median for numerical features to impute missing values and the mode for the categorical features. We applied this in train sets and used the median and mode derived from train set to impute into validation and test sets. While other approaches could also be considered like ffill and bfill but sensor data often has variability and irregular patterns, which makes these methods less suitable due to their potential to introduce bias from preceding or succeeding values. That’s why using the median and mode was a more generalized approach since it maintains dataset integrity and stability for models and also reduces the risk of overfitting to specific temporal patterns.

\end{itemize}

\section{Models Training and Evaluation}
Throughout this thesis, we have performed different statistical analyses and trained different machine learning models for loneliness detection in different contexts and research questions. Here, we will present the generic approach of the trained models, while specific details will be presented in each chapter. We used UCLA response thresholds as we discussed above to divide loneliness into 2 categories for binary classification problem (lonely/not lonely). We used these class categories to label our data. We chose binary classification due to the relatively small size of the datasets and the limited 10-week period of data collection. Since loneliness levels change very slowly, categorizing it into multiple levels or treating it as a continuous variable would have been challenging with the limited data available. We mainly use logistic regression, k-nearest neighbours (KNN), support vector machine, random forest, and XGBoost algorithms for this classification.

We chose these ML algorithms because they are widely used in supervised classification, easy to train, and interpretable \cite{garcia2018mental}. Moreover, these ML algorithms have also been used in previous studies that we discussed in Chapter 2. We used the mutual information (MI) method with cross-validation to select optimal features for our models. Mutual information measures how much a feature tells us about the target variable. Higher MI scores indicate more informative features. It works by first ranking features in descending order based on their MI scores and then calculating the cumulative sum of these scores. Features are then selected until their cumulative MI reaches a 95\% threshold of the total MI. This approach helps to ensure that the selected features collectively explain a substantial portion of the variance in loneliness while avoiding redundancy and potential overfitting.

To handle biases in the model due to class imbalance (lonely/not lonely), we used the synthetic minority oversampling method (SMOTE) \cite{chawla2002smote}. We implemented SMOTE for each fold of cross-validation on the training data. This method creates synthetic examples of the minority class without discarding existing data or introducing excessive duplication. It leads to a more balanced training set for our classification models. This approach helps the model to get trained equally for both classes and reduce the bias toward the majority class to improve the overall performance of the classification models \cite{elreedy2019comprehensive}. For hyperparameter optimization, we used grid search to select the best parameters. It works by defining a range of hyperparameter values for each model. These ranges were decided based on pilot runs. Then, this systematically tests model performance for each combination of hyperparameters. Finally, it selects the combination that provided the best cross-validated performance. We used both nested cross-validation for hyperparameter tuning and k-fold cross-validation for performance assessment, depending on the specific requirements of each chapter. Detailed explanations of the cross-validation procedures used in each chapter are provided in their respective methodology sections. The performance metrics used for evaluation included accuracy, precision, sensitivity, and F1 Score.

\begin{enumerate}
  \item Accuracy: Provides an overall evaluation of the model's ability to correctly classify students as \textquotesingle lonely\textquotesingle \space or \textquotesingle not lonely\textquotesingle. It is calculated as the proportion of correct predictions (true positives and true negatives) out of the total number of cases examined.

  \begin{equation}
  \text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}
  \end{equation}

  where TP is True Positives, TN is True Negatives, FP is False Positives, and FN is False Negatives.

\item Precision: This metric focuses on the model's accuracy in identifying individuals who are truly lonely. It is the proportion of correctly identified lonely individuals (true positives) among all those classified as \textquotesingle lonely\textquotesingle. High precision is important in loneliness detection to minimize false alarms.
  
  \begin{equation}
  \text{Precision} = \frac{TP}{TP + FP}
  \end{equation}
  
  \item Sensitivity (Recall): This metric measures the model's ability to identify all truly lonely individuals within the dataset. It is the proportion of correctly identified lonely individuals (true positives) out of all actual lonely individuals. High sensitivity is crucial in ensuring that no lonely individuals are missed, as overlooking them could result in a lack of timely support and intervention.

 Sensitivity has greater relevance than specificity in the context of loneliness detection as the primary goal is to identify individuals experiencing loneliness correctly. Missing a lonely individual (false negative) can have significant implications which makes it crucial that the model captures as many true positives as possible. Therefore, we focused on reporting sensitivity along with precision and accuracy to provide a balanced view of the model’s performance in identifying lonely individuals. Specificity, on the other hand, measures the model's ability to correctly identify those who are not lonely (true negatives). While it is important to minimize false positives, specificity is less critical in this context, where the priority is on avoiding missed cases of loneliness.


\begin{equation}
  \text{Sensitivity} = \frac{TP}{TP + FN}
\end{equation}

    \item F1 Score: This provides a balanced evaluation of the model by considering both precision and sensitivity as it provides the harmonic mean of both metrics. It shows the model's ability to accurately detect loneliness while minimizing both false positives and false negatives.
\begin{equation}
  \text{F1 Score} = 2 \times \frac{\text{Precision} \times \text{Sensitivity}}{\text{Precision} + \text{Sensitivity}}
\end{equation}

\end{enumerate}