A scoping review on monitoring mental health using smart wearable devices

: With the continuous development of the times, social competition is becoming increasingly fierce, people are facing enormous pressure and mental health problems have become common. Long-term and persistent mental health problems can lead to severe mental disorders and even death in individuals. The real-time and accurate prediction of individual mental health has become an effective method to prevent the occurrence of mental health disorders. In recent years, smart wearable devices have been widely used for monitoring mental health and have played an important role. This paper provides a comprehensive review of the application fields, application mechanisms, common signals, common techniques and results of smart wearable devices for the detection of mental health problems, aiming to achieve more efficient and accurate prediction for individual mental health, and to achieve early identification, early prevention and early intervention to provide a reference for improving the level of individual mental health.


Introduction
Mental health problems are also known as mental health disorders, and they have become an important factor affecting individual growth and social stability [1]. According to the "Global Burden of Disease Study", the global prevalence of mental health disorders has always been above 10%, and about 450 million people have mental health disorders [2,3]. Long-term and persistent mental health problems increase the risk of cardiovascular disease, which is the leading cause of increased mortality worldwide [4]. Mental health disorders not only damage the physical and mental health of individuals, but they also increase the overall medical burden on society [5]. It is predicted that the total global medical expenditure for mental health problems will reach 16 trillion US dollars by 2030 [6]. Therefore, it is imperative to detect the mental health condition of individuals at an early stage and prevent the occurrence of mental health problems.
Smart sensors can accurately detect subtle changes (such as insomnia, headache, rapid heartbeat and muscle tension) in the body caused by stress in order to prevent and reduce the adverse effects of mental health problems [7,8]. With the increasing need for mental health care, the application of portable smart devices for monitoring mental health has received widespread attention and has been welcomed by most people [9]. At present, there are many studies on the use of wearable devices for mental health monitoring. Previous reviews have integrated the use of wearable sensors to detect stress levels, and there is a lack of exhaustive reviews on detection topics such as mental health problems or mental illness [8,[10][11][12][13]. However, although some of the studies covered comprehensive information, the classification of detected signals were not described in detail in these reviews [14][15][16][17]. To fill this gap, this paper analyzes and integrates a large number of studies, covering the types of equipment used for data collection, common locations of sensors, common signals, advantages and disadvantages of technology, types of mental health states, etc. In this section, we discuss some existing studies. Abbreviations of nomenclatures used in this review paper are enumerated at the beginning of the article.
First, smart wearable devices have been widely used in the study of classifying and identifying individuals' daily stress and mental fatigue. Betti et al. [18] measured the electroencephalography (EEG), electrocardiography (ECG) and electrodermal activity (EDA) of healthy adults using wearable devices and a support vector machine (SVM) classification algorithm for statistical analysis; the results show that the accuracy of the classification algorithm based on salient features can reach 86% for detecting individual stress levels. Goumopoulos et al. [19] proposed a method to detect mental fatigue; they used a wearable device to collect the heart rate variability (HRV) of healthy subjects, and then selected an appropriate machine learning model to predict mental fatigue with high accuracy. Additionally, the role of wearable devices in the prediction and classification of mental health assessment and emotional states cannot be ignored. Sano et al. [20] explored the relationships between the academic performance, sleep quality, stress perception and mental health of college students using wearable devices and smartphones; they found that college students' academic performance, sleep quality and stress perception can significantly predict their mental health level, and that the accuracy rate of the algorithm is as high as 92%. Costa et al. [21] found that wearable devices can acquire and analyze social signals and identify the current negative emotions of individuals, and that, when it identifies a negative emotion, the device activates the emotional regulation system to help the individual stabilize their emotions. Overall, wearable devices have been widely used in the field of mental health with strong practicability. The main contributions of this paper are given as follows: 1) Recent artificial intelligence (AI)-based wearable devices methods and techniques are reviewed in this paper. This study comprehensively explores the application of AI-based wearable devices in the field of mental health, focusing on stress levels, mental health problems and mental illness.
2) This study covers comprehensive and detailed information, including types and application mechanisms for AI-based wearable devices, common locations of sensors, common signals, the advantages and disadvantages of technology and types of mental health states.
3) The articles included in this paper are from 2017 to 2022, and the articles are relatively new and cutting-edge.
The remainder of this paper is organized as follows. Section 2 introduces the review of body signals and mental health. Section 3 summarizes the methodology and Section 4 gives the literature survey. Finally, application, discussion and conclusion are given in Sections 5, 6 and 7.

Body signals and mental health
The autonomic nervous system has two branches: the sympathetic nervous system (SNS) and parasympathetic nervous system (PNS). The SNS can rapidly stimulate some vital organs (such as the heart, lungs, and sweat glands), and the PNS can inhibit this excitation; they complement the regulation of important bodily functions and maintenance of physiological homeostasis [22]. When an individual is stressed, the individual will activate the protective mechanism; the SNS will be activated, the heart rate will increase, the breathing will increase, sweat will be secreted and other physiological systems will also be affected [23,24]. Additionally, the hypothalamus axis is a hormonal system that is initiated in response to stressors and drives a cascade of endocrine responses, culminating in the secretion of cortisol [22]. Therefore, individual health, especially mental health, can be known by observing the activity of the SNS and hypothalamus axis. Finally, language signals can also reflect an individual's mental health condition. Language is the basis for inferring individual thought processes; it provides a special perspective for thinking observations [25]. When an individual is in a state of tension, the respiratory rate increases and the glottis pressure increases, prompting an increase in the pitch of the vocalized part [25]. Therefore, clinicians mainly diagnose and treat mental illness through language and pronunciation [26]. As mentioned above, subtle changes in the body caused by stress can be detected by sensors connected to the body and hardware devices to accurately predict mental illness [27]. Therefore, AI-based wearable devices can collect one or more signals of the individual to identify and analyze the mental health level.

Search strategy
This review incorporated searches for relevant papers in the databases of Web of Science, Embase, MEDLINE and Engineering Village. The selected papers ranged from 2017 and 2022, which covers the recent developments in this area. The basic sets of keywords were "Psychological assessment", "psychological prediction", "psychological monitoring", "mental health", "psychological health", "mental illness", "psychological disorders" and "wearable device". A total of 1537 papers were retrieved by using all keywords, and 25 papers were finally included. The search methodology for this article was designed to provide a detailed review of published research papers on wearable sensorbased mental health detection. Figure 1 shows a flowchart describing the search strategy used for this review article.

Inclusion criteria
1) The date of publication of the selected papers was between 2017 and 2022. 2) Conference papers and journal papers were included. 3) The research direction for the paper was mental health detection (focusing on stress, mental health problems, mental illness, etc.) and AI-based wearable devices for data collection.

Exclusion criteria
1) Research published in a language other than English was excluded. 2) Studies structured as editorials, press releases, research highlights or letters, review articles, reviews and technical papers were excluded. 3) Duplicate papers, related review papers and papers not related to the topic were excluded.

Wearable devices for mental health monitoring
We use smart/embedded wearable devices, such as smartwatches, that collect signals from the body [28]. This data can be processed by sensors on smart/embedded devices. WiFi is used to send the collected data to cloud servers [7]. In cloud computing, advanced AI models can be applied to mental health monitoring big data to generate simple knowledge through the use of expensive processing and advanced AI algorithms [29], such as Welch's algorithm [30], SVM algorithms [31] and period graphs [32], for the analysis of data. Using these AI algorithms, then by analyzing the information, we will be able to extract sensor data with the appropriate classification characteristics and then classify their mental health states [33]. Artificial intelligence algorithms such as information and SVMs are used again. The main function of an analog classifier is to classify different classifiers [34]. By ensuring the rapidity of calculation, feature patterns are transformed into related classes and the classes most suitable for a single feature are predicted. During the classification process, each feature will be analyzed to find the most suitable feature [35]. They will be analyzed separately to predict stresses. Once the classification is complete, the smart device displays their mental health rating on the screen. Therefore, people can improve their mental health through self-management (as shown in Figure 2). The virtual reality-augmented reality (VR-AR) based digital twin is the critical element for the intelligent development of AI-based wearable devices [36]. Finally, all of the elements are integrated into a smart device's ecology for the future direction framework. Mental health detection systems based on wearable devices mainly consist of sensors, data acquisition, data pre-processing, feature extraction, machine learning algorithms and mental health condition classification. The general scheme of mental health prediction is shown in Figure 3. Section 4 will systematically introduce the current related research work according to the workflow of the wearable mental health detection system, as well as classify, compare and statistically analyze the existing research results.

Data acquisition devices
Currently, the commonly used signals in mental health detection are HRV, EEG, galvanic skin response (GSR), skin temperature (ST), blood pressure (BP), sleep, salivary cortisol and blood oxygen saturation (SpO2). In most studies, researchers use multiple indicators to accurately reflect an individual's mental health. At the same time, the wearable device can also collect indicators such as steps, activity category, posture and activity trajectory. According to the needs of users and the functions achieved by the device, the device will be worn on different parts of the body. Common wearing positions are shown in Figure 4.

Pre-processing and feature extraction
To solve the noise phenomenon in the data acquisition module, data pre-processing is required to reduce the impact of noise on the performance of mental health detection. Common filtering algorithms in mental health detection systems are wavelet transforms, the Kalman filter, etc. These filters can filter out high-frequency noise signals in the data [37,38].
The pre-processed data need to extract corresponding features for training the mental health detection model. Among them, the common feature extraction method is to use the sliding window algorithm to extract the features in the corresponding window, and then put the trained features into the trainer for training to form a mental health detection model. Table 1     The amount of information must be kept at a high level after dimensionality reduction, and the interpretation of principal components is usually fuzzy

Application
Mental health problems are often accompanied by obvious body responses, and wearable sensors play an important role in mental health and its related applications. They can collect an individual's EEG, heart rate (HR), HRV, GSR, BP, body temperature, respiratory rate, etc., to monitor mental health. This section summarizes and discusses the signals commonly used in mental health detection, and briefly summarizes the relevant research on AI-based wearable devices in mental health detection over the past five years. The main contents include mental health disease types, parameters, results, etc., as shown in Table 3.

Mental health detection using EEG
EEG is a detection method that records the electrical activity of the cerebral cortex by connecting multiple electrodes to the scalp [57] to reflect the emotional changes of individuals to specific stimuli.
The stress state of the human body is mainly identified by the changes in the alpha and delta bands of the EEG. When people are under high levels of stress, alpha decreases and delta increases in the EEG, whereas, when they are relaxed or inactive, alpha increases and delta decreases [58]. Ahn et al. [52] used a binaural hanging device that combined EEG and ECG signals and found that the accuracy of EEG and ECG signals in continuous stress assessment was 87.5%. Additionally, based on the multi-modal emotion recognition framework of EmotionMeter, some scholars found that the individual's happiness, sadness, fear and neutral emotions can be distinguished and identified by EEG signals [46]. Finally, EEG combined with ECG signals, body temperature and other data can reliably detect the mental fatigue state of an individual, which provides support for the popularization and specific implementation of mental health assessments [38]. Table 3 shows the studies and their details on the use of EEG for mental health detection. EEG is one of the most effective physiological signals for detecting psychological stress, and the combination of ECG signals and machine learning applications can improve the accuracy of psychological stress detection.

Mental health detection using ECG
Stressors induce changes in myocardial activity, promote blood distribution to vital organs and cause changes in the HR and HRV [59]. ECG is a reliable technique that uses electrodes attached to the body to record the electrical potential produced by heart muscle cells to observe changes in HRV [59]. Studies have shown that HR and HRV signals are closely related to mental health problems (such as anxiety and depression) [60], and they have an important role in mental health detection. Tiwari et al. [43] found that HRV data obtained through smart t-shirts and smart bracelets can more accurately assess the degree of anxiety and depression of individuals. Second, HRV characteristics are closely associated with an individual's clinical course and mood changes. Gentili et al. [55] showed that the HRV characteristics (including time domain, frequency domain, nonlinearity) of patients with bipolar disorder can significantly predict their clinical course and emotional changes. Additionally, Akbulut et al. [47] believe that the stress state of an individual can be accurately identified by signals such as the HRV obtained by wearable devices. Table 3 presents an overview and the details of mental health testing using ECG.

Mental health detection using GSR
Sweat is a biological fluid produced by sweat glands. Sweat glands are distributed almost all over the body, with high-density distribution in the palms, soles and forehead; they are only stimulated by the sympathetic nerves, which are activated under tension [14], so the level of sweat secretion is the body's response to stress. Since sweat contains ions, small molecules and even macromolecular substances such as proteins, it has electrical conductivity [61]. Therefore, the EDA signal is obtained by connecting two electrodes on the skin surface, and the voltage between them is measured to monitor the stress. Sriamprakash et al. [33] collected the GSR, HRV and other indicators, extracted individual stress characteristics by using Welch's algorithm and periodograms, etc., and used SVM and KNN algorithms to classify levels of mental health. Tonacci et al. [62] captured the ECG and skin conductance (SC) signals of subjects by using wearable sensors and compared the effects of cognitive stress and olfactory stimuli on human mental health. Additionally, EDA and GSR also change with different emotions; this can effectively reflect the state of emotional calm or emotional arousal [63].
Evidence shows that the emotional state of patients with metabolic syndrome can be analyzed by using indicators such as the GSR, HRV and body temperature detected by wearable devices [47]. The above studies have confirmed that combining GSR with HRV, body temperature and other parameters can improve the prediction rate for individual stress, and accurately provide and detect information about individual emotional states.

Mental health detection using ST
High levels of stress can cause muscle tension, increase the HR and increase the work of the vital organs; additionally, the blood will be shunted from the limbs to directly act on the vital organs. In just a few minutes, the body surface temperature can change [64]. Most of the studies involved the collection of body temperature data to detect mental health conditions. Fraiwan et al. [65] designed a mobile mental health monitoring system for continuous monitoring of changes in ST, skin electrical impedance and HRV; they found that, when the body was exposed to relaxation or stimulation, the skin electrical resistance value and HRV exhibited significant variability. Engelniederhammer et al. [66] monitored the EDA and ST in subjects wearing a smart wristband (Bodymonitor); the results showed that being in a crowded environment triggers aversive emotional responses (i.e., stress, anger and fear) in subjects. Sano et al. [54] had 201 college students wear two sensors on each wrist: a Q-sensor (for measuring SC, ST, triaxial acceleration (ACC)) and motion recorder (for measuring acceleration and ambient light data); the study showed that wearable sensor eigenvalues are more accurate than cell phones for classifying poor mental health (87%) and stress (78.3%) when using machine learning techniques. In conclusion, the ST, combined with parameters such as EDA, is more accurate for detecting emotional changes and stress levels.

Mental health detection using speech and behavior
With the development of artificial intelligence, the natural language processing program of the computer can more accurately identify the subtle barriers in language before the onset of mental illness, achieving the effect of accurately predicting the mental illness [27]. Corcoran et al. [27] confirmed that the automatic learning of speech classifiers with reduced semantic consistency, increased semantic consistency differences and reduced use of case pronouns can predict psychotic episodes with an accuracy rate of 83%. Language signals help to improve the accuracy of mental disease prediction and have become a powerful tool for psychiatric diagnosis and treatment.
In addition, mental illness is associated with changes in an individual's behavior, especially physical movement and social habits. AI-based wearable devices (accelerometers, global position system sensors) can detect these abnormal behaviors and effectively reduce the incidence of individual mental illness. Zanella-Calzada et al. [45] obtained data on motor activity recorded by wearable devices from a depression database, extracted the statistical features and then fed them to an RF classifier to detect subjects with depression; the results showed that those subjects with the presence of depression had an 86.7% likelihood of being correctly classified, while the specificity showed that those subjects with the absence of depression had a 91.9% likelihood of being classified with a correct response. Additionally, Narziev et al. [67] recruited 20 participants from four different levels of depression groups and established a machine learning model to automatically classify the categories of depression in a short period of time; the results of this pilot study revealed high correlations between participants' ecological momentary assessment self-reports and passive sensing (sensor data) in terms of physical activity, mood and sleep levels. All of the above studies have shown that the role of speech signals and behavioral signals in the detection of mental health states such as depression, mood disorders or schizophrenia cannot be ignored.

Discussion
Multiple studies have integrated the use of wearable sensors to detect stress levels, and there is a lack of exhaustive reviews on topics such as mental health or illness detection. However, some reviews comprehensively cover information, but do not provide details such as machine learning methods, types of mental health states and the classification of detection signals. To fill this gap, this paper presents an extensive review of mental health detection using wearable sensors.
The significant observations from this review are listed below.  Most wearable sensors are placed on the wrist (such as smartwatches and smart wristbands), chest (smart t-shirts, chest straps, etc.) and head (head-mounted devices, etc.) for data collection.
 The Empatica, EmotionMeter and SHIMMER platforms have been popularly used for data collection.
 HRV, EEG, GSR and ST are the most distinctive signals for detecting mental health conditions.  SVM, LR and KNN have been widely used in the classification of mental health conditions.  Most of the disease types studied focused on psychological stress levels and mental health problems (anxiety and depression).
The identified challenges are described below.  Compliance: The collection of subjects' physiological indicators and daily activity data is affected by individual living habits, which makes the detection process more complicated and thus degrades the performance of the mental health detection system. Therefore, the mental health assessment based on wearable devices can only do a rough screening and cannot directly assess the disease accurately.
 Singleness of indicators: most studies involve the collection of only a single category of signals, such as physiology, speech and behavior, while fewer collect a combination. Since different categories of data and signals reflect different aspects of mental health, the integrated collection of all types of signals is essential for the accurate screening of mental health problems.

Conclusions
Globally, mental health problems are still serious social public health problems, and mental health has always been detected via subjective and time-consuming methods. Wearable sensors have great potential as a continuous monitoring system for mental health. As the needs for mental health care services increase, so does the production of AI-based wearable devices. This paper reviewed the types of equipment, common locations of sensors, common signals, advantages and disadvantages of technology and types of mental health states in previous studies. First of all, the most common types of states in mental health testing are psychological stress and mental health problems (anxiety and depression), and most wearable sensors are placed on the wrist, chest or head. Additionally, it was found that there are many AI-based wearable devices on the market that mainly collect physiological signals for mental health detection, and that there are few studies on detecting mental health with speech signals and behavioral signals. Among the physiological signals, the HRV, EEG, ST and GSR signals are the most effective parameter indicators for detecting the mental health condition and this conclusion is helpful for researchers to create devices to improve people's mental health level. Finally, this paper integrated important information, such as the technical advantages and limitations of previous studies; it was found that the most common classifiers are SVM, KNN and LR algorithms, and that these algorithms provide the most reliable results for mental health detection. In the future, with the continuous innovation of science and technology, deterministic annealing neural networks [69,70], convolutional neural networks [71][72][73], the logarithmic descent direction algorithm [74], Laplacian-Hessian regularization semi-supervised extreme learning machine [75,76] and back-propagation neural networks [77] are also expected to be applied and explored in the field of mental health detection. In a word, the goal of mental health testing is to develop an efficient and affordable model with high accuracy. This review will help future researchers design optimal wearable devices to achieve their goals for mental health detection.