Predicted and user perceived heat strain using the ClimApp mobile tool for individualized alert and advice

Thermal models and indices integrated into a mobile application could provide relevant information regarding thermal stress and strain to the general public. The aim of the current paper is to


Introduction
Globally, emerging trends of increased duration, frequency and intensity of heat waves are being reported (Trancoso et al., 2020;Baldwin et al., 2019;Rahmstorf and Coumou, 2011).Record high ambient temperatures are now more common and the most extreme temperatures are even more intense (Hansen and Sato, 2016).In some climate regions the heat is already considered to be intolerable and has a severe impact on humans' daily life and health (Levy and Patz, 2015;Raymond et al., 2020).Heat limits the time people are able to be physically active and reduces work productivity, as more breaks are required to avoid health problems like dehydration, spasms and fatigue or even more serious illnesses such as heat stroke (Levy and Patz, 2015;Binazzi et al., 2019).A study predicted that heat acclimatized people in Perth in 2070 will not be able to perform manual labor for 15-26 days a year due to the extreme heat, while currently it is only one day per five years (Maloney and Forbes, 2011).For unacclimatized individuals of low physical fitness and low body mass, performance decrement is probably greater, as these factors are shown to be most detrimental for working in the heat (Foster et al., 2020;Foster et al., 2021).Furthermore, the advancing ageing population will see a rise in the number of heat related morbidity and mortality as people get older (Basu, 2009;Baccini et al., 2008).
Over the last decades more than one hundred heat stress indices have been developed to quantify heat stress and strain and reduce the health risks associated with high temperatures (Havenith and Fiala, 2016;Beshir and Ramsey, 1988).These indices vary significantly in their complexity and applicability (Havenith and Fiala, 2016).Simple indices are used more frequently as they are easier to understand for the general population.However these are often less accurate, omit important factors that affect the heat exchange between the body and the environment, and may underestimate the heat stress or strain perceived during high intensity exercise or outdoor manual labor (Grundstein and Vanos, 2020;Havenith and Fiala, 2016).This can lead to dangerous situations as potentially inappropriate measures may be taken due to over-or underestimation of heat stress or strain based on misuse or misinterpretation of these thermal indices.More complex indices and thermal models provide a better estimation of the heat stress or strain as these take into account more input parameters and individual variation.However, these indices and models may be difficult for the general public to interpret and use.To date, complex indices are mainly used in research (Havenith and Fiala, 2016).A solution to make these complex indices, models and international standards more accessible could be to integrate them into mobile applications.The user can fill in personal information as input to estimate the heat strain and receives output in the form of individualized advice to adapt behavior to the current thermal environment.The input of personal information is essential for individualized advice as heat strain is dependent on factors such as activity level, clothing, age, sex, heat adaptation and weight (D'souza et al., 2020;Shoenfeld et al., 1978;Chung and Pin, 1996;Ambrosio et al., 2019).More heat is produced in the body when the activity level increases and the clothing insulation influences the amount of heat that can be lost via the skin (Parsons, 2002).Age influences the heat dissipation as the thermoregulatory system of children is not yet fully developed and in older adults the function declines (Basu, 2009;Meade et al., 2020;Xu et al., 2012;Gomes et al., 2013).Furthermore, women have lower sweat rates and a higher heat storage than men, which mainly results in higher experienced heat strain at heavy heat loads (Foster et al., 2020;Yanovich et al., 2020).Heat adaptation occurs after sufficient and repeated heat exposure and results in, amongst others, improved sweating response and a lower heat storage (Périard et al., 2015;Foster et al., 2020).Weight influences the experienced heat strain as well, as individuals with a higher body mass can store more heat than individuals with a lower body mass (Foster et al., 2020).
A few mobile applications based on thermal indices are already developed, like the Hot Environmental Assessment Tool (Sauter, 2012), the Occupational Safety and Health Administration -National Institute for Occupational Safety & Health (OSHA-NIOSH) Heat Safety Tool (National Institute for Occupational Safety and Health -OSHA-NIOSH Heat Safety Tool App, 2018) and the Predicted Heat Strain mobile application (University of Queensland -Predicted Heat Strain Mobile Application).However, these mobile applications lack accuracy or user-friendliness due to the need to manually input weather information (Sauter, 2012), weather information based only on ambient temperature and humidity or a tool too complicated for laymen with usability issues (National Institute for Occupational Safety and Health -OSHA-NIOSH Heat Safety Tool App 2018; University of Queensland -Predicted Heat Strain Mobile Application).Furthermore, there is a chance the heat strain estimated by the thermal index does not coincide with the thermal perception of the user.This can be due to individual variation or parameters that are not included in the thermal index, but do influence thermal strain.Before these complex thermal indices integrated into mobile applications are provided to the general public, the outcome of the predicted thermal strain by the mobile application needs to be validated against the thermal perception of the user.
In the current paper, we aim to briefly describe the development of a new mobile application, ClimApp, which combines weather data with thermal models and indices, and user data to provide the user with individualized and timely advice on appropriate actions in thermally challenging environments and activities.Next, we aimed to validate the output of the app in the heat with the thermal perception of users and test the user-friendliness by a questionnaire.

Methods
The methodology is separated into two distinct sections.The first section briefly describes the development of ClimApp and the second section describes the study that was performed to validate the output and test the user-friendliness of ClimApp in the heat.

Development of ClimApp
ClimApp is developed by a group of experts in the field of thermal physiology, occupational, environmental and public health, computer science, ergonomics, thermal comfort and climate science from the Lund University, University of Copenhagen, Technical University of Denmark and the Vrije Universiteit Amsterdam.The mobile application aims to provide the user with a prediction of the thermal strain and advice on precautionary measures based on information about meteorological data, individual user characteristics and thermal models and indices.ClimApp provides information about thermal strain, and not thermal stress, as individual user characteristics are included in the output such as acclimatization status and metabolic rate.Projected users include outdoor workers and (the caregivers of) vulnerable groups, such as children and elderly.It can be used by employers of outdoor workers to decide how many breaks they should take and how much water they should drink in the heat, or by (the caregivers of) vulnerable people to estimate if cooling needs to be provided, to name some examples.The interface of ClimApp is shown in the appendix and the mobile application can be downloaded for iOS and Android.Below a brief description of the development of ClimApp is provided.A more elaborate and technical description can be found in Kingma et al. (2021).

Meteorological data
Meteorological data are extracted and computed from the Open Weather Map API based on the user's GPS location (Open Weather Map).The data consists of air temperature ( • C), wind speed at 2 and 10 m high (m/s), humidity (%), cloud coverage (%) and solar radiation (W/m 2 ) for now and a forecast for the next 24 h (with 3 h time resolution).

Individual user characteristics
The user can choose to fill in their age, sex, height, weight and if they consider themselves to be heat acclimatized.Heat acclimatization is defined as being exposed to the same or more extreme hot conditions for at least one week prior to the assessment period.The mobile application user fills in the appropriate activity level, ranging from rest (sitting at ease) to severe (very intense occupational activity at fast maximum speed).Finally, the user supplies information about the clothing composition.

Thermal indices
Several heat balance models and thermal indices are used in ClimApp.For cold stress and strain the required clothing insulation (IREQ) (ISO-11079:2007) and Wind Chill index (Engineering-ToolBox, 2003) are used.As this paper focusses on the heat, the IREQ, Wind Chill index and the use of the mobile application regarding cold stress and strain are not discussed.For heat stress and strain the Wet Bulb Globe Temperature (WBGT) (ISO-7243:2017) and Predicted Heat Strain (PHS) (ISO-7933:2004) are used.The WBGT together with the Heat Shield Risk Level (HRL) (Morabito et al. 2019) are combined into the so-called ClimApp index.WBGT is a heat stress index that represents the thermal load of an environment a person is exposed to (ISO-7243:2017).In ClimApp, the weather parameters (air temperature ( • C), wind speed at 2 and 10 m high (m/s), humidity (%), cloud coverage (%) and solar radiation (W/m 2 )) derived from the Open Weather Map API are used to calculate the WBGT using the method of Liljegren et al. (2008).In ISO 7243 reference values (WBGT ref ) are provided for five levels of metabolic rate and heat acclimatization status (ISO-7243:2017).As default, a long sleeve cotton shirt and cotton pants are assumed to be worn as work clothing in the heat.Otherwise the WBGT is corrected with a clothing adjustment value (CAV) to obtain a WBGT value (WBGT eff ) representative for the perceived heat stress with that type of clothing (Eq.1) (ISO-7243:2017).
The predicted thermal strain of the user in ClimApp, based on the WBGT and the Heat Shield Risk Level (HRL) in the heat, is shown as the ClimApp index (Morabito et al., 2019).The ClimApp index ranges from − 4 to + 4, where values below 0 indicate cold strain and values above 0 indicate heat strain.In the heat, the HRL is used to determine the ClimApp index values by defining a ratio of the WBGT eff over the reference values of the WBGT (WBGT ref ) (Eq.2) (Morabito et al., 2019).The HRL is categorized as not significant (HRL < 0.8), low (0.8 < HRL < 1), moderate (1 < HRL < 1.2) and high risk (HRL > 1.2).The WBGT ref is calculated separately for acclimatized (Eq.3) and unacclimatized (Eq.4) people with metabolic rate (M) in Watts.
The ClimApp index is calculated with the HRL as follows: • 1 < ClimApp index < 2: the recommended alert limit is being approached and moderate heat strain can be expected.
Furthermore, the PHS is used to calculate the total amount of sweat loss and the duration until a rectal temperature of 38 • C is reached.In the PHS the heat balance between the human body and the environment is calculated, based on weather condition, the activity level, clothing and heat acclimatization (ISO-7933:2004).Based on this information the maximum exposure time is advised in ClimApp and how much fluid intake would be sufficient to replace the fluid lost by sweating.

Ethical approval
Participants gave consent in ClimApp to use their data for research purposes.For participants under the age of 18 years parents or caregivers gave written consent.

Participants
For this study a heterogeneous group of participants were recruited to test the validity of ClimApp to have a wide variation of potential users.Participants were recruited from different places and sources in the Netherlands, such as via a summer camp in Leusden, The Netherlands, advertisements on social media and sport clubs, by email or asking acquaintances.Participants were excluded if they did not have access to a technical device to install ClimApp or were unable to understand the used technology in the mobile application.

Study procedure
An instruction sheet displaying all necessary steps and information on how to download and use ClimApp was sent to each participant by email.Additionally, a video incorporating these instructions was recorded and sent to participants for extra clarification.All participants were requested to follow all steps which consisted of installing the mobile application, allowing the application to use GPS and fill in personal settings such as age, height, weight, sex and if the users considered themselves to be heat acclimatized.After these initial steps the participants were free to upload data entries at any time or day of the week.These data entries were requested during the four weeks of August 2020.A data entry consisted of entering clothes worn at that moment and the activity level.Once this was filled in, ClimApp provided as output the expected thermal strain at that moment.Next, the participant was requested to enter if the perceived thermal strain was similar, lower or higher than the predicted heat strain by ClimApp.This deviation in thermal perception from the ClimApp index (TP dev ) ranges from − 4 to +4 and is 0 if the ClimApp index was the same as the perceived thermal strain, higher than 0 if the participant felt warmer than the predicted thermal strain of ClimApp, and lower than 0 if the participant felt colder than the predicted thermal strain of ClimApp.
At the end of the study, all participants were requested to fill in a user satisfaction questionnaire about ClimApp.This questionnaire aimed to provide insight in the user experience of the application.

Statistical analysis
The data were analyzed using RStudio 1.1.463and Stata 16.0.The ClimApp index was validated using TP dev .The output of Cli-mApp was considered to be an accurate representation of the heat strain if the TP dev was 0, a good representation if TP dev was between − 0.5 and 0.5, a moderate representation if TP dev was between − 1.0 and 1.0, and a poor representation if TP dev was < -1.0 or > 1.0.
As the data were hierarchically structured, multilevel mixed-effects linear regression analysis was used to test statistically if the predicted heat strain of ClimApp coincides with the thermal perception of the user.The residuals were normally distributed and a model with a random intercept and slope for participant was fitted.The variable 'participant' was chosen to be included as level since the number of entries per participant differed.Three models were fitted with TP dev as dependent variable and the empty model is shown (model A) for calculation of the explained variance.The independent variables were the WBGT eff ( • C), age (per decade), sex and BMI (model B), the WBGT eff ( • C) (model C) and the ClimApp index (model D).Explained variance (R 2 ) is calculated for model B, C and D using the following equation (eq.5) (Xu, 2003).
In equation (4) σ 2 is the level-one random error variance of the full model (i.e.model B, C or D) and σ 2 0 is the level-one random error variance of the empty model (i.e.model A).
The user satisfaction questionnaire about ClimApp was quantitatively analyzed by calculating the percentage of participants providing a certain answer.The internal consistency between the statements of the questionnaire was calculated with Cronbach's alpha (Field et al., 2012).

Validation of ClimApp
In total, 134 individuals (62 males, 72 females; characteristics presented as median (range); age: 25 (8-81) years, height: 177 (130-198) cm, weight: 67 (40-115) kg, BMI: 21.9 (13.7 -32.6)) participated in this study.The total number of data entries were 1302  with a median number of eight per participant and a range of 1 to 46.Data were entered all over the Netherlands, and a few in Belgium and Germany, with a median (range) ambient temperature of 20.2 • C (14.5-34.8• C), relative humidity of 72% (21-100%) and WBGT of 20.2 • C (12.0-33.3• C).Fig. 1 shows the ClimApp Index (A), WBGT ref ( • C) separately for acclimatized and unacclimatized individuals (B), and TP dev (C) at WBGT eff ( • C) measured in this study, as well as the TP dev at the ClimApp index provided as output in this study (D).The ClimApp index shows values above zero only, indicating there was only heat strain and no cold strain during data collection.Table 1 shows the number of data entries representing a certain value or range of TP dev filled in by the participants.476 Data entries (36.6%) were 0, indicating the predicted heat strain by ClimApp was exactly the same as the thermal perception of the participants, which is considered as an accurate representation of the perceived heat strain.354 data entries (27.2%) were between − 0.5 and 0.5 (excluding 0), which is considered as a good representation of the perceived heat strain.254 Data entries (19.5%) were between − 1 and 1 (excluding − 0.5 to 0.5), which is considered as a moderate representation of the perceived heat strain.218 Data entries (16.7%) were higher than 1 or lower than − 1, which is considered as a poor representation of the perceived heat strain.
Table 2 shows the results of the multilevel mixed-effects linear regression analysis for the relation between TP dev with the WBGT eff separately (model B) and with age (per decade), sex and BMI (model C), and with the ClimApp index (model D).Results show a significant relation (p < 0.000) between TP dev and the WBGT eff , but not with age, sex and BMI.Equation ( 6) shows the relation between TP dev and the WBGT eff .At a WBGT eff of 13.5 • C TP dev exceeds 0 and increases with 0.08/ • C, indicating thermal perception increasingly exceeds the predicted heat strain by ClimApp as WBGT eff gets higher.A significant relation (p < 0.000) is shown between TP dev and the ClimApp index.Equation ( 7) shows that the deviation in thermal perception of the predicted heat strain by ClimApp increases once the ClimApp index is higher.The explained variance (r 2 ) of TP dev by the WBGT eff and ClimApp index is low with respectively 0.17 and 0.11.

User satisfaction questionnaire about ClimApp
Of the 134 participants, 45 (26 males, 19 females) filled in the user satisfaction questionnaire about ClimApp.Ages ranged between 16 and 40 and most people were in the age group 21-25 (18 participants) or 26-30 (12 participants).In the age groups 16 -20, 31-35 and 36 -40 there were respectively five, three and seven participants.Five participants mainly worked outside, three took care of an older adult and one took care of a child.The remaining participants' occupation were not specified in the questionnaire.Fig. 2 shows the answers of the participants on the user satisfaction questionnaire about ClimApp.The internal consistency of the statements in the questionnaire was good with an Cronbach's alpha of 0.87 (Field et al., 2012).Most participants did not think they would use ClimApp frequently with 77.8% (strongly) disagreeing with the statement 'I think that I would like to use this mobile application frequently'.The majority of the participants believed the mobile application was not complex (S2: 64.5% (strongly) disagree), and thought the mobile application was easy to use (S3: 71.1% (strongly) agree) and they did not need the support of a technical person to be able to use the mobile application (S4: 80% (strongly) disagree).The various functions in the mobile application are well integrated with 37.8% (strongly) agreeing with question 5 and 37.8% did not agree/disagree.Most participants did not think there was too much inconsistency in the mobile application (S6: 62.2% (strongly) agree).Part of the participants believed that most people could learn to use the mobile application quite easily (S7: 48.9% (strongly) agree), found the mobile application not cumbersome to use (S8: 68.9% (strongly) disagree) and did not need to learn a lot of things before to get going with the mobile application (S9: 73.3% (strongly) disagree).35.6% of the participants thought ClimApp was useful, but 51.1% of the participants believed the information was irrelevant for them.

Discussion
To our knowledge, this is the first study aimed to investigate the validity and user-friendliness of thermal models and indices incorporated in a mobile application to serve as a tool for the general public.It appears that the mobile application ClimApp is able to predict heat strain of the user rather well with 83.3% of the data entries of the participants ranked as a moderate, good or excellent representation of the perceived heat strain.However, the thermal perception increasingly exceeds the predicted heat strain by Cli-mApp as ambient temperature rises, although WBGT eff only represents 17% of the variance in TP dev .The representation of the perceived heat strain becomes particularly poor at a WBGT eff higher than 28 • C, since at that point TP dev exceeds 1.These observations appear to be in line with another study where the WBGT was used as thermal index and resulted in a decrease in reliability once the heat strain risk condition and workload became more intense (Dillane and Balanay, 2020).It could be that thermal indices, such as in this case, predict the heat strain correctly, but that the thermal perception is higher.Since thermal perception plays an important role in thermal behavior and acts as a warning mechanism of the human body, it could be that the experience of the heat is more intense than the actual heat strain (Flouris and Schlader, 2015).This way the individual is more inclined to change their behavior before they experience the negative effects of the heat.If this is the case, it would make the use of a mobile application such as ClimApp, which has as a goal to warn about the risks of the heat, less valuable as the human body already does this itself.On the other hand, an individual could miss out on opportunities if activities are stopped too early without it being necessary from a health perspective.Therefore, it is important that the thermal perception accurately represents the experienced heat strain.However there are many cases of heat related disorders which could have been prevented if appropriate precautionary actions were taken.It could be that in some cases the thermoregulatory system of the human body does not function properly, which may be the case in vulnerable populations such as children and elderly.Other possibilities are that people do not adapt their behavior well enough based on their thermal perception or are unable to adapt their behavior.For example in occupations where it is difficult to self-pace such as in the military or fire brigade.It can also be that people are unaware of appropriate precautionary measures to reduce the perceived heat strain.In these cases, a mobile application as ClimApp can be of added value, as it advices the user on appropriate behavioral changes for the specific situation.
Furthermore, no relation was found between TP dev and age, BMI and sex.A previous study showed that these factors do influence thermal sensitivity and comfort (Thapa, 2019).Therefore, ClimApp seems to predict heat strain well for people of different age, BMI and sex.However, more older adults need to be included in a future study to confirm if ClimApp indeed provides an accurate prediction of heat strain for older adults, as only 10 of the 134 participants in this study were 65 years or older.
Overall the participants were positive about the user-friendliness of the mobile application and believed ClimApp was easy to learn and use.It has been shown before that the user-friendliness of a mobile application is critical for it to be used (Zhang and Adipat, 2005).Especially for a mobile application as ClimApp, it is important that it is easy to use and to interpret the outcome, as one of the main aims of ClimApp was to make thermal models and indices more accessible for the general public.However, most participants did not think they would use ClimApp frequently (77.8%) and believed the information was irrelevant for them (51.1%).This is surprising as previous research shows the number of heat illness cases are increasing and the negative effects of heat occur in all populations exposing themselves to the heat (O'Connor and Casa, 2017).A study investigating the public perception of the effect of heat on human health in the United States showed that people living in colder regions underestimate the effect of heat more than people living in areas with higher ambient temperatures (Howe et al., 2019).Meanwhile these people are more likely to experience the negative health effects of the heat as they are not acclimatized to the heat and are less familiar with adapting their behavior.Since the Netherlands has a temperate marine climate with an average temperature of 10.5 • C (Folkerts et al., 2020; Central Intelligence Agency -The World Factbook) it is a relatively cold country and therefore it may be the case that Dutch citizens underestimate the effect of the heat as well.The same study showed older adults underestimate the effect of heat (Howe et al., 2019), which is worrying since they are most vulnerable for heat-related morbidity or mortality (Baccini et al., 2008;Koppe et al., 2004).In the Netherlands since 2007 a heat health warning system is activated when a heat wave is expected, with the purpose of increasing awareness of the health risks of the heat to reduce heat related morbidity and mortality (Lowe et al., 2011).However, a study looking into the effects of this heat health warning system showed that care organizations were not familiar with it and did not prioritize the heat as a risk factor (Van Loenhout et al., 2016).Furthermore, it has been reported that certain vulnerable groups, such as those who are socially isolated, have been overlooked in the heat health warning system (Van Loenhout et al., 2016).It appears there needs to be more focus on making people aware of the negative health effects of the heat and reach all groups of individuals.ClimApp could play a role as a tool to make heat warnings more accessible for everyone and make people aware of appropriate behavior during heat strain.

Limitations and future directions
Although a large number of participants were included in this study, the larger part were healthy young (median age 25 years old) adults.Young and fit adults can be at high risk for, for example, occupational heat strain, since they can achieve larger metabolic rates than their older and less fit counterparts (Foster et al., 2020).However, in general young adults are not the most vulnerable for the effects of the heat.A future study should include more vulnerable people such as older adults, outdoor workers or people with certain diseases that makes them more vulnerable to the heat, to make sure mobile applications, such as ClimApp, are applicable for these populations as well.Another limitation is the amount of people that filled in the user satisfaction questionnaire, which were only 34% of the participants.The young participants (<18 years) and older participants (>40 years old) were not included in the user satisfaction questionnaire, so the results are valid for adults aged 18-40 years only.Furthermore, only 5 out of the 45 participants who filled in the user satisfaction questionnaire were outdoor workers and only 3 caregivers of children or elderly, and therefore the participants perhaps believed the information was less relevant for them.It is recommended to include the young and older age groups and outdoor workers in future surveys since the appreciation of a mobile application may be dependent on age.
In this study we chose to measure deviation in thermal perception from the ClimApp score instead of absolute thermal perception, which may have influenced the outcome.However, in other fields, for example in wetness perception (Tiest et al., 2012), methods rating the difference instead of absolute scores are often used as it is considered more reliable.Therefore, we believe the results in this study are valid.
ClimApp can be used in both the cold and the heat.However, the current study was only focused on validation in the heat.Therefore, a future study should focus on validating ClimApp in the cold as well.

Conclusion
The mobile application ClimApp seems to be a user-friendly and valid tool to predict heat strain of people of different age, BMI and sex.However, ClimApp may underestimate perceived heat strain when ambient temperature increases.Furthermore, more awareness needs to be created for the negative health effects of the heat, since most participants believed the information provided in ClimApp was irrelevant to them.ClimApp could play a role as a tool to make heat warnings more accessible for everyone and make people aware of appropriate behavior during periods with high ambient temperatures.
M.A.Folkerts et al.

Fig. 1 .
Fig. 1.The ClimApp Index (A), reference values of the WBGT (WBGT ref ) ( • C) separately for acclimatized and unacclimatized individuals (B), and the deviation in thermal perception from the ClimApp index (TP dev ) (C) at the effective Wet Bulb Globe Temperatures (WBGT eff ) ( • C) measured in this study.D shows the TP dev at the ClimApp index provided as output in this study.

Fig. 2 .
Fig. 2. Number of participants providing a certain answer on the user satisfaction questionnaire about ClimApp.

Table 1
Number of data entries representing a certain value or range of the deviation in thermal perception from the Cli-mApp index (TP dev ) filled in by the participants.

Table 2
Multilevel mixed-effects linear regression analysis for the relation between the deviation in thermal perception of the ClimApp index (TP dev ) filled in by the participants (Model A, empty model) with the effective Wet Bulb Globe Temperature (WBGT eff ), age (per decade), sex and BMI (model B), and separately with the WBGT eff (model C) and the ClimApp index (model D).The models are fitted with a random intercept and slope at the level of participant.