Modeling social interaction dynamics measured with smartphone sensors: An ambulatory assessment study on social interactions and loneliness

More and more data are being collected using combined active (e.g., surveys) and passive (e.g., smartphone sensors) ambulatory assessment methods. Fine-grained temporal data, such as smartphone sensor data, allow gaining new insights into the dynamics of social interactions in day-to-day life and how these are associated with psychosocial phenomena – such as loneliness. So far, however, smartphone sensor data have often been aggregated over time, thus, not doing justice to the fine-grained temporality of these data. In this article, we demonstrate how time-stamped sensor data of social interactions can be modeled with multistate survival models. We examine how loneliness is associated with (a) the time between social interaction (i.e., interaction rate) and (b) the duration of social interactions in a student population (Nparticipants = 45, Nobservations = 74,645). Before a 10-week ambulatory assessment phase, participants completed the UCLA loneliness scale, covering subscales on intimate, relational, and collective loneliness. Results from the multistate survival models indicated that loneliness subscales were not significantly associated with differences in social interaction rate and duration – only relational loneliness predicted shorter social interaction encounters. These findings illustrate how the combination of new measurement and modeling methods can advance knowledge on social interaction dynamics in daily life settings and how they relate to psychosocial phenomena such as loneliness.


Introduction
New technological advancements allow researchers to capture individuals' experiences at a fine-grained level as they go about their day-to-day lives (Miller, 2012). Assessments in daily life settings, so-called ambulatory assessments (Trull & Ebner-Priemer, 2014), have the key advantage over more traditional lab-based or survey-based methods in that they allow researchers to study participants' experiences in natural settings and their momentto-moment changes.
In particular, the study of social interactions has profited from measurements in daily life settings, as the fine-grained temporal level matches the timescale in which social interactions happen. Ambulatory assessed data on social interactions have provided insights into numerous psychosocial phenomena (e.g., Elmer & Stadtfeld, 2020;Quoidbach et al., 2019;Vogel et al., 2017). For example, Mehl et al. (2010) showed that well-being was associated with spending less time alone and more time in conversations (also see Milek et al., 2018).
The development of research questions and theories on causes and consequences of social interactions in daily life has gone hand-in-hand with new technological advancements in measuring the complete set of social interactions that individuals experience in their day-to-day life. For this, scholars have used a variety of methods, ranging from event-contingent experience sampling (e.g., Reis & Wheeler, 1991), diary methods (e.g., Gochmann et al., 2022), electronically activated recorders (Mehl et al., 2001), to sensors worn as a nametag . Recently, also the measurement of social interactions via a participant's smartphone has been proposedwith the key advantage that these assessments do not need any active input from the participant, which provides a low burden for participants and, as such, allows researchers to record interactions over more extended periods of time (Bachmann, 2015;Fulford et al., 2021). So far, however, sensor data have often been aggregated over time (e.g., Elmer & Stadtfeld, 2020;Milek et al., 2018), thereby not doing justice to the fine-grained temporality of these data. Thus, as of yet, it has not yet been demonstrated how fine-grained smartphone-sensor data on social interaction dynamics can be modeled and how this granularity allows gaining new insights into the relationship between social interactions and psychosocial phenomena, such as loneliness. This article aims to fill this gap.
In the remainder of this section, we first describe the theoretical relationships between social interactions and loneliness that motivated our exemplary research questions.
Second, we introduce the current study and its unique multi-method design, including smartphone sensors.

The relationship between social interactions and loneliness
The feeling of loneliness arises when individuals experience a discrepancy between the desired and actual quantity or quality of social interactions (Peplau & Perlman, 1982). Prolonged feelings of loneliness can have serious consequences for mental and physical health: Loneliness is associated with numerous mental and physical illnesses, which contributes to a lower life expectancy (S. Cacioppo et al., 2015;Holt-Lunstad et al., 2015). The feeling of loneliness is, by definition, intertwined with our experience of social relationships. Existing research suggests that broader measures of social relationships, such as the quality of friendship and the number of friends one has, are relevant for the development of loneliness (Schwartz-Mette et al., 2020). Yet, much less is known about the day-to-day social experiences related to loneliness, such as social interactions and solitude 1 (van Roekel et al., 2016). The use of (smartphone) sensors to capture social interaction in daily life settings, therefore, has the potential to advance our understanding of the relationship between social interactions and loneliness.
Social interactions are essential for forming stable social relationships and are thus, over time, related to trait loneliness (Schwartz-Mette et al., 2020). Yet, social interactions and the feeling of loneliness are also more directly intertwined: The loneliness model by Cacioppo and Hawkley suggests that when basic human needs of social interactions are thwarted, individuals start to feel lonely as a result of this threat (J. T. Cacioppo et al., 2006;J. T. Cacioppo & Hawkley, 2009;Hawkley & Cacioppo, 2010). Thus, loneliness can result from the subjective evaluation of the fit between the expected and the actual number of social interactions (De Jong Gierveld et al., 2016;Peplau & Perlman, 1982). Importantly, loneliness cannot only be the result of (a lack of) social interactions, but loneliness can also affect how individuals engage in social interactions (Qualter et al., 2015).
On the level of day-to-day social interactions, two mechanisms of how loneliness affect social interactions have been proposed: On the one hand, loneliness could result in a motivation to reconnect and thus predict more or longer social interactions, but on the other hand, loneliness could result in withdrawal and thus predict fewer or shorter interactions (Qualter et al., 2015). As a consequence of such behavioral tendencies of state loneliness, individuals who do not frequently interact with others may fail to build stable social relationshipsthus contributing to higher trait loneliness (Schwartz-Mette et al., 2020).
Thus, based on current findings, it is unclear whether loneliness mainly results in increases in interactions or withdrawal from interactions. Moreover, as existing findings are primarily based on participants' self-reports, effects may be overestimated because of common method biases (Podsakoff et al., 2003). We thus investigate if loneliness predicts social interactions measured with automatically sensed social interaction data.
We differentiate between joining and leaving social interactions as these are two processes with different underlying social mechanisms (Hoffman et al., 2019).
Differentiating between joining and leaving social interactions can help to determine, for example, whether loneliness is more associated with the initiating of social interactions (i.e., joining process or the rate of interactions) or with leaving interactions once they are established (i.e., leaving process or the duration of interactions). Thus, we examine whether loneliness is associated with (a) the time between social interactions (i.e., interaction rate; Research Question 1 [RQ1]) and (b) the duration of social interactions (RQ2)?

The present study
To investigate our research questions, we used data from a student population in which loneliness was measured before a 10-week ambulatory assessment phase (Wang et al., 2014). During the ambulatory assessment phase, the StudentLife sensing application on students' smartphones measured when the students were in a social interaction (N obs = 74,645). The assessment of psychological data via smartphone sensors is often referred to as digital phenotyping, which is the "moment-by-moment quantification of the individual-level human phenotype in-situ using data from smartphones and other personal digital devices" (Torous et al., 2016, p. 2). The StudentLife smartphone application, assessing phenomena with a range of sensors, such as GPS and microphone (Wang et al., 2014), is unique among the set of sensing applications, as it assesses digital phenotypes of sociability together with survey-based measures of loneliness. For a comparison of digital phenotyping apps in mental health research, see Mendes et al. (2022). This study design is unique in two ways, allowing us to gain new insights into the interplay between the dynamics of social interactions and feelings of loneliness.
First, the combination of traditional survey methods (assessing loneliness) with stateof-the-art passive sensing technologies (measuring social interactions) provides a multimethods setting, where common-method biases are reduced (Goossens & Beyers, 2002). The time-stamped nature of the smartphone sensor data allows us to differentiate processes of joining and leaving social interactions (RQ1 and RQ2). These processes can be modeled with multistate survival models, which we introduce in more detail in the Methods section. Multistate survival models have rarely been applied within psychological research, despite their importance for modeling the timing of mutually exclusive statessuch as being in an interaction and being alone (Stoolmiller & Snyder, 2006). This article demonstrates how multistate survival models can be applied to model the dynamics of time-stamped social interaction data.
Second, we consider that there are different types of loneliness relating to different spheres of social interactions (S. Cacioppo et al., 2015;Luhmann et al., 2016). Some individuals feel lonely because they seek a close and intimate attachment (intimate loneliness; S. Cacioppo et al., 2015). Other sources of loneliness can come from a feeling of lacking social relationships and a lacking of belonging in different social networks such as family networks, friendship networks, or classroom networks (relational loneliness; Maes et al., 2019). The third type of loneliness comes from a feeling of lacking connections to similar others in society (collective loneliness; S. Cacioppo et al., 2015). In this study, we differentiate between these three types of loneliness: intimate, relational, and collective loneliness.
Taken together, in this study, we explore relationships between loneliness and (a) the time between interactions (i.e., interaction rate), and (b) the duration of social interaction in a student sample, using novel measurement and analysis methods. To test the robustness of our findings, we report additional analyses, including effects of smartphone usage, extraversion, and the overall level of loneliness on social interaction dynamics. All data and analysis scripts can be accessed at osf.io/c94vs

Participants
Forty-eight students of a computer science program participated in the Dartmouth StudentLife study (Wang et al., 2014). The data of this study are publicly available (studentlife.cs.dartmouth.edu). Unfortunately, no information about participants' gender and age was made available in the dataset. The data owners were also not willing to share this information upon request. However, in the data-introduction paper, Wang et al. (2014) report that the sample was predominantly male (38 participants, 79%). Most students were undergraduate students (63%). Most participants identified as either Caucasian (N = 23, 48%) or Asian (N = 23, 48%). Two participants (4%) identified as African-American. The data collection was approved by the Institutional Review Board at Dartmouth College.
Participants were recruited through a computer science class at Dartmouth College in 2013. All 75 participants of the class were invited. Sixty participants agreed to participate in the study. Twelve students decided to drop the class (N = 5) or the study participation (N = 7). The remaining 48 students participated in the ambulatory assessment part. Of those, 45 participants filled out the T1 loneliness survey. Our analyses on predicting social interaction dynamics from loneliness (RQ1, RQ2), thus were carried out with the sample of 45 participants.

Procedure
The study consisted of online surveys and a 10-week ambulatory assessment phase covering the entire duration of one university term at Dartmouth (United States). Students were incentivized with two prize draws at the end of weeks three and six to win technical devices such as smartphones.
During the online survey that was administered before the ambulatory assessment phase, participants reported on various mental health dimensions (e.g., perceived stress, depressive symptoms, loneliness) and Big Five personality traits (John & Srivastava, 1999).
During the ambulatory assessment phase, participants were asked to carry a smartphone with the StudentLife sensing app. Students that did not have a smartphone were offered to use a Nexus 4s smartphone during the study. Each student received a one-to-one tutorial on what the application does and how to use it. The StudentLife app had two purposes: (1) Prompting participants with experience sampling questionnaires and (2) passively sensing participants' behavior. These experience sampling data were not used in this study because participants did not fill them out regularly. The StudentLife sensing app measured participants' physical movement through GPS and accelerometer sensors (for more details, see Wang et al., 2014). Relevant to this study is the automated classification of conversation data through the smartphone's microphone, which provides information about the start and the end time of participants' conversation periods. From this data, the number of interactions and their duration can be derived.

Measures
Social Interactions. Interactions can be defined as "reciprocal influence of individuals upon one another's actions when in one another's immediate physical presence" (Goffman, 1956, p. 18). In the present study, we used automatically recorded conversations as indicators of interactions; thereby, we are confident that there is an exchange between people and that these people are in each other's immediate physical presence.
Using the input from the smartphone's microphone, a machine learning algorithm was applied to classify the audio signal into periods in which participants were part of a conversation. The sensing algorithm ran locally on the smartphone so that no audio data had to be transmitted to the researchers. The algorithm was shown to validly and reliably detect social interactions in prior studies (Lane et al., 2011;Rabbi et al., 2011). For instance, Lane et al. (2011) compared self-reports of interactions with those obtained through the smartphone's microphone and the sensing algorithm. They concluded that because of the algorithm's sensitivity towards classifying audio snippets that are "ambient sound from activities that are not actual conversations (e.g., when the user is watching TV)" (p. 7) as interactions, 14% of the detected interactions were not reported in the selfreports. Evaluations of similar detection algorithms show a comparably high accuracy, with up to 92% correctly identified interactions (Feese & Tröster, 2013). Conversation information during lectures and class meetings was removed from the dataset (Wang et al., 2014).
Loneliness. Loneliness and its three subscales (intimate, relational, and collective loneliness) were measured with the Revised UCLA Loneliness Scale (Russell, 1996). The scale consists of 20 items assessing how often the participant felt the way described in the item. Subscales for intimate, relational, and collective loneliness were calculated in line with Luhmann et al. (2016). Intimate loneliness was measured with three items such as "there are people I feel close to" (α = .87), relational loneliness with three items such as "there are people I can talk to" (α = .67), and collective loneliness with three items such as "I have a lot in common with the people around me" (α = .78). Cronbach's α for the entire scale (omitting the subscales), as used in sensitivity analyses, was .93. The average loneliness score was 1.96 (SD = 0.58).

Control variables
Smartphone usage. During the ambulatory assessment phase, the smartphone also collected data on its use (i.e., when the screen was unlocked). To assess the association between short-term phone usage and the rate and duration of social interactions, we coded a dummy variable as one when the participant was using the smartphone within the last hour before the start of a social interaction event, and zero otherwise. On average, participants were using their smartphones in 74.5% (SD = 10.4%) within the hour before the start of a social interaction event.
Extraversion. Prior to the ambulatory assessment phase, participants responded to the Big Five Inventory (John & Srivastava, 1999), which assesses the Big Five personality traits. The Extraversion subscale consists of eight items such as "I see myself as someone who is talkative", measured on a five-point scale ranging from "Disagree strongly" (1) to "Agree strongly" (5). The average of these items constitutes the Extraversion score, which in this sample was 2.98 (SD = 0.72). The Extraversion score is only used in the sensitivity analyses reported in the Supplementary Materials.

Analytical strategy
Is loneliness associated with (a) the time between social interactions (i.e., interaction rate; RQ1) and (b) the duration of social interactions (RQ2)?. To model the effects of loneliness on the dynamics of social interactions, we use multistate models (Stoolmiller & Snyder, 2006). Multistate models are based on Cox's proportional hazard model (Cox, 1972), which estimates how independent variables are associated with the probability of observing a single type of event at a particular point in time (e.g., social interaction event, illness event).
In multistate models, the transition times between two or more mutually exclusive states are modeled. In our data, participants transition between "alone" and "interaction" states, as illustrated in the data excerpt shown in Table 1. Table 1 shows the first six observations for the individual with ID = 1. The start and end columns indicate from when to when that individual was in a particular state (see state column). In our case, the transition times from being in an "alone" state to an "interaction" state (joining social interactions) and the transition times from being in an "interaction" state to an "alone" state (leaving social interactions) are being modeled. Put simply, the dependent variable of the multistate analysis is the time until a transition is observed (see the last column in Table 1). By estimating the relative risk (i.e., hazard ratio) of a covariate (e.g., loneliness) on individuals' tendencies to change to another state, we can assess how loneliness is related to joining and leaving social interactions. In addition, we can take the participant's past social behavior and temporal dynamics into account (e.g., changes in fluctuations in the structure of interactions governed by the time of the day). Multistate models also take into account that observations are nested within individuals by estimating robust standard errors (Putter et al., 2007). For a detailed mathematical introduction to multistate models, see Putter et al. (2007).
Although the application of multistate models is common in neighboring disciplines such as medicine and sociology (e.g., Bijwaard, 2014;Putter et al., 2007), they have rarely been applied within psychological research. One of the few exceptions is the analysis by Stoolmiller and Snyder (2006), who study emotional expression within child-parent interactions in a laboratory setting.
To test RQ1, we add the three loneliness subscales measured at T1 as predictors of transitions from the alone to the interaction state (rate) and from the interaction to the alone state (duration). Further covariates, controlling for general social interaction dynamics, consist of dummy variables for the time of the day (night, morning, afternoon, evening), whether or not the interaction took place on the weekend (dummy variable with 0 = weekday, 1 = weekend), window variables capturing the number of social interactions in the past two and 24 hr, and the mean duration of social interactions in the past 2 hr. These variables account for tendencies of interactions to occur at a higher rate during the daytime (than at night) and for the influence of previous interactions. With these variables, we can control for fluctuations in interaction patterns due to situational factors such as time of day. For example, the covariate entailing the number of interactions within the past 2 hr captures a tendency of individuals to repeatedly interact within a short (i.e., two-hour) time window. This can, for instance, be because the smartphone is placed somewhere where there are frequent signal interruptions (e.g., in a backpack) or because the person had many short interactions within the given time window (e.g., because they were attending a social mixer). This variable thus captures some unobserved heterogeneity between situations. Such time and window-control variables are commonly used when modeling interaction dynamics (Hoffman et al., 2019;Stadtfeld & Block, 2017).
We further control for the effect of short-term effects of student's smartphone usage on social interactions by including a variable in the model that captures whether or not the student had been using the smartphone in the previous hour (value = 1) or not (value = 0). We did not compute the "time in state" for the overnight observation; hence, the time gaps between the last measure in the evening and the first measure in the morning are not part of the analyses. RQ1: Is loneliness associated with the time between social interactions (i.e., interaction rate)? Table 2 shows the results of the multistate analysis. The second to fourth columns in Table  2 show the estimates for the transitions from the alone state to the interaction state (i.e., the effect of interaction rate). Although all estimates with regards to loneliness were negative, indicating a lower tendency of individuals with higher loneliness scores to move from an alone state to an interaction state, none of the three loneliness subscales measured were significantly associated with the rate of joining social interactions. Other variables in the model, however, were important predictor variables: Students were more likely to engage in social interactions during the day, if they were in more interactions in the past 2 hr and 24 hr, and if they were in longer interactions within the past 2 hr. Students that were on their smartphones prior to the social interaction were less likely to engage in social interactions.

RQ2: Is loneliness associated with the duration of social interactions?
Columns five through seven in Table 2 show the estimates for the transitions from the interaction state to the alone state (i.e., the leaving rates or the interaction duration). In this sub-model, only relational loneliness measured was significantly associated with the rate of leaving social interactions. The effect of β ¼ 0:099 suggests that individuals with one unit higher in relational loneliness at T1 were expðβÞ ¼ 1:10 times more likely to leave social interactions than those with a one-unit lower value in relational loneliness. In other words, those individuals at the high end of the relational loneliness scale (value = 4) are 35% ðexpð3 * βÞ ¼ 1:33Þ more likely to leave an interaction at any given timepoint than an individual at the low end of the scale (value = 1), thus, the duration of their interactions was shorter. With regards to the covariates, students left social interactions more likely on weekends, less likely during the day, and less likely if they had longer interactions within the past 2 hr. Also, those who had more interactions within the past 2 hr were slightly more likely to leave social interactions. Being on the phone prior to the start of the social interaction was associated with a lower risk of leaving social interactions.

Discussion
In this article, we demonstrated how fine-grained social interaction data collected through participant's smartphones can be modeled with multistate survival models to provide new insights into the relationship between social interaction dynamics and psychological phenomena, such as loneliness. For this, we used a multi-method combination of novel, automatically sensed social interaction data and survey methods. Using data from a longitudinal ambulatory assessment study design, we investigated how types of loneliness (intimate, relational, collective) predict the rate (RQ1) and duration (RQ2) of social interactions.
Our analyses provide no suggestion that intimate and collective loneliness predict the time between social interactions (i.e., interaction rate) or the duration of social interactions. Relational loneliness was associated with leaving social interactions more quickly but not with joining them at a higher rate. In other words, we found evidence that relational loneliness is related to shorter social interactions but not necessarily to the rate of interactions. Earlier research using self-reported interactions suggested that loneliness is associated with less time spent in interactions (Lee & Ko, 2017;Wheeler et al., 1983). In the present study, we cannot confirm this notion. Only for relational loneliness, we found that it may be related to fewer interactions. This could indicate that especially people who are relationally lonely are more withdrawn from social interactions (Watson & Nesdale, 2012). Relational loneliness reflects a feeling that one does not belong to a peer group. Many interactions in students' lives take part in group settings (Elmer & Stadtfeld, 2020). Thus, it is likely that especially those students who feel a lack of belonging are likely to leave such interactions more quickly. As earlier studies did not differentiate between different types of loneliness, it is not clear whether their results were also mainly driven by students experiencing relational loneliness. Yet, in a sensitivity analysis, we also examined loneliness as a unidimensional construct, comparable to previous studies. Here, we did not find significant associations with loneliness (see Supplementary Materials for details). Thus, the differences could also reflect a difference between self-reported and automatically detected interactions, showing the value of using different measures to examine such questions.
Beyond the substantive findings on the relationship between loneliness and social interactions, this article contributes to the literature on social-interaction research by demonstrating how social interactions can be studied using passive-sensing methods and multistate survival analyses. A particular advantage of passively sensing social interactions compared to survey-based measures is that social behavior can be measured in real-life settings without any input required from the participant, thus not interrupting the natural flow of day-to-day life (e.g., by filling out an ESM survey). As a result, specific measurement biases are reduced, as participants seem to be prone to miss reporting experience sampling surveys when in a social interaction (Sun et al., 2021).
Although passive-sensing and digital phenotyping are promising avenues for psychological science to study people unobtrusively in real life (Torous et al., 2016), their application still warrants more systematic validation studies (e.g., Elmer et al., 2019), data processing guidelines (Montag et al., 2020), and tailored analysis methods (Onnela, 2021).
A limitation of our empirical demonstration is that we examined a small, specific population of individuals (i.e., 45 male-majority computer science students). Hence, we cannot generalize our findings to other groups of populations. Future studies should examine the interplay between social interaction dynamics and loneliness in larger and more diverse populations.
Another limitation is that we only measured trait loneliness. It would be relevant to investigate the interplay between social interactions and state loneliness on a moment-tomoment basis. Ambulatory assessment methods using automated social interaction sensor technologies in combination with experience sampling methodsthat allow assessing self-reports of state loneliness and qualitative characteristics of social interactionsthus hold great potential in the investigation of short-term (weekly, daily, momentary) loneliness and social interaction dynamics.
Finally, it would have been interesting to examine not only how loneliness predicts social interaction dynamics but also how social interaction dynamics predict changes in loneliness. Although in our study loneliness was also measured after the ambulatory assessment phase, the number of participants with valid loneliness change scores (N = 36) was very low, not yielding sufficient statistical power to study changes in loneliness. Using procedures to conduct analyses with small sample sizes (e.g., bootstrapping), we, nevertheless, report an exploratory analysis on how social interaction dynamics are associated with changes in loneliness in the Supplementary Materials. These exploratory analyses suggest that individuals who engage in longer interactions tend to report a decrease in collective loneliness (for a detailed discussion, see Section three in the Supplementary Materials).

Conclusion
Drawing on smartphone sensors to study fine-grained social interaction dynamics in daily life settings can advance our understanding of people's social life. This study advances knowledge on social interaction dynamics with a unique multi-method study design by exemplifying how the richness of sensor data extracted from individuals' smartphones can be used to draw conclusions about the interrelations between social interaction dynamics and loneliness.

Acknowledgments
The authors would like to thank Laura Bringmann, Marie Stadel, and the reviewers for inputs on various versions of the manuscript.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/ or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work of Timon Elmer was supported by a personal grant from the Swiss National Science Foundation (Grant No. P2EZP1_188022). The work of Gerine Lodder was supported by a personal grant from the Dutch Research Council (Grant No. 016.Veni.195.186).

Open research statement
As part of IARR's encouragement of open research practices, the author(s) have provided the following information: This research was not pre-registered. The data used in the research are available. The data can be obtained at: https://osf.io/c94vs. The materials used in the research are available. The materials can be obtained at: https://studentlife.cs.dartmouth.edu

Supplemental Material
Supplemental material for this article is available online. Note 1. It is important to note that being alone (i.e., solitude) and feeling lonely are two distinct constructs. Whereas solitue is the objective social state of being alone, loneliness describes the subjective feeling of a discepancy between the actual and desired social state (Peplau & Perlman, 1982).