The details of past actions on a smartphone touchscreen are reflected by intrinsic sensorimotor dynamics

Unconstrained day-to-day activities are difficult to quantify and how the corresponding movements shape the brain remain unclear. Here, we recorded all touchscreen smartphone interactions at a sub-second precision and show that the unconstrained day-to-day behavior captured on the phone reflects in the simple sensorimotor computations measured in the laboratory. The behavioral diversity on the phone, the speed of interactions, the amount of social & non-social interactions, all uniquely influenced the trial-to-trial motor variability used to measure the amount of intrinsic neuronal noise. Surprisingly, both the motor performance and the early somatosensory cortical signals (assessed using EEG in passive conditions) became noisier with increased social interactions. Inter-individual differences in how people use the smartphone can help thus decompose the structure of low-level sensorimotor computations.


Subjects
Participants (n = 57) were recruited using campus-wide announcements at the University of Zurich and ETH Zurich between December 2014 and August 2015. The sample consisted of subjects within a narrow age group [26 females; 23 (20th percentile) to 28 (80th percentile) years old]. The age at which the volunteers reportedly began using the phone was also narrowly distributed [19 (20th percentile) to 25 (80th percentile) years old]. Previous reports on inter-individual variability in cortical somatosensory signal-to-noise ratio, touchscreen use-dependent plasticity and use-dependent reduction in sensorimotor variability employed a sample size between 15 -28 [1][2][3][4] . We hypothesized a weaker impact of the social touches on the touchscreen than the explanatory variables studied before, i.e., deliberate laboratory practice, touchscreen use in general and the presence of autism spectrum disorder. Therefore, we doubled the sample size and employed more regression parameters than the previous studies to increase the sensitivity of our analysis. All experimental procedures were conducted according to the Swiss Human Research Act approved by the cantons of Zurich and Vaud. The procedures also conformed to the Declaration of Helsinki. All of the experimental protocols were approved by the ethical committees of cantons of Zurich and Vaud. The volunteers provided written and informed consent before participating in the study. Righthandedness, and ownership of a non-shared touchscreen smartphone were pre-requisites for participation. The handedness was further verified by a questionnaire 5 .

Smartphone data collection and analysis
A custom-designed background App was installed on the volunteers' smartphones to quantify the touchscreen behavior (App available through QuantActions GmbH, Lausanne, Switzerland). Briefly, the App recorded the timestamps of touchscreen events and the label of 3 the App on the foreground. The App recorded the touchscreen events with an interquartile error range of 5 ms. Data were stored locally and transmitted by the user at the end of the observation period via secure email. Smartphone data were processed using custom written scripts on MATLAB (MathWorks, Natik, USA). The number of touches on each App category ("Social", "Non-social", or "Uncategorized", see figure and list below for a sample) was divided by the length of the recording period to determine the number of touches per day. Apps that functioned to enable social interactions between a circle of friends or acquaintances were labeled as "Social" and Apps that clearly did not feature this functionality were labeled as "Non-social".
Apps whose label was poorly registered by the operating system, untraceable on Google Play, or that contained both social and non-social properties, e.g., gaming Apps with social messaging, were labeled as "Uncategorized". The touches that were separated by less than 50 ms were eliminated from further analysis as considered an artefact. The rate of touchscreen events was determined as

Simple reaction time task and analysis
Volunteers responded to a brief (10 ms) tactile pulse by depressing and releasing a button mounted on a micro switch. The tactile pulse was presented by using a computercontrolled solenoid tactile stimulator (Heijo Research Electronics, London, UK). The stimulating magnetic rod (2 mm in diameter) generated a supra-threshold 2-mN contact. The finger stimulated and used to perform depressing and releasing actions was the thumb. The micro switch (extracted from RX-300 optical mouse, Logitech, Lausanne, Switzerland) was operated by press-downwards and release-upwards movements of the thumb. The mechanical parts for the release-upwards movement malfunctioned in two volunteers and in one of the two 5 the parts for press-downwards malfunctioned as well, therefore the corresponding data were eliminated from further analysis.
The task was repeated 500 times within an experimental session, with 2 min break in the middle of the session. The pulses were delivered with 3 ± 1 s gap and the button presses generated analogue signals that were digitized at 1 kHz. The reaction time (time taken to execute button depression) and movement time (time taken to execute button release) were fitted with three ex-Gaussian parameters. This form of fitting separates skewed reaction time data into a Gaussian region and an exponential region. Mean of the Gaussian region was captured by parameter µ, and variability of the Gaussian region by parameter σ. The exponent τ captured unusually slow responses. The parameters were estimated using previously described MATLAB scripts 6 . 6

EEG data acquisition and analysis
A subset of volunteers (randomly chosen, n = 43) participated in EEG experiments. The volunteers were seated upright for the EEG and the right, stimulated, hand was concealed by using a textile baffle. Computer-controlled solenoid tactile stimulator was attached to the right thumb tip. To ease the tedium of the hours-long measurements required for gathering the tactile evoked potentials data (SSEPs), volunteers were allowed to view a movie (David Attenborough's Africa series); white noise, played to mask the sound generated by the stimulator, was mixed with the movie soundtrack and delivered through headphones. Due to technical malfunction during the measurements, one volunteer was eliminated from further analysis. The number of trials was set to 1000 and the stimuli were separated by 2-4 s. A nonalcoholic and caffeine-free drink break was offered every 10 min, for a maximum of 10 min.
To record the EEG signals, 64 electrodes were used (62 equidistant scalp electrodes and two ocular ones), against a vertex reference (EasyCap, Herrsching, Germany), as previously reported 4 . The electrode locations were digitized in a 3D nasion-ear coordinate frame (ANT Neuro and Xensor software, Netherlands) for a representative volunteer. The signals were recorded and digitized by BrainAmp (Brain Products GmbH, Gilching, Germany) at 1 kHz.
Offline data processing was accomplished using EEGLAB, a toolbox designed for EEG analysis on MATLAB 7 . The data were referenced to the average of all scalp electrodes and band-pass filtered between 1 and 80 Hz. "Epoched" trials over 80 μV were eliminated to remove large signal fluctuations, e.g., from eye blinks. The data were further processed using independent component analysis. Components dominated by eye movements and other measurement artifacts were eliminated by using the EEGLAB plug-in SASICA 8 . The signalto-noise ratio was estimated using the linear modeling toolbox LIMO EEG (EEGLAB plug-in) 9 . In this toolbox, R 2 values were estimated for each volunteer based on single trials, as a sum of squares of the putative signal divided by the sum of squares of the residuals. Essentially, the 7 predominant notion in the sensory evoked potential research field is that the average over multiple trials extracts a signal that is otherwise hidden in the measurement noise and background neuronal processes 10 . The signal-to-noise ratio in this case captures how well the estimated mean (putative signal) represents the data. To normalize the data across the sampled population, the square root of the putative signal-to-noise ratio was used for subsequent analyses using multiple linear regression.

Correlational statistics
All analyses involving the reaction and movement times were conducted by robust-bisquare-multiple linear regression analysis (implemented in MATLAB). The fitted model was evaluated using ANOVA with a level of significance set at p = 0.05. The following main regression equation was used: As a control, we repeated the analysis with shuffled App categories. Essentially, for the original analysis, the Apps were labeled as "Social", "Non-social", and "Uncategorized" Plots for displaying multiple linear regression results in two dimensions (adjusted response plots) were generated using a built-in MATLAB function (plotAdjustedResponse).
Formulation of this plotting method and its advantages are described elsewhere 11 .
The EEG data were correlated with touchscreen parameters using robust regression, the iterative least squares method (implemented in LIMO EEG). The correlation coefficients were estimated across all electrodes and for the time period from -30 to 200 ms relative to the stimulation onset. The regression statistics were corrected for multiple comparisons by using 1000 bootstraps and spatiotemporal clustering, as implemented in LIMO EEG.