https://orcid.org/0000-0002-1100-1493
Figures
Abstract
Background
Conceptualising internet addiction and assessing its symptoms has presented a significant challenge for researchers over the past 25 years. Recently, the Internet Disorder Scale (IDS-15), which is based on the criteria for Internet Gaming Disorder (IGD) from DSM-5, has emerged as a promising instrument to assess internet addiction. The main objective of the present study was to evaluate the psychometric properties of the Slovenian IDS-15.
Methods
The sample was recruited from the National Survey on the Use of Tobacco, Alcohol and Other Drugs that was conducted in 2018 on a nationally representative sample (N = 16,000; age range: 15–64 years; 62.4% response rate). The final sample comprised 9,161 participants, with 80.9% reporting having used the internet at least once a week (n = 7,413). A structured questionnaire was designed and internet addiction was assessed using the IDS-15.
Results
Confirmatory Factor Analysis showed acceptable fit to the proposed four-factor structure of the IDS-15. The reliability, and criterion, convergent and discriminant validity were also found to be adequate with a notable exception of the first item of the scale, as shown by its lower factor loading and higher variability. Additionally, latent profile analysis was used to distinguish between internet users with low (n = 3,818; 51.5%), medium (n = 3,111; 42.0%) and high (n = 484; 6.4%) addiction risk. Furthermore, the high-risk class was associated with higher IDS-15 factor scores, higher frequency of internet use in leisure time, and lower age of first internet use.
Conclusions
The present study provides new insights about the strengths and shortcomings of the IDS-15. Moreover, the results provide an insight into the prevalence of internet addiction in Slovenia, as well as associations with other potential factors. The results serve as the basis for further analyses on internet addiction epidemiology, policymaking activities, and design for targeted public health interventions in Slovenia.
Citation: Žmavc M, Pontes HM, Griffiths MD, Selak Š (2022) Psychometric properties of the Slovenian version of Internet Disorder Scale–IDS-15. PLoS ONE 17(10): e0276663. https://doi.org/10.1371/journal.pone.0276663
Editor: Frantisek Sudzina, Aalborg University, DENMARK
Received: May 20, 2022; Accepted: October 11, 2022; Published: October 21, 2022
Copyright: © 2022 Žmavc et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript. Any additional data that support the findings of this study are available from the Slovenian National Institute of Public Health upon request (https://www.nijz.si/en).
Funding: The study was funded by the Ministry of Health of the Republic of Slovenia within the regular annual budget. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Excessive use of digital technologies and associated harm to users’ wellbeing has received significant attention from researchers over the past 25 years. More specifically, internet addiction (IA) has been a highly debated topic, lacking consensus on many of the core aspects of the concept. This is evident even at the terminological level, where ‘internet addiction disorder’ [1], ‘pathological internet use’ [2], ‘problematic internet use’ [3], and ‘internet use disorder’ [4] have been commonly used as alternatives to IA [5]. IA broadly refers to “excessive or poorly controlled preoccupations, urges or behaviours regarding internet use that lead to impairment or distress” [6]. The authors describing IA-related concepts have drawn from various theoretical foundations and empirical findings, such as the cognitive-behavioural approach [2], the components model of addiction [7], neuropsychological and neuroimaging findings [8], the DSM-IV criteria for gambling addiction [5], impulse-control disorders in general [3] or the DSM-5 criteria for Internet Gaming Disorder (IGD) [9].
According to Griffiths et al. [10], there is an important distinction between addiction to the internet and addictions on the internet. Here the authors assert that excessively using the internet as a medium to access particular addictive content does not equate to being addicted to the internet [11, 12]. On the other hand, Király et al. [13] argued that the internet is often more than a medium and it significantly changes aspects of the original activity. For example, online gaming has a stronger social dimension (and consequently, a stronger pull) than offline gaming due to social interactions between players. For some users, the internet as a whole (through anonymity, diversity of content, interactivity and accessibility) provides an environment that consistently facilitates feelings of belongingness, which can be highly psychologically and physiologically rewarding [10]. The same could be argued for feelings of autonomy and competence, two other basic psychological needs [14]. Davis [2] distinguished between specific and generalised Pathological Internet Use (PIU), describing the latter as a general, multidimensional overuse of the internet, which may include wasting time without a clear objective. He asserted that the pathology of individuals with generalised PIU would likely not even exist in the absence of the internet. The dichotomy of generalised versus specific PIU has been supported by many researchers (e.g. [15, 16]). In a recent literature review, Kamolthip et al. [17] concluded that these conditions require different treatments and should be evaluated independently.
Moreover, while many recent conceptualisations of PIU have focused on specific PIU, such as social media addiction [18], IGD [19], internet gambling addiction [20] and internet pornography addiction [21], research efforts towards a generalised PIU concept are arguably equally important [22]. Furthermore, the emerging and closely related concept of smartphone addiction should also be noted. Montag et al. [4] argued that devices such as smartphones are characterised by specific behavioural usage patterns and technological features, and that a particular device may be preferred for a specific (type of) online activity. Following this logic, they propose the following dichotomy; PIU predominantly mobile and PIU predominantly non-mobile.
The lack of consensus in the conceptual background of IA has also been evident from the various attempts to assess the phenomenon. Methodological strengths and shortcomings of existing instruments assessing IA have been described in a review by Király et al. [23], where inconsistencies between instruments have been observed at many levels (i.e., theoretical basis, factor structures and psychometric properties). This led Pontes and Griffiths [24] to develop a new psychometric tool (i.e., Internet Disorder Scale; IDS-15) by adapting the DSM-5 criteria IGD [19] to that of IA. The IDS-15 contains 15 items, defining four qualitatively distinct IA-related domains: (i) escapism and dysfunctional emotional coping, (ii) withdrawal symptoms, (iii) impairments and dysfunctional self-regulation, and (iv) dysfunctional internet-related self-control.
To date, three IDS-15 validation studies have been published. The first IDS-15 validation study by Pontes and Griffiths [24] included 1,094 participants aged from 16 to 70 years old (Meanage = 33 years) and collected the data utilising opportunistic sampling. The scale demonstrated acceptable factorial, convergent, and discriminant validity. The first-order Confirmatory Factor Analysis (CFA) including its four-factor model showed adequate fit to the data. Internal reliability estimates for the four dimensions of IDS-15 ranged from .79 to .85. The overall IDS-15 score was positively associated with weekly time spent on the internet, age of first internet use, and relationship status. Additionally, high addiction risk was negatively associated with age and positively associated with frequency of cigarette smoking while drinking alcohol frequency was unrelated to IA risk.
The Italian IDS-15 validation study [25] included 1,094 participants aged from 16 to 70 years old (Meanage = 33 years) and collected the data utilising opportunistic sampling. The scale demonstrated adequate construct validity through a second-order CFA and showed acceptable indices for both first- and second-order structural models. Criteria for satisfying convergent and discriminant validity were also met. Internal reliability estimates ranged from .76 to .92 for each dimension of the IDS-15. The analyses also supported measurement invariance of the model across genders. Furthermore, positive correlations were observed between IDS-15 scores, IGD, and social media addiction.
Similar to the previous studies, the Persian IDS-15 validation study [26] included 1,272 high-school students (Meanage = 15.5 years) and collected the data utilising a multistage randomised sampling process. The scale demonstrated acceptable construct validity based on the indices derived from a second-order CFA. The authors reported excellent fit of all items using Rasch analyses. Internal reliability for scale dimensions was sufficient (.76-.90), as was the test-retest reliability of each item (.70-.88). The study found that being female and older were among the factors associated with IA.
The three IDS-15 validation studies provide a solid starting point towards establishing IDS-15 as a consistently reliable and valid tool to assess IA based on the DSM-5 framework. However, Koronczai et al. [27] proposed six criteria which any suitable IA instrument should meet: comprehensiveness, brevity, reliability and validity for different genders, age groups, cultures, and validation on clinical samples. The current empirical evidence for IDS-15 is still lacking for criteria 4–6 because the validity for various age groups and cultures has yet to be established, and showing how the IDS-15 score relates to clinical diagnoses of IA is still required. Therefore, the aim of the present study was to provide further evidence for the reliability and validity across genders and across cultures using a large representative dataset from a national survey conducted in Slovenia. Additionally, the study explicitly tested whether the proposed factor structure model for the IDS-15 fitted the data across various age groups. Following the example of the validation studies summarised above, users of varying risks for IA were differentiated using latent profile analysis (LPA). The intention was to obtain an approximate estimate of addicted internet users and to aid future efforts for determining a cut-off score for IA on the IDS-15. Lastly, the study calculated the correlations between IDS-15 scores (as well as risk category) and sociodemographic characteristics, gaming, gambling, and drug, alcohol and tobacco use.
Methods
Participants
The sample for the IDS-15 validation study comprised 7,413 participants, all of who reported using the internet at least once a week (47.0% males, n = 3,484, and 53.0% females, n = 3,929). Participants’ age ranged from 15 to 64 years, with an average age of 39.2 years (SD = 13.7 years). Education levels also varied, with 11.8% (n = 875) reported having achieved primary or lower education, 50.4% (n = 3,736) secondary education, and 37.8% (n = 2,802) tertiary education. In terms of employment status, 67.7% (n = 5,019) were employed, 18.3% (n = 1,357) were students, 6.9% (n = 511) were unemployed and 7.1% (n = 526) were retired. Each of the 12 Slovenian regions was represented in the sample, although more than a quarter of participants lived in the Central Slovenia Statistical Region.
Procedure
Sampling.
The sample was recruited from the National Survey on the Use of Tobacco, Alcohol and Other Drugs that was conducted in 2018, in collaboration with the Statistical Office of Slovenia. A total of 16,000 inhabitants of Slovenia living in private (not institutionalised) households (age range: 15–64 years) were invited to participate (8,000 in the spring and 8,000 in the autumn of 2018). The final sample comprised 9,161 participants (62.4% response rate). A two-stage sampling explicitly stratified by size and type of settlement and implicitly by statistical regions was carried out. Data was collected using a mixed method via Computer-Assisted Web Interviewing (CAWI) and Computer-Assisted Personal Interviewing (CAPI). Namely, all individuals selected in the sample received a notification letter inviting them to complete the survey over the web. In the letter, they were informed about the voluntary participation in the survey, about the purpose and procedure of the survey, and assured that their responses would be confidential and carefully protected. By continuing with the survey questions, they consented to participate in the study. The web survey was available online throughout the data collection period. Individuals who did not respond were additionally contacted by an interviewer and invited to participate in the study. Following their agreement to participate, the data collection took place in their homes using computer-assisted face-to-face interviews. Thorough instructions for survey application were given to the interviewers. The structured questionnaire was designed for data collection using the methodology of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) to ensure comparability with other studies within the European Union. Additionally, assessment tools for IA, gaming, and gambling were included in the survey. IA was assessed using the IDS-15. A majority of the participants (80.9%) reported using the internet at least once a week (n = 7,413).
Adaptation of the Slovenian IDS-15.
The IDS-15 was translated from the original English version into Slovenian by two experts in the field of digital addiction. This was then additionally checked by the lead methodologists of the research project (experts with a background in psychology, psychometrics, and sociology and with many years of experience in coordinating the implementation of national surveys in public health). The translation process followed the three-step Eurostat translation protocol [28]. No adaptations due to cultural differences were deemed necessary.
Measures
Internet addiction.
The IDS-15 is a psychometric tool, which assesses the severity of IA among individuals, based on APA framework for IGD proposed in the DSM-5 [24]. The items refer to users’ online leisure activity, excluding internet use for work or school, from any device with internet access in the past 12 months. The IDS-15 contains 15 items comprising four IA-related domains: Escapism and Dysfunctional Emotional Coping (EDEC), Withdrawal Symptoms (WS), Impairments and Dysfunctional Self-Regulation (IDSR), and Dysfunctional Internet-related Self-Control (DISC). Participants rate items using a five-point Likert Scale, from 1 (“Strongly disagree”) to 5 (“Strongly agree”). Total scores range from 15 to 75, with higher scores indicating higher severity or higher risk of IA. The psychometric properties of the Slovenian IDS-15 are reported in the Results section.
Gaming disorder.
The Internet Gaming Disorder Scale (IGDS9-SF) [29], a nine-item screening tool, was used to assess the severity of IGD and its harmful effects by examining online and offline gaming over the past 12 months. The items (e.g., “Do you systematically fail when trying to control or cease your gaming activity?”) are answered using a five-point Likert scale from 1 (“Never”) to 5 (“Very often”). Total scores range from 9 to 45, with higher scores indicating higher severity of IGD [30, 31]. The IGDS9-SF is an established cross-culturally validated measure to assess IGD [30, 31]. The Slovenian version was adapted and validated by Pontes et al. [32] in a study on IGD among Slovenian youth. Cronbach’s α of the Slovenian IGDS9-SF in the present study was .89.
Gambling disorder.
The Berlin Inventory of Gambling Behavior–Screening (BIG-S) [33], a reliable and valid screening tool, was used to assess gambling addiction. The inventory consists of 14 questions, which represent an operationalisation of the ten DSM-5 criteria for gambling disorder. The items (e.g., “Have you repeatedly spent more time or money on gambling than originally intended?”) are answered in a dichotomous format (yes/no) and refer to gambling-related behaviour throughout the lifespan. Total score ranges from 0 to 10, with a score of 5 and above being indicative of gambling disorder. The Slovenian BIG-S was adapted according to the same protocol as the IDS-15 (see ‘Adaptation of the Slovenian IDS-15’). Cronbach’s α of the Slovenian BIG-S was .81.
Statistical analysis
The statistical methods used comprised the following; (i) frequencies and descriptive statistics of key variables, (ii) CFA using robust maximum likelihood estimation (MLR) for evaluating construct validity, along with average variance extracted (AVE) and maximum shared variance (MSV) indices for determining convergent and discriminant validity, (iii) CFA for multiple groups to evaluate measurement invariance between genders and age groups (iv) internal consistency and composite reliability (CR) indices for purposes of reliability estimation, (v) bootstrapped correlation matrix with the associated bias-corrected accelerated 95% confidence interval for evaluating criterion validity, and (vi) LPA with associated fit indices (Aikake Information Criterion—AIC, Bayesian Information Criterion—BIC, Bootstrapped Likelihood Ratio Test—BLRT p-value, and entropy) to identify latent groups of internet users. LPA is a statistical technique used to identify groups of individuals with similar patterns of scores in any set of variables [34].
To evaluate the model fit given by CFA, standard thresholds for acceptable fit were referred to (comparative fit index (CFI) > .90, Tucker-Lewis index (TLI) > .90, root mean square error of approximation (RMSEA) < .10, standardised root mean square residual (SRMR) < .10) and good fit (CFI > .95, TLI > .95, SRMR < .08, RMSEA < .08) as proposed by Hooper et al. [35]. The obtained AVE and MSV values were evaluated based on suggestions of Hair et al. [36]. Lastly, to assess measurement invariance, the fit of Model 1 was evaluated with groups based on gender and age, with constrained factor structure (and loadings, intercepts) across groups. Sufficient configural (and metric, scalar) invariance was concluded, if the resulting changes in CFI, RMSEA and SRMR fit indices were within the thresholds recommended by Chen [37] (ΔCFI < .010, ΔRMSEA < .015, ΔSRMR < .010). All statistical procedures were performed with the R software environment for statistical computing and graphics, version 1.4.1717 [38], using “semTools” [39], “lavaan” [40] and “tidyLPA” [41] statistical packages.
Ethical considerations
This study uses data from the National Survey on the Use of Tobacco, Alcohol and Other Drugs 2018, conducted by Slovenian National Institute of Public Health, which is an authorised producer of national health and health care statistics (playing a role of other national authority in the official statistics). The legal framework for this survey was set in the National Statistics Act and Annual Programme of Statistical Surveys for 2018 which was approved by the Information Commissioner of Slovenia. Thus, since all national health and healthcare statistics are carried out in the same manner, no Institutional review board or ethical committee was needed to additionally approve or waive the study.
Results
Descriptive statistics
Table 1 shows descriptive statistics of the total IDS-15 score and the average item score for each of its four dimensions in the overall sample. Additionally, weekly internet use in leisure time was reported: less than seven hours weekly (n = 4,448, 60.0% of users), 8–14 hours (n = 1,542, 20.8%), 15–20 hours (n = 556, 7.5%), 21–30 hours (n = 378, 5.1%), 31–40 hours (n = 193, 2.6%), and more than 40 hours (n = 296, 4.0%).
Reliability
Reliability of the scale, calculated with McDonald’s ω, was .94. Table 2 shows the reliability coefficients (Cronbach’s α, McDonald’s ω, and composite reliability) for each of the four factors of the IDS-15. All coefficients indicated adequate internal reliability (> .70) of the IDS-15 factors, although EDEC was the least reliable of the four.
Factorial validity
CFA was performed to assess the validity of the proposed four-factor structure of the IDS-15 (i.e., Model 1). Additionally, the analysis evaluated the fit of a one-factor model comprising all 15 items (i.e., Model 2). Table 3 shows the goodness of fit indices, as well as the models’ overall evaluation (see ‘Statistical analysis’). Model 1 provided the best (although still only acceptable) fit to the data. Factor loadings of all items (see Table 6) were statistically significant. The poor fit of Model 2 suggests that the latent structure of IA as assessed by the IDS-15 is likely not unidimensional.
For the purposes of establishing whether the IDS-15 assesses the same construct similarly across different subgroups (i.e., measurement invariance), CFA for multiple groups was utilised (using Model 1). In the case of gender, the changes of CFI, RMSEA and SRMR fit indices when constraining factor structure, loadings or intercepts (Table 3), were within recommended thresholds (see ‘Statistical analysis’). When inspecting measurement invariance across age groups, the changes of fit indices were only acceptable when constraining factor structure and loadings, but not intercepts (i.e., CFI > .010)
Convergent and discriminant validity
Convergent validity, shown by AVE values (see Table 4) over .50 is somewhat questionable in the case of the EDEC factor, while being satisfactory for the remaining latent factors. Discriminant validity of the factors was acceptable, as evidenced by values of MSV, which were all lower than AVE values for each factor. As an additional indication of discriminant validity, the square roots of AVE for each factor (i.e., bold diagonal values in Table 4) were higher than its correlation with other factors.
Criterion validity
Table 5 shows the correlations between each IDS-15 dimension, total score or IA risk class (see ‘Latent profile analysis’ below), and key demographic or criterion variables. Importantly, IDS-15 dimensions show positive correlations with the IGD (r > .20), positive correlations with hours of weekly internet use (r > .20) and negative correlations with the age of first internet use (r < -.15). These are interpreted as support to the criterion validity of the IDS-15. The IDS-15 dimension scores were mostly unrelated to tobacco and alcohol use but showed low but significant negative correlations with illegal drug use. Additionally, dimension scores were negatively correlated with pathological gambling behaviour. In terms of demographic characteristics, individuals’ age was a strong predictor of high IDS-15 scores, with younger participants being at more risk. Lower education was a risk factor for the EDEC and WS dimensions but not for the IDSR and DISC dimensions. Dimension correlations with gender were relatively small, although male internet users had slightly higher IDS-15 scores. Lastly, monthly income was mostly unrelated to IDS-15 scores.
Item-level analysis
Psychometric analysis at the item-level is presented in Table 6. The factor loadings of items obtained from the CFA (Model 1) were generally satisfactory, except in the case of item EDEC-1 (i.e., “I never go online to feel better”). Consequently, the deletion of this item resulted in a significant improvement of EDEC factor reliability (+ 0.09). EDEC-1 also had the highest standard deviation and mean among all items of the IDS-15. Additionally, items EDEC-2 (i.e., “I think that being online can greatly change my mood for the better”) and DISC-1 (i.e. “I could easily stop spending time online if I wanted to without any problem”) displayed slightly higher standard deviation, mean and lower factor loadings compared to other items. DISC-1 was also the only other item besides EDEC-1 whose deletion improved the corresponding factor reliability.
Latent profile analysis
In order to describe various types of internet users based on their risk for IA, LPA was performed. The optimal number of latent classes was determined based on multiple indices, presented in Table 7. The AIC and BIC values were decreasing up to the 10-class solution, and the associated BLRT p-value was within statistical significance for up to seven classes (presumably due to the larger sample size). Due to relatively low classification certainty as well as difficulty of interpretation of this many latent classes, we opted to compare the class solutions based on the entropy criterion (a measure of classification certainty, where 1.00 represents complete certainty). Based on the highest entropy values, as well as highest minimum and maximum average latent class probability (Min. prob., Max. prob.), the three-class solution was ultimately recognised as optimal. In line with previous IDS-15 validation studies, these categories were labelled as “low addiction risk” (n = 3,818; 51.5%), “medium addiction risk” (n = 3,111; 42.0%), and “high addiction risk” (n = 484; 6.4%).
Characteristics of each IA class are shown in Table 8. High-risk users show higher scores across all IDS-15 dimensions. Notably, risk classes are particularly distinguished in terms of their WS factor score. In addition, significantly more high-risk users reported using the internet for at least 20 hours per week in leisure time, compared to the other two classes. Lastly, higher IA risk class corresponds with lower age of first internet use.
Discussion
The aim of the present study was to continue the efforts of evaluating the psychometric properties of the IDS-15, previously done by Pontes and Griffiths [24], Monacis et al. [25], and Lin et al. [26]. More specifically, the study developed and evaluated the Slovenian IDS-15 on a large and nationally representative sample of Slovenian internet users (aged 15–64 years). Firstly, the reliability analysis showed that the Slovenian IDS-15 is a reliable instrument, both on the level of the scale (ω = .94) and its four sub-factors (α > .70). Furthermore, the CFA results indicated that the four-factor structure proposed by Pontes and Griffiths [24], sufficiently fitted the data. However, importantly, the factor loading of Item EDEC-1 was noticeably lower compared to other items, despite its statistical significance. Its (relatively) high variance, mean, and alpha-if-item-deleted also indicated there is room for improvement. On the other hand, the discriminant validity of the scale was adequate, which means that the IDS-15 factors assess distinct constructs pertaining to IA. On the final indicator of construct validity (i.e., measurement invariance), the present study confirmed the findings of Monacis et al. [25] and Lin et al. [26] regarding measurement invariance across both genders on a configural, metric and scalar level. Moreover, measurement invariance across age groups was established on a configural and metric (but not scalar) level in the present sample.
The LPA identified three latent profiles of internet users with regard to their risk of IA. A large majority of participants were classified as either low addiction risk (51.5%) or medium addiction risk users (42.0%), whereas only 6.4% of users were recognised to have a (comparatively) high-risk for IA. This figure is similar to the global prevalence rate of IA reported in a previous meta-analysis study [42], and a more recent one by Pan et al. (7.0%) [43]. The proportion of high-risk users was significantly higher in the three previous validation studies [24–26], which reported values between 26% and 42%. This discrepancy could be attributed to either the lower age of participants (adolescents and students) or the opportunistic sampling in the previous studies. Furthermore, individuals with higher addiction risk (as well as IDS-15 score and factor scores) obtained higher scores on all IDS-15 factors, were more likely to spend more time on the internet for leisure purposes, were younger, started using the internet earlier, and had a higher IGD score, which is consistent with the findings of Pontes and Griffiths [24]. Additionally, these results are comparable with the findings of a study on problematic internet use of Slovenian adolescents by Macur and Pontes [44], where it was reported more withdrawal symptoms (factor Obsession), lack of control (factor Control Disorder), and impairments (factor Neglect) among “high PIU risk” individuals.
Criterion validity analysis further showed that effects of gender, level of education, and monthly income were weak and inconsistent across IDS-15 factors, indicating that IA is not restricted to specific demographic groups. Frequencies of tobacco and alcohol use were not associated with IDS-15 scores. Furthermore, individuals reporting problematic gambling behaviour and illegal drug use were even less likely to obtain high IDS-15 scores. These findings strengthen the case of IA being a completely separate disorder to substance (and gambling) addictions. Based on the results above, the IDS-15 has adequate criterion validity.
The undesirable performance of item EDEC-1 (i.e., “I never go online to feel better”) may be due to two potential reasons. First, the wording of the item, specifically the use of the word “never”. For the average internet user, who has been accessing internet daily for multiple years, it would be very difficult to claim they have “never” gone online to feel better. The tendency to make such a claim, may be, at least in part, due to factors unrelated to internet use. Moreover, items using “never” and “always” are common in social desirability scales (i.e., [45]). The possibility that social desirability may be responsible for a non-negligible share of the variance of EDEC-1 should therefore be considered. A second potential reason for the subpar performance of EDEC-1 may be due to the reversed item effect, since EDEC-1 is the only reversed item in that factor (for an elaboration of the reversed item method bias, see [46]). Considering the apparent lack of observed issues with EDEC-1 in previous validation studies, the item translation was re-examined, but no obvious improvements could be made. Therefore, it is suggested that there be an alternative (non-reversed) item wording which would address both criticisms: “I often go online to feel better”.
An additional note related to the practice of using reversed items should be mentioned in the case of the IDS-15. The fourth IDS-15 factor, Dysfunctional Internet-related Self-Control (DISC) consists entirely of reversed items. The proposed grouping of these items, indicated by CFA, may not only be due to their content, but perhaps also due to their wording direction. Looking more closely, Items IDSR-3 and IDSR-4 target an ability, very alike to the one targeted by the DISC items (i.e., the ability to control/regulate time spent on the internet). Since the discriminant validity of the DISC factor seems completely sufficient, the question of which unique tendencies are assessed by the factor in question (as opposed to IDSR) should be further explored.
Since the publication of the first IDS-15 validation study, the criteria for Gaming Disorder (GD) have been introduced in the ICD-11 [47], and supported by a consensus of experts [48]. Importantly, escapism/mood regulation and withdrawal symptoms (two IDS-15 factors) are not used to assess GD in the ICD-11. Since the IDS-15 is partly based on the DSM-5 criteria for IGD, it should be explored to what extent these changes also apply to IA. Lastly, it was expected that both individuals with generalised PIU and specific PIU would obtain a high IDS-15 scores. It is recommended the scale application to be supplemented with individual interviews and questions regarding the type and frequency of their online activities to differentiate between the two PIU types. Additionally, questions regarding devices used can help to differentiate between PIU predominantly mobile and PIU predominantly non-mobile, as suggested by Montag et al. [4].
Study limitations
Despite careful planning, design and implementation of the study as well as the sample population being representative, there are some limitations that should be mentioned. First, there is a possibility of bias associated with the response rate. Heavy internet users may have been more likely to decline their participation in a lengthy survey, due to their preoccupation with online activities or their general preference for engaging, online content. This could have led to underestimating the extent of IA symptoms in the population. Second, the absence of cognitive interviews during the adaptation process of the Slovenian IDS-15 may mean there is room for improvement in terms of item wording. Third, all the data were based on self-reports which are subject to well established method biases.
Conclusions
The results of the present study contribute to the scientific literature concerning the validity of IDS-15. The study provided the data from the Slovenian version of IDS-15 scale, which add to the cultural variability of currently validated versions of the IDS-15 (i.e., English, Italian, Persian). To add to previous validation attempts of the IDS-15, the present study obtained a larger and more nationally representative sample of internet users between the ages of 15 and 64 years. The procedures and methods adopted to evaluate psychometric properties of previous versions of the IDS-15 were followed in the present study, with the addition of measurement invariance analysis across different age groups. The findings illustrated strengths and potential shortcomings of the IDS-15. The reliability and validity of the Slovenian IDS-15 and its factors were in general satisfactory. However, due to the concerns related to Item EDEC-1, testing its recommended reworded version would be sensible. Nonetheless, the results of the present study valuably contribute to the field of digital addictions. Namely, they provide an insight in the prevalence of IA in Slovenia, as well as associations with other factors that may help to explain this phenomenon. Therefore, the results reported serve as the basis for further analyses on IA epidemiology, policymaking activities, and design for targeted public health interventions in Slovenia, as well provide a basis for future studies. Future studies of the IDS-15 would make a significant contribution by addressing any of the issues raised in the Discussion, by further testing either formulation of Item EDEC-1, employing qualitative approaches to assess validity, establishing measurement invariance beyond gender, and obtaining data from a clinical sample.
Acknowledgments
We would like to thank Darja Lavtar and Maruša Rehberger from the National Institute of Public Health, Slovenia, who contributed to study design and implementation of the survey and provided information regarding the methodology of the survey.
References
- 1. Young KS. The evolution of Internet addiction. Addict Behav. 2017;64:229–30. pmid:26059165
- 2. Davis RA. Cognitive-behavioral model of pathological Internet use. Comput Human Behav. 2001;17(2):187–95.
- 3. Shapira NA, Lessig MC, Goldsmith TD, Szabo ST, Lazoritz M, Gold MS, et al. Problematic internet use: Proposed classification and diagnostic criteria. Depress Anxiety. 2003;17(4):207–16. pmid:12820176
- 4. Montag C, Wegmann E, Sariyska R, Demetrovics Z, Brand M. How to overcome taxonomical problems in the study of Internet use disorders and what to do with “smartphone addiction”? J Behav Addict. 2021;9(4):908–14.
- 5. Young KS. Internet addiction: The emergence of a new clinical disorder. Cyberpsychology Behav. 1998;1(3):237–44.
- 6.
Weinstein A, Curtiss Feder L, Rosenberg KP, Dannon P. Internet Addiction Disorder: Overview and Controversies [Internet]. Rosenberg KP, Feder LC, editors. Behavioral Addictions: Criteria, Evidence, and Treatment. Elsevier; 2014. 99–117 p. Available from: http://dx.doi.org/10.1016/B978-0-12-407724-9.00005-7
- 7. Griffiths M. A “components” model of addiction within a biopsychosocial framework. J Subst Use. 2005;10(4):191–7.
- 8. Brand M, Young KS, Laier C. Prefrontal control and Internet addiction: A theoretical model and review of neuropsychological and neuroimaging findings. Front Hum Neurosci. 2014;8(MAY):1–13. pmid:24904393
- 9. Pontes HM, Griffiths MD. The Development and Psychometric Properties of the Internet Disorder Scale–Short Form (IDS9-SF). Addicta Turkish J Addict. 2017;3(3):1–16.
- 10. Griffiths MD, Kuss DJ, Billieux J, Pontes HM. The evolution of Internet addiction: A global perspective. Addict Behav. 2016;53:193–5. pmid:26562678
- 11. Griffiths MD, Szabo A. Is excessive online usage a function of medium or activity? An empirical pilot study. J Behav Addict. 2014;3(1):74–7. pmid:25215216
- 12. Pontes HM, Szabo A, Griffiths MD. The impact of Internet-based specific activities on the perceptions of Internet addiction, quality of life, and excessive usage: A cross-sectional study. Addict Behav Reports. 2015;1:19–25. pmid:29531976
- 13. Király O, Griffiths MD, Urbán R, Farkas J, Kökönyei G, Elekes Z, et al. Problematic internet use and problematic online gaming are not the same: Findings from a large nationally representative adolescent sample. Cyberpsychology, Behav Soc Netw. 2014;17(12):749–54.
- 14. Deci EL, Ryan RM. Self-determination theory: A macrotheory of human motivation, development, and health. Can Psychol. 2008;49(3):182–5.
- 15. Leung H, Pakpour AH, Strong C, Lin YC, Tsai MC, Griffiths MD, et al. Measurement invariance across young adults from Hong Kong and Taiwan among three internet-related addiction scales: Bergen Social Media Addiction Scale (BSMAS), Smartphone Application-Based Addiction Scale (SABAS), and Internet Gaming Disorder Scale-Short. Addict Behav [Internet]. 2020;101(April 2019):105969. Available from: https://doi.org/10.1016/j.addbeh.2019.04.027
- 16. Chen IH, Strong C, Lin YC, Tsai MC, Leung H, Lin CY, et al. Time invariance of three ultra-brief internet-related instruments: Smartphone application-based addiction scale (SABAS), Bergen social media addiction scale (BSMAS), and the nine-item internet gaming disorder scale-short form (IGDS-SF9)(study Part B). Addict Behav. 2020;101(105960). pmid:31072648
- 17. Kamolthip R, Chirawat P, Ghavifekr S, Gan WY, Tung SE, Nurmala I, et al. Problematic Internet use (PIU) in youth: a brief literature review of selected topics. Curr Opin Behav Sci. 2022;46(1011).
- 18. Andreassen CS, Pallesen S. Social network site addiction-an overview. Curr Pharm Des. 2014;20(25):4053–61. pmid:24001298
- 19.
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th Ed.). Washington, DC: American Psychiatric Publishing, Inc.; 2013.
- 20. Griffiths MD, Kuss DJ. Online addictions: Gambling, video gaming, and social networking. Sundar S, editor. The Handbook of the Psychology of Communication Technology. 2015. 384–403 p.
- 21. Love T, Laier C, Brand M, Hatch L, Hajela R. Neuroscience of internet pornography addiction: A review and update. Behav Sci (Basel). 2015;5(3):388–433. pmid:26393658
- 22. Pontes HM, Kuss DJ, Griffiths MD. Clinical psychology of internet addiction: A review of its conceptualization, prevalence, neuronal processes, and implications for treatment. Neurosci Neuroeconomics. 2015;4:11–23.
- 23. Király O, Griffiths MD, Demetrovics Z. Internet Gaming Disorder and the DSM-5: Conceptualization, Debates, and Controversies. Curr Addict Reports. 2015;2(3):254–62.
- 24. Pontes HM, Griffiths MD. The development and psychometric evaluation of the Internet Disorder Scale (IDS-15). Addict Behav. 2017;64:261–8. pmid:26410796
- 25. Monacis L, Sinatra M, Griffiths MD, de Palo V. Assessment of the Italian Version of the Internet Disorder Scale (IDS-15). Int J Ment Health Addict. 2018;16(3):680–91. pmid:29904328
- 26. Lin CY, Ganji M, Pontes HM, Imani V, Broström A, Griffiths MD, et al. Psychometric evaluation of the Persian Internet Disorder Scale among adolescents. J Behav Addict. 2018;7(3):665–75. pmid:30264609
- 27. Koronczai B, Urbán R, Kökönyei G, Paksi B, Papp K, Kun B, et al. Confirmation of the three-factor model of problematic internet use on off-line adolescent and adult samples. Cyberpsychology, Behav Soc Netw. 2011;14(11):657–64. pmid:21711129
- 28.
Eurostat. European Health Interview Survey (EHIS wave 3) Methodological manual [Internet]. Luxembourg: Publications Office of the European Union; 2018. Available from: https://ec.europa.eu/eurostat/documents/3859598/8762193/KS-02-18-240-EN-N.pdf/5fa53ed4-4367-41c4-b3f5-260ced9ff2f6?t=1521718236000
- 29. Pontes HM, Griffiths MD. Measuring DSM-5 internet gaming disorder: Development and validation of a short psychometric scale. Comput Human Behav [Internet]. 2015;45:137–43. Available from: http://dx.doi.org/10.1016/j.chb.2014.12.006
- 30. King DL, Chamberlain SR, Carragher N, Billieux J, Stein D, Delfabbro PH, et al. Screening and assessment tools for gaming disorder: A comprehensive systematic review. Clin Psychol Rev. 2020;77(101831). pmid:32143109
- 31. Poon LY, Tsang HW, Chan TY, Man SW, Ng LY, Wong YL, et al. Psychometric Properties of the Internet Gaming Disorder Scale–Short-Form (IGDS9-SF): Systematic Review. J Med Internet Res. 2021;23(10):e26821. pmid:34661543
- 32. Pontes HM, Macur M, Griffiths MD. Internet gaming disorder among slovenian primary schoolchildren: Findings from a nationally representative sample of adolescents. J Behav Addict. 2016;5(2):304–10. pmid:27363464
- 33. Wejbera M, Müller KW, Becker J, Beutel ME. The Berlin Inventory of Gambling behavior—Screening (BIG-S): Validation using a clinical sample. BMC Psychiatry. 2017;17(1):1–8.
- 34.
Collins LM, Lanza ST. Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Hoboken, New Jersey: John Wiley & Sons, Inc.; 2010.
- 35.
Hooper D, Coughlan J, Mullen MR. Evaluating Model Fit:a Synthesis of the Structural Equation Modelling Literature presented at the 7th European Conference on Research Methodology for Business and Management Studies,. Conf Pap Sch Manag [Internet]. 2008;0–10. Available from: https://arrow.tudublin.ie/buschmancon
- 36.
Hair JF, Anderson RE, Babin BJ, Black WC. Multivariate data analysis: A global perspective. 7th ed. Upper Saddle River, New Jersey: Pearson Prentice Hall; 2010.
- 37. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Vol. 14, Structural Equation Modeling. 2007. p. 464–504.
- 38.
R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021.
- 39. Jorgensen TD, Pornprasertmanit S, Schoemann AM, Rosseel Y. semTools: Useful tools for structural equation modeling. R package version 0.5–5. 2021.
- 40. Rosseel Y. lavaan: An R package for structural equation modeling. J Stat Softw. 2012;48:1–36.
- 41. Rosenberg J, Beymer P, Anderson D, van Lissa C j., Schmidt J. tidyLPA: An R Package to Easily Carry Out Latent Profile Analysis (LPA) Using Open-Source or Commercial Software. J Open Source Softw. 2018;3(30):978.
- 42. Cheng C, Li AYL. Internet addiction prevalence and quality of (real) life: A meta-Analysis of 31 nations across seven world regions. Cyberpsychology, Behav Soc Netw. 2014;17(12):755–60. pmid:25489876
- 43. Pan YC, Chiu YC, Lin YH. Systematic review and meta-analysis of epidemiology of internet addiction. Neurosci Biobehav Rev [Internet]. 2020;118(March):612–22. Available from: pmid:32853626
- 44. Macur M, Pontes HM. Internet Gaming Disorder in adolescence: investigating profiles and associated risk factors. BMC Public Health. 2021;21(1):1–9.
- 45. Greenwald HJ, Satow Y. A Short Social Desirability Scale. Psychol Rep. 1970;27(1):131–5.
- 46. Weijters B, Baumgartner H, Schillewaert N. Reversed item bias: An integrative model. Psychol Methods. 2013;18(3):320–34. pmid:23646990
- 47.
World Health Organization. ICD-11: International classification of diseases (11th revision) [Internet]. 2019. Available from: https://icd.who.int/
- 48. Castro‐Calvo J, King DL, Stein DJ, Brand M, Carmi L, Chamberlain SR, et al. Expert appraisal of criteria for assessing gaming disorder: an international Delphi study. Addiction. 2021;116(9):2463–75. pmid:33449441