Is Social Simulation a Social Science Outstation? A Bibliometric Analysis of the Impact of JASSS

This paper examines the bibliometric impact of JASSS on other ISI- and Scopus-indexed sources by examining inward and outward citations and their inter-relation. Given the prestige of JASSS, this analysis can measure the growth and dynamics of social simulation and give us an indication of the direction in which social simulation is moving. Results show that the impact of JASSS is higher in computer sciences, physics and ecology than it is in the social sciences, even though JASSS-indexed articles tend to be more concerned with social science-related topics. Looking at inter-journal citations revealed an interesting citation structure: JASSS collected its largest percentage of citations from non-social science-focused journals while directing more citations within its own articles toward works published in social science journals. On the one hand, this would confirm that social simulation is not yet recognised in the social science mainstream. On the other hand, this may indicate that the cross-disciplinary nature of JASSS allows it to promulgate social science theories and findings in other distant communities.

Social simulation is a multi-disciplinary field now reaching a mature stage in its development, as has been attested to in a number of recent studies (e.g., Meyer, Lorscheid and Troitzsch 2009;Squazzoni 2010;Meyer, Zaggl and Carley 2011). To better understand this multi-disciplinary nature, we carried out a bibliometric analysis of JASSS based on the best available databases of peer-reviewed, scholarly publications, such as ISI and Scopus.
Our aim was to understand the position of JASSS in the scholarly journal environment and to examine the structure of citations and cited sources. Since JASSS is considered the most prestigious social simulation publication, examining this journal can provide a realistic picture of the overall situation.

1.2
The rest of this paper is organised as follows. The first section presents a bibliometric analysis of the impact of JASSS on other ISI-and Scopusindexed sources. We have focused on these databases because they are considered to be highly representative of the best scholarly peer-reviewed publications and complement one another both in terms of coverage and type of information. Indeed, while ISI is the most respected bibliometric index, Scopus covers a larger sample of conference proceedings, which are extremely relevant to this investigation into the impact of social simulation. Results have shown that JASSS has had a truly cross-disciplinary impact and that this impact was more prevalent in computer science, physics and ecology as compared with the social sciences.

1.3
In the second section, we looked in greater detail at the relation between social simulation and other fields by analysing the structure of inward and outward citations. We restricted our focus to the years 2009-2010, as appropriate citation data were available only for this period. As we will see, looking at citations and cited sources reveals an interesting citation structure: JASSS collected the largest percentage of its citations from non-social science journals, while directing more citations within its own articles toward works published in social science journals. We have also found a structure of tighter relations between a small set of journals that represent the true hub of social simulation.

1.4
In the third section we have reported on JASSS top-20 cited articles in the ISI, Scopus and Google Scholar indices. We restricted our attention to these articles, which comprise approximately one-third of all journal citations, in an effort to reveal differences and similarities between the respective indices. In the final section, we clarify certain limitations of our work and highlight potential implications germane to the further development of JASSS.
JASSS' bibliometric impact (ISI andScopus databases, 2002-2010) 2.1 From 2002 to 2010, JASSS published 324 articles, receiving 983 and 1283 citations, respectively, by all ISI-and Scopus-indexed sources (excluding journal self-citations). The journal achieved an increasing impact, especially in the most recent years of the period under consideration. Figure 1 illustrates the total number of citations on ISI-and Scopus-indexed sources. It is worth noting that the citation growth dynamics were similar across the two indices, although the size of the growth in the most recent years was more pronounced among ISI-indexed sources. 2.2 As shown in Figure 2, which illustrates the impact factor JASSS achieved in the Journal Citation Report by Thomson Reuters, the journal's impact factor increased significantly, particularly from 2008 to 2010. This allowed the journal to not only position itself as a recognised title of quality among the list of interdisciplinary social science journals-where it ranked 8th among the 84 titles listed as of 2010-but also to achieve respectable positions if included on other, related, disciplinary lists. Considering the situation as of 2010, for instance, were JASSS included in the list of political sciencefocused journals, it would have ranked 15th of 141, 16th of 132 journals in the list of sociology-focused journals, 49th of 308 if included in the list economics-focused journals, and 21st of 78 if included in the list of environmental studies-focused journals.

2.3
As shown in Tables A.1 and A.2 of the appendix and as noted above, JASSS collected 983 citations from 178 ISI-indexed sources and 1283 citations from 468 Scopus-indexed sources (excluding self-citations), respectively. Looking at the top positions on the ISI list, it is worth noting that a significant percentage of the citations came from physics and related journals or from book series focused on complexity, simulation and artificial intelligence. Physics journals such as International Modern Physics C (6.6% of total citations), Physica A (5.9%) and Physical Review A (3%) topped the list, along with Lecture Notes in Artificial Intelligence (10% of all citations), Lecture Notes in Computer Science (4.7%), Advances in Complex Systems (3.8%) and Computational and Mathematical Organization Theory (2.6%). Another considerable percentage of citations came from ecology-and environmental sciences-focused journals such as Environmental Modelling & Software (2.6% of all citations), Computers, Environment and Urban Systems (1.8%), Ecology and Society (1.6%), Geoforum (1.4%) and Ecological Modelling (1.3%).

2.4
The picture is similar among Scopus-indexed sources, although in this case physics-focused journals had a less pronounced presence at the top of the list and there were more sources which only rarely cited articles published in JASSS. The top positions were occupied by well-known sources such as Lecture Notes in Computer Science (9.74% of all citations), Advances in Complex Systems (2.10%) and Computational and Mathematical Organization Theory (1.40%). As was seen among ISI-indexed sources, another significant percentage of citations came from ecology-and environment sciences-focused journals such as Ecological Modelling (2% of all citations), Ecology and Society (1.16%), Environmental Modelling and Software (1.40%) and Computers, Environment and Urban Systems (1.16%).
2.5 Table 1 shows the impact of JASSS on other ISI-indexed sources classified by discipline. This classification was carried out by hand, one-by-one, to ensure a precise allocation of journals and fields and to resolve any instances of multiple allocation. Results showed, first, that JASSS had an impact on a diverse range of disciplines. Second, the results showed that computer science-focused journals attracted 30% of all citations, followed by physics (21%). Figure 3 shows a comparison between ISI-and Scopus-indexed journals. Although ISI did not afford full coverage of the period considered, we can state that approximately 75% of all citations of JASSS articles came from the fields of computer science, physics, the natural sciences and ecology, while only approximately 20% came from social science-focused journals.

2.6
The picture is different if we consider sources cited by articles published in JASSS. It was unfortunately not possible to perform this analysis on only ISI-cited sources in 2009 and 2010. We also restricted our interest to journal citations by removing any source indicating a monograph or an edited volume (in this case, included in the ISI database). As is shown in Figure 4, our results indicate that approximately half of all citations concerned social science-focused journals, whereas computer science-focused journals accounted for 22% of all citations and physics-focused journals 8%. It is worth noting that in this case even the humanities were represented, accounting for approximately 3% of all citations.

2.7
Taken together, these findings reveal that the citation structure of JASSS was unbalanced in terms of inward and outward citations: more citations were collected from non-social science sources, while more citations were made of social science-related sources. We will examine this in more detail below. . Percentage of ISI-(left) and Scopus-indexed (right) sources which cite JASSS out of the total number of citations, presented per academic discipline for the period 2002-2010. Note that, for the sake of simplicity, in the category "ecology", we have included environmental sciences and geography (as most such articles dealt with socio-ecological systems); in "natural sciences" we have included physics, mathematics, statistical mechanics, agriculture and water management; in "social sciences", we have included economics, psychology, political sciences, cognitive sciences and management and business. "Humanities", when cited, included history and philosophy. 2.8 Although we could not control for the relative weight of the individual disciplines in terms of the number of potential citations and the number of journals, this citation structure nevertheless did confirm the recent findings of Meyer, Lorscheid and Troitzsch (2009) relating to the multidisciplinary nature of social simulation. The impact of JASSS in fact touches upon a wide variety of disciplines. On the other hand, our results indicate that: (i) the influence of JASSS is more prominent in computer science, physics, ecology and the natural sciences; and, (ii) articles published in JASSS are more thoroughly embedded in social science literature than in other fields.
The inter-journal citation structure 3.1 Looking at inter journal co-citations allows us to better understand the relationship between social simulation and more traditional areas of study such as physics, computer science, ecology and social sciences. Doing so can also help us verify the discrepancy found in the previous analysis-i.e., the unbalanced relationship between inward and outward citations at the single-discipline level.
3.2 To achieve this we first ranked the inward and outward ISI citations made in 2009 and 2010, which we used to create the index Cit, which measured their inter-journal relationships, as follows: (1) where j stood for JASSS, i for each journal included in the list of citations, m i→j the number of citations of JASSS-published works collected from any other journal i and n j→i the number of citations of works published in any other journal i in JASSS. The index takes a range of possible values between 1 (=JASSS was cited by i at least once but JASSS never cited i) and -1 (=JASSS cited i at least once but was never cited by i), with 0 indicating a perfect balance of inward and outward citations. To control for the different dimensions of the inter-relation j ↔ i, as any value of Cit could be achieved by different combinations of in-outward citations, we added a second index, Cit w , which measured the total number of inward-outward citations, as follows: (2) 3.3 Let us suppose, for instance, that JASSS provided 20 citations of journal i and received 10 citations in turn. This would yield values of Cit = 0.33 and Cit w = 30. The same Cit = 0.33 could be achieved had JASSS been cited 205 times by the journal i, which received 103 citations in turn; on the other hand, such a scenario would yield a value of Cit w = 308. This allowed us to also consider differences in the size and scope of inter-journal relations.
3.4 Following Leydesdorff (2007) and using our inter-journal citation indices, we first mapped the total JASSS network for the period considered (see Figures 5 and 6). Although certain differences exist between the two years in terms of the number and type of ties and the journals included in the analysis, we again found that stronger inter-journal connections existed in the fields of natural and computer sciences.

3.5
To examine the data in greater detail, we remapped the network to present values of Cit within the range -0.5 : 0.5-i.e., focusing only on inter-journal relations with a consistent number of exchanged citations. Figures 7 and 8 show that, were we to exclude the importance of the inter-journal relations with CMOT in 2010-which could be viewed as a relationship between two journals which are thematically related rather than a relation between JASSS and a mainstream social science journal, as well as a less consistent relationship with JEBO and Sociological Research Online-the hub of social simulation more often included computer science-and physics-focused journals.    The JASSS article stars 4.1 As expected, the impact of JASSS was strongly dependent upon the success of a restricted number of articles. Tables 2, 3 and 4 show the top-20 cited JASSS articles from 2002 to 2012 in the ISI, Scopus and Google Scholar databases, respectively. These articles collected approximately onethird of all JASSS citations in the period considered in this analysis. First, it is interesting to note that there are significant differences between the three lists, and especially between ISI and the other lists. Epstein's 2008 article on modelling (Epstein 2008), for instance, ranked second in ISI, while it ranked 20 th and 11 th in Scopus and Google Scholar, respectively. Two influential articles on opinion dynamics (Hegselmann and Krause 2002;Deffuant et al. 2002), which accounted for approximately 10% of all JASSS citations in the Scopus and Google Scholar databases during the period considered-where they were ranked first and second, respectively-ranked relatively low in ISI. Second, it is worth noting that the top positions were frequently occupied by either methodological or epistemological articles, which accounted for approximately half of the top 20 such articles on each list.
4.2 Figure 11 shows the percentage of un-cited articles from 2002 to 2012 retrieved from Scopus. Were we to consider the growth in the number of JASSS articles published in the most recent years of the study period-with an average number of 32 articles published per year during the period 2002-2005 as compared with an average of approximately 40 articles published per year in the more recent period-this figure indicates that a greater number citations were concentrated among a relatively small number of articles, which were thus responsible for the increasing impact of the journal. In 2010, for example, approximately 60% of JASSS articles did not receive a single citation. This would indicate that there is room for stronger selection of submissions.  (5)    Our results indicate that the impact of JASSS is cross-disciplinary but is consistently concentrated more among computer science, the natural sciences, physics and ecology. The impact of the journal on the social sciences is less significant. This finding could be subjected to multiple interpretations. On the one hand, this could be seen to confirm the fact that social simulation is not yet recognised as a relevant pursuit in the social sciences, as has been previously suggested by Richiardi et al. (2006). On the other hand, and especially if we were to consider the citation structure, we could conclude that JASSS serves the important function of reverberating social science concepts, theories and models in other, originally distant academic disciplines.

5.2
In both cases, this is a typical pattern for any cross-disciplinary, specialised journal. Such journals indeed seek to connect distant fields and bridge the gaps which exist between disciplines which lack a single, incremental, institutionalised and discipline-specific organisational field (e.g., Huutoniemi et al. 2010). This might also explain the relative instability of the impact factor trends and the greater variety of inter-journal connections among crossdisciplinary journals (e.g., Rinia et al. 2001;Levitt and Thelwall 2008).

5.3
While it is expected that, for instance, a JASSS article which investigates a cognitive science-related topic by adopting a social simulation approach will be cited more widely by an artificial intelligence-than a psychology-focused journal, the relative weakness of the multi-disciplinary links of JASSS in their reverberations within the domain of the social sciences is a problem which still must be seriously addressed by future research. A stronger capacity to provide feedback on social sciences is in fact a strategic concern for social simulation, given its stated mission of influencing the manner in which social science investigations are typically conducted.

5.4
In this respect, it is worth considering one severe limitation of our bibliometric analysis, which is that our databases did not in fact address the impact of JASSS-published articles on monographs and edited books. Such publications remain one of the most common forms of communication within the social sciences. As has been suggested by the JASSS-related findings of Meyer, Lorscheid and Troitzsch ( 2009) as well as those pertaining to CMOT by Meyer, Zaggl and Carley (2011), it would be reasonable to expect that the importance of citations in scholarly journals will continue to grow at the cost of citations of books and book chapters. This is a sign of the times. Monographs and book chapters, however, are still a suitable communication tool for social simulation. Excluding this scholarly publishing environment from an impact analysis therefore could have led us to have underestimated the impact of JASSS on the social sciences. This limitation could in turn be resolved, at least in part, by extending our analysis to include Google Scholar; this database covers a broad set of knowledge output sources, however, the screening of which would require a considerable effort for any more in-depth analysis.

5.5
Despite the required caveats typical of any impact analysis performed on only a single journal (e.g., Leydesdorff 2008), our results have allowed us also to highlight certain policy implications. First, our results testify to the problem of concentrating on individual impact measures. If we look at aggregate indicators, such as the impact of JASSS in various fields, data from a single source can be sufficient. On the other hand, it is difficult to use data from a single source to evaluate specific aspects in greater detail, such as the impact of individual articles or topics. To sum up, working with multiple sources remains necessary if we are to provide a comprehensive picture of a journal, field or research area.

5.6
Second, there is an important message in these findings for cross-disciplinary journals such as JASSS, which represent an innovative field whose disciplinary/thematic connections are in a state of flux and are thus difficult to map. Our work indicates the importance of monitoring the impact and citation structure and dynamics of such journals, as doing so can provide information on the nature of innovation, reveal the penetration of certain topics of study into various areas and testify to typical academic 'political' problems of this type of research within institutionalised disciplines. In the case of JASSS in particular, our results could motivate targeted actions, such as the organisation of special issues and initiatives in an effort to consolidate existing links with already strongly connected areas and strengthen those areas with weaker ties.