Probing novelty at the LHC: Heuristic appraisal of disruptive experimentation

In this paper, ‘novelty’ is explored through a recent historical episode from high energy experimental physics to offer an understanding of novelty as disruption. I call this the ‘750 GeV episode’, an episode where two Large Hadron Collider (LHC) experiments, CMS and ATLAS, each independently observed indications of a new resonance at approximately 750 GeV. With further data collection, the initial excess was determined to be a statistical fluctuation. The approach taken, in the analysis of interviews conducted with physicists who were involved in the ‘750 GeV episode’, is to consider novelty as a valued difference. Following this conceptually driven approach, disambiguate between several notions of novelty through the identification of varied differences. This disambiguation is achieved through exploring differences expressed in comparison to varied expressions of the standard model, and through exploring varied ‘types’ of difference (properties and entities) to introduce disruptive exploratory experimentation, a complementary understanding ‘exploratory experimentation’ (Elliott, 2007; Steinle, 1997, 2002). I show that the kinds of novelty framed as most valuable are those that violate expectations and are difficult to incorporate into the existing structures of knowledge. In such instances, disruption to the existing ontology or ways of knowing is valued. This positive appraisal of disruption, and contradiction over confirmation, is considered in the recent context of high-energy physics, where several physicists have claimed that there is a lack of promising directions for the future, or even that the field is in a ‘crisis’. I show that the role of disruption explains the differences between the differing notions of novelty. Furthermore, I show that the positive appraisal of disruption is based on forward looking assessments of future fertility, or heuristic appraisal (Nickles, 1989, 2006). Within the context of concerns of a lack of available promising future directions, disruption becomes a generator of alternative futures. © 2019 Published by Elsevier Ltd.


Introduction
In this paper, I will explore 'novelty' through a recent historical episode of LHC physics. I call this the '750 GeV episode', an episode where two LHC experiments, CMS and ATLAS, each independently observed indications of a new resonance in the same mass region. Several physicists indicated, both at the time and then in reflections, that if the significance of the result increased with more data, to the point where a discovery claim could be made, then this would be more novel 1 than the Nobel Prize winning discovery of the Higgs Boson. 2 This is quite extraordinary given the significance of the experimental discovery of the Higgs, which was last confirmed prediction 3 of the standard model of particle physics 4 and determined the mass of the Higgs (which had not been predicted). Whilst the observed resonance ultimately turned out to be a statistical fluctuation, these expressions of greater novelty motivated a deeper investigation of the episode as a case study, including interviews with those who conducted the search and analysis in CMS and ATLAS to probe their reflections on the 750 GeV episode and novelty.
The Large Hadron Collider at CERN is one of the largest and most complex experiments ever built, consisting of a 27 km ring in which protons are accelerated and made to collide in bunches of proton collisions in four detectors. Each of these detectors was independently built and is run a by large experimental collaboration: the ALICE, ATLAS, CMS, and LHCb experiments. ATLAS and CMS are multi-purpose detectors, designed to understand the origin of electroweak symmetry breaking, to search for physics beyond the standard model, and to perform precision measurements of processes within and beyond the standard model (ATLAS Collaboration, 2003;CMS Collaboration, 2002). These epistemic goals make the ATLAS and CMS experiments the ideal place to explore novelty in the context of the practices of scientific experimentation. Not only do these epistemic goals suggest that the LHC experiments are aimed at being novelty-producing machines, but in their diversity they also hint at diverse understandings of 'newness'. This presents an opportunity to examine situated expressions of novelty: a concept often used but rarely interrogated.
ATLAS and CMS completed the first period of data taking at 13 TeV in October 2015. Researchers in ATLAS and CMS each independently performed a high mass search in the diphoton channel. Both signature searches, i.e. a search for a diphoton signature, saw a small excess at approximately 750 GeV in their results in November 2015. Each of these results independently indicated that there might be an as yet undiscovered particle with a mass of 750 GeV. 5 Both results were made public when they were included in the 'Jamboree' session at CERN on December 15 (Kado, 2015;Olsen, 2015). However, news of the results leaked out of the collaborations and some theorists wrote papers that were uploaded to the arXiv within hours of the official announcements by ATLAS and CMS. 6 In each of the announcements from ATLAS and CMS spokespersons the low local and global significance of the results was emphasised, and a number of experimentalists indicated that this result could be a statistical fluctuation that would disappear with more data. 7 In weeks that followed hundreds of theory papers were uploaded to the arXiv with diverse potential theoretical explanations for the excess, and over 500 papers were published by the end of the episode.
Both of the analysis groups within ATLAS and CMS immediately attempted to check the results but LHC had just entered into a short shut down and no new data could be produced. ATLAS looked at the 8 TeV data to see if an excess could be observed. CMS determined a way to use some 13 TeV data that had been collected whilst the magnet was not working, the so-called '0 T data'. Both presented at the Moriond conference in March 2016, where CMS presented a result with a slightly higher local and global significance due to the inclusion of the 0 T data (Delmastro, 2016;Musella, 2016). Data collection began again in March 2016. When the analysis was completed over May and June no excess was observed. The results were presented publicly at the ICHEP conference in August 2016 (Lenzi, 2016;Rovelli, 2016). Further analysis determined that the 2015 result was a statistical fluctuation and that no systematic error was responsible for the result.
In order to explore more deeply novelty in the context of the case study, I conducted semi-structured oral history interviews with experimentalists who worked on the 750 GeV analysis in CMS and ATLAS, at different levels (PhD candidate to coordinators). In this paper, I locate and explore different expressions of novelty found in interviews with experimental physicists. In order to do this, I will consider novelty as a relational concept, i.e. I locate novelty in the interviews where two or more things are described as connected, in some way differentiated, and where the difference is positively valued. This conceptually driven analysis allows for the exploration of the diversity of novelty systematically, where the same criteria for the identification of novelty are applied across all interview materials. Instead of attempting an exhaustive taxonomy of novelty, some of the differences between understandings of novelty, that are significant within the context of the case study, will be explored. I will outline and explore some differing expressions of ontological novelty found in relation to different expressions of the standard model (differing ontologies). Also located are different kinds of novel contributions to the high-energy physics ontology: properties vs entities identified through differing expressions of epistemic practices. Across each of these differing expressions of novelty are differing positive appraisals in that there are different attributions of value.
The kinds of novelty framed as most valuable are those that violate expectations and are difficult to incorporate into the existing structures of knowledge. In such instances, disruption to the existing ontology or ways of knowing were valued. This positive appraisal of disruption, and contradiction over confirmation, is explored in the recent context of high-energy physics, where several physicists have claimed that there is a lack of promising directions for the future. I show that the role of disruption explains the differences between the differing notions of novelty. Furthermore, I show that the positive appraisal of disruption is based on forward looking assessments of future fertility, or heuristic appraisal. Within the context of concerns of a lack of available promising future directions, disruption becomes a generator of alternative futures.
The paper proceeds as follows. In section two theories of novelty from history and philosophy of science are analysed. In section three I outline my approach to novelty, both conceptually and in terms of a systematic methodology for the identification of novelty in the interviews. In section four, I identify different notions of novelty present in the interview material. In section five, I explain some of the differences between the notions of novelty can be attributed to the role of expectations and disruption. I also show how the positive appraisals of disruption are heuristic appraisals that are future focused and based on assessments of the future fertility of generating alternative futures for the high-energy physics research program. Finally, I reflect on the context in which these assessments were made and the uniqueness of valuing disruption over confirmation.
2. Theories of novelty in the history and philosophy of science literature 'Novelty' as a concept has received surprisingly little attention in scholarship (considered broadly), as well as in history and philosophy of science (HPS). North, in his book length treatment, written from the perspective of cultural studies, points to the historic lack of philosophical engagement: "it doesn't take much looking to discover that there isn't any such tradition, no standard text, no omnibus history" (North, 2013). Whilst North does not resolve any difficulties in clarifying the concept of novelty, through discussion and comment on some historical philosophical treatments of novelty he makes clear the inherent ambiguity of the concept. In this section, I will explore treatments of novelty from different traditions in HPS in order to develop an approach to novelty that offers a middle ground by drawing upon the insights from each of these bodies of literature.
Conceptual engagement with novelty in HPS literature comes from two significant areas: debates over scientific realism and the no miracles argument (NMA), and debates over emergence. Conceptual clarity of novelty was sought following the challenge issued by Laudan (1981) to the NMA. Those seeking to defend a version of scientific realism argued for a more precise definition of successful science that did not include the counter examples identified by Laudan. A variety of authors argued that the relevant class of successful scientific theories are those theories that make novel predictions, which are confirmed by experiment. 8 For this approach to work, a precise understanding of novelty was required. Two broad options were debated: 'temporal novelty' and 'use novelty' (i.e. novel predictions from a theory that are confirmed later in time), and predictions that do not use the empirical content in the theoretical construction of a prediction (Ladyman, 1999;Leplin, 1997;Musgrave, 1988, pp. 229e252;Psillos, 1999;Worrall, 1985). There were a variety of positions offered on 'use novelty', such as from Leplin (1997) who argued that for a prediction to be novel it had to be both independent and unique. That is to say, for Leplin, the empirical content predicted by the theory if it is already known must not be necessary for the theory, that the theory must explain and predict the empirical content, and that no other theory provides a reason to expect the empirical content. 9 In all cases of use and temporal novel predictions, the argument is that only the truth of the theory generating the novel prediction can explain the successful novel prediction. Herein lies the difficulty with the debates over novelty in this literature for the purposes of this paper: the focus is exclusively on the theoretical generation of novelty. Furthermore, the authors are not interested in how scientists use and draw upon the concept on novelty implicitly or explicitly; the perspective of the experimentalist, and even the theorist, is missing.
The second area that also requires investigation into the concept of novelty is the literature on emergence. In this literature, authors consider when an assemblage of known objects is, or should be, considered novel. In order to debate issues such as reductionism, a definition of novelty is required to establish the class of phenomena categorised as emergent. For example, Jeremy Butterfield established, informally and formally, definitions of novelty in order to discuss examples of emergence that he argues are compatible with reductionism (Butterfield, 2011a;2011b). The informal definition is as follows: "'novel' means something like: 'not definable from the comparison class', and maybe 'showing features (maybe striking ones) absent from the comparison class'." The informal definition is relational in approach in that it relies on comparisons, but for Butterfield novelty occurs where a comparison cannot successfully be made. For the purposes of this paper this literature is also unhelpful in that the definitions are theory-focused. However, like the literature on novelty and scientific realism, these debates show how different understandings of novelty can be established for the purposes of different positions.
Treatments of novelty linked to progress, rather than truth, have been written from within a tradition that attempts to formulate general historical and philosophical accounts of science (for example Kuhn and Lakatos). Both Kuhn and Lakatos are well known for their discussions of novelty: Kuhn developed the concept of the anomaly and Lakatos pinned progressive research programmes with the requirement that they make novel predictions that become novel facts. It is perhaps unsurprising, given Lakatos' tradition of revisionist histories of science, to discover that he too did not delve deeply into the use of the concept of novelty by his historic actors. Instead, he defined a novel prediction as being derived from instances where theoretical content is greater than empirical content (i.e. the theoretical content that does not have an empirical basis is understood as a novel prediction). In his account, a novel fact is a novel prediction which is later empirically confirmed (temporal novelty) (Lakatos, Worrall, & Currie, 1978, p. 184).
Kuhn's 'Structure' (Kuhn, 1970) has been interpreted and reinterpreted many times in the service of divergent positions on revolution and change in rationalist and relativist accounts of scientific progress. He himself famously revised (or extended, depending on interpretation) his own position in later works (Kuhn, 1977, pp. 74e86). In 'Structure' anomalies are a part of normal science and are paradigm-dependent i.e. an anomaly is a result that contradicts the paradigm, or cannot immediately be reconciled within the paradigm. Anomalies are never explicitly sought (p. 52e53, 64, 169) and, where found, are at least initially considered to be the fault of the scientist (p. 35), rather than the paradigm. If an anomaly continued to resist reconciliation with the paradigm, Kuhn claimed that historically this has resulted in a period of crisis (p.75), which was sometimes followed by a revolution. One of the well-known and well-documented difficulties with Kuhn's history of science is that he did not engage with experiment: as Hacking has noted, "immense experimental or instrumental novelty is simply missing in Kuhn's theoretical stance" (Hacking, 2012, p. xvii). For the purposes of this paper, from Kuhn we can take a notion of novelty that is in relation to the current knowledge (rather than adopting the notion of paradigms) and can consider how this might be expanded to include experiment. However, a notion of novelty that occurs inevitably as a standard element of science does not allow for an exploration into how scientists aim for novelty, may positively appraise novelty, or make novelty possible.
Detailed historical approaches have not significantly engaged with the concept of novelty; indeed, the concept has often been rejected as uninteresting. This is in part due the association between novel results and discovery and the long-standing tradition to be sceptical of the concept of 'discovery' following the widespread adoption of the attributional model of discovery (Brannigan, 1981). This is also in part due to the influence of laboratory studies, which took an ethnographic approach to examine in detail scientists, and their research practices, in situ. Novelty, in this approach, is determined to be an uninteresting final product and examination of novelty was not in keeping with the aim of laboratory studies to examine science in process in very fine detail 10 . Latour and Woolgar argued that they "found it extremely difficult to formulate descriptions of scientific activity which do not yield to the misleading impression that science is about discovery (rather than creativity and construction). It is not just that a change in emphasis is required, rather, the formulations which characterise historical descriptions of scientific practice require exorcism before the nature of this practice can be best understood" (emphasis authors' 8 The notion that predictions which are in some sense 'novel' or 'new' have epistemic value can also be found in many other earlier works, such as Whewell's work on confirmation where he argues that a hypothesis "ought" to entail predictions which precede observation (Whewell, 1885, p. 88). See Musgrave for discussions of predictivism in Descartes, Leibniz and others (Musgrave, 1974). 9 This position was criticized by Ladyman (1999). See Psillos (2009)  S. Ritson / Studies in History and Philosophy of Modern Physics xxx (xxxx) xxx own) (Latour & Woolgar, 1979, p. 129). The more recent constructivist tradition maintains this denial of the value of examining and explaining novelty. Pickering (1995) associated novelty explicitly with his notion of "resistance" (p. 119) as part of his argument for science proceeding through a "dialectic of resistance and accommodation" (p. xi). Pickering further argues that emergence is "brute chance" and criticises searches for mechanistic explanations of emergence (p. 24). In this paper, I pursue an integrated approach, offering a middle ground between detailed studies that do not prioritise novelty and formal or generalist perspectives that do not prioritise the perspective of scientists or experiments. Rather than attempting to explain novelty, as is the aim in the literature concerning emergence, or attempting to define novelty via necessary and sufficient conditions, I follow the various uses of novelty by experimentalists in order to gain insight into how and where the concept of novelty is invoked by experimentalists. This approach allows for conceptual engagement with novelty, whilst also building upon the insights of researchers working from detailed historical accounts who examined the local and contextual nature of epistemic practices (in this paper, understandings of novelty come from the actors rather than from considerations of novelty as discovery or as a final product). This also draws on the insights from the philosophical literature where the importance and also inherent ambiguity of the concept of novelty was highlighted. In examining how novelty is framed, in the appraisals of experimentalists, this paper does not suggest that novelties merely exist in discourses of appraisal and does not argue against the relevance of novelty for understanding emergence, scientific realism, or scientific progress. Rather, the integrated approach of this paper allows for both the identification of diverse notions of novelty from the perspective of the experimentalists involved in the episode in question, as well as a conceptual analysis of novelty from the historic context in which the concepts of novelty are expressed.

Locating novelty in the interviews
I conducted interviews in 2017 with experimental physicists from ATLAS and CMS, chosen for their experience, each having worked in some capacity on the experimental identification or analysis of the 750 GeV resonance. The interviews were conducted as semi-structured oral history interviews, taking into consideration the historic context of the 750 GeV episode and the context in which the interviews were conducted. At the time of the interviews, the excess had been found to be a statistical fluctuation and no other evidence for physics beyond the standard model had been found at the LHC. Slight modifications were made to the questions asked based on who was being interviewed and their experience, and designed to investigate the interview participant's experiences relevant to the episode. Questions were asked on: the interview subject's background and experiences within ATLAS or CMS; the development of the 750 GeV episode and the interview subject's involvement; the interview subject's perspectives and reflections on the 750 GeV episode; and the interview subject's perspectives and reflections on novelty. All participants have been anonymised in this text and assigned unrelated surname pseudonyms. The length of the interviews was between one and three hours.
As the aim of this paper is to explore how novelty is framed by experimentalist interview participants who were involved in the 750 GeV episode, an initial requirement was to formulate criteria to identify consistently across the interviews where the interview participants drew upon novelty either explicitly or non-explicitly. As was outlined in the discussion of the debates over novelty in the philosophical literature, it is difficult to establish when a difference is sufficiently different to be considered novel. There is the problem that if defined too broadly, as the quality of being 'new', almost anything the participants described as being different in some small way could have been construed as novel, rendering the exploration of the content somewhat vacuous. One option was to follow the use of our actor's category: new physics, a category often employed. However, to restrict ourselves to the actor's category results in the loss of some conceptual clarity and the omission of novelty that does not refer only to results, which therefore excludes processes.
The approach adopted was to consider novelty as a valued difference. This working understanding was applied in a close analysis of the interviews, using the understanding of the concept as an analytic lens through which to identify the perspectives of the experimentalists interviewed, and locate novelty in various relations, or expressed differences, which are in various ways valued. This approach avoids using a definition that is too broad, as not all differences are considered, but also allows for the identification of diverse framings of novelty in the interviews. This approach has the further advantage of foregrounding the role of appraisal in how novelty is framed through the exploration of the attribution of value. I certainly do not claim to have defined novelty, or to have identified necessary and sufficient conditions for the condition of novelty, or claim that there are no other promising conceptual approaches. Instead, the claim is that this approach to the analysis is productive in that it allows for restricted diversity in the identification of notions of novelty, and for the exploration of the role of appraisal within the interviews from the case study.
Following this approach, I identified reference to novelty in the interviews only where both aspects could be located: difference and value. When locating differences, 'difference' was approached conceptually as necessarily comparative, in that in order for something to be described as different it has to be with respect to some reference. Value, or attribution of positive appraisal, was located in various ways, sometimes from interview questions, in that I explicitly asked participants for an assessment. These notions of novelty are then analysed as being formed in reconstructions of a historical episode; reconstructions informed by the local context from which the interviews took place. As I show below in more detail, and through the exploration and analysis of examples, different understandings of novelty come from diversity in what and how something is different (diverse relations in the interviews), and what and how something is valued (diverse assessments in the interviews). The choice of the examples for this paper was motivated by illustrating this diversity and to disambiguate the notions of novelty in this historical episode.
The different 'notions' of novelty analysed in the interviews are organised in this paper into two groups of reference classes, where the reference class determines a comparison from which a difference can claimed. Each of these classes is an assemblage of the current state of knowledge. The paper is organised to proceed stepwise through an exploration of examples from each of the identified reference classes. This approach highlights some of the differences between the overlapping reference classes and allows for the exploration of the different reasoning behind appraisals of value. By proceeding stepwise through the reference classes of relations to the current knowledge, the role of disruption to the current knowledge in the reasoning behind positive appraisal is foregrounded. Not only is the role of disruption established, as one that differentiates, but in keeping with exploring novelty as a valued difference, the role of disruption in positive appraisals of difference is explored. Perhaps surprisingly, disruption is shown to be a positive measure of novelty.

Differing notions of novelty
4.1. First group of reference classes: the verified SM, the predicted SM, and BSM approaches Many of the relations in which novelty was identified in the interviews were in claims 11 concerning relations to the ontology of particle physics, which in particle physics today means the standard model (SM). The SM is an assemblage of theoretical and experimental insights describing particles and their interactions that was developed by experimental and theoretical physicists beginning in the 1970s. The contemporary SM includes three fundamental forces (or interactions), the strong, weak and electromagnetic forces and is comprised of two gauge theories; namely, the electroweak theory which unifies and describes the weak and electromagnetic interactions, and the theory of quantum chromodynamics (QCD) that describes strong interactions. The SM has been extraordinarily successful in that it has been confirmed by thousands of experiments, to very high degree of precision. Despite this, the SM remains incomplete, in that it cannot account for the fourth fundamental force: gravity (as well as dark matter, dark energy, neutrino masses, and matter-antimatter asymmetry), and has a number of perceived conceptual problems. 12 There are several different classes of expressions of the SM in which relations were located in the interviews, each of which forms a different comparison point, which I organise here into three reference classes. These include comparisons made with respect to the experimentally verified results, results predicted by SM, and results predicted by the many extensions of the SM. Here experimentally verified results include the collection of currently accepted experimental results that have been incorporated into the SM framework. From these three reference classes I identify three notions of novel results. The first notion is identified by those results that are expressed as having a difference in observation status from the experimentally verified SM, such as the self-coupling of the Higgs that has been predicted by the SM but not observed. The second and third notions are those that are identified as having a difference in observational status from the experimentally verified SM as well as a difference in theoretical status in that they are not described theoretically by the SM. The third notion of novel results are those that also differ with respect to what is described by the so-called beyond the standard model (BSM) approaches, such as the minimal supersymmetric standard model (MSSM) which attempt to extend the SM. This reference class differs from the first two in that there are a very large number of BSM models, conventionally separated into supersymmetry (SUSY) models, and all other non-SUSY BSM models, which are referred to as exotica by the ATLAS and CMS collaborations. 13 The accumulative potential of each of these reference classes is clear, in that a result that is considered different with respect to BSM models could also be different with respect to the predicted SM and the verified SM. Each of these classes of reference is similar in that each is considered with respect to the current state of knowledge; though the classes differ in degree of acceptance. 14 Whilst the differences between the reference classes may seem to be just steps further away from the existing knowledge, interesting differences become apparent through the exploration of the differing ways in which examples from each notion of novelty are positively appraised.
In order to explore the first group of reference classes against which novelty is claimed, I will draw upon a quote from an interview participant from the CMS experiment, Carraway, who described the first observation of 750 GeV excess in December 2015 relation to the now historic discovery of the Higgs: "There was a lot of anticipation, people were really watching ATLAS and CMS to show something, and then by chance something came out. It didn't happen since … there is nothing to compare. Of course there was the Higgs discovery, but it was very different, because everybody knew that things have to go that way, maybe that year or the following year. But we were pretty sure that the Higgs was there, so the discovery was a confirmation. Here, things could have been more exciting." Here we have two relations: a relation expressed of the differences between the historic development of the Higgs and the 750 GeV excess and a relation between the undiscovered Higgs to the discovered Higgs. We can also see that the historic novelty of the Higgs is described through the relation to the experimentally verified SM, the process of discovery is framed as searching for something that was expected but at the time not yet experimentally verified. This is an example of the first reference class in that the difference between the discovered Higgs to the undiscovered Higgs is experimental verification. In contrast the 750 GeV excess was "very different" in that there were no expectations generated by the SM. The importance of expectations in the relations in which I interpret appeal to novelty, non-explicitly in this case, is quite apparent here in that the relation to the reference class can be one of different and expected, as the Higgs is described here, or different and not expected, as the 750 GeV excess is described. The interpretation of the attribution of value is quite straightforward: both results are described as exciting, albeit the 750 GeV as "more exciting".
To tease out the differences between the second and third reference classes I will explore an example where the interview participant, Wilson, was asked to simply describe the 750 GeV episode. The example below is also helpful in interpreting the value placed on the 750 GeV excess: "The interesting thing, let's say, there were many interesting things at the time. But, one thing that I can highlight … of course it was unexpected, as anything is as opposed to the Higgs, although it was, let's say, partly unexpected it was something that was not predicted … in this case we really had zero expectations, we were just searching and something popped up … 11 Note that by using the terminology of claim here, I do not wish to contribute to the debates concerning the semantic component of scientific realism. I take an agnostic stance with respect to whether the interview subjects interpret their claims literally or take a more instrumentalist stance. This is because the argument of this paper is orthogonal to the realism debates and instead focuses on heuristic appraisal of future fertility, where such appraisals are compatible with either a realist or instrumentalist stance. 12 This is a very simplified view of the standard model, intended here just to give a broad introduction. Some of the conceptual problems with the standard model include the hierarchy problem and the perception that the standard model contains 'too many' free parameters. However, there is yet no consensus as to nature of these problems, or the weight that can or should be attributed to them. 13 SUSY models invoke a symmetry that transforms bosons into fermions and vice versa, implying that every SM particle must have a superpartner with spin differing by ½. In many of these models, the lightest SUSY particle is predicted to be stable and at located at a mass scale that is detectable by ATLAS and CMS. The term 'exotica' is used by ATLAS and CMS to identify non-SUSY BSM models, a diverse group including, for example, models with extra dimensions, dark matter models, and extended Higgs models. For a prospects analysis for both ATLAS and CMS prior to run 2 see (Cakir, ATLAS, & CMS, 2015).
14 The model based extensions of the SM are constrained by experimental and theoretical results and whilst degree of belief in various models differs across researchers, models such as MSSM have a long history of belief and searches for experimental evidence for these models was cited as part of the physics argument for the construction of the Large Hadron Collider.
I mean, it was, let's say that it was not unforeseen by any possible model of new physics that you can imagine, but it was for sure not the most conventional signature that one could imagine." Wilson separates out different senses in which novel results can be unexpected. The 750 GeV excess was unexpected in that there were no existing predictions from the SM (the first reference class). Furthermore, once the excess had been observed it was not easily reconciled with the existing extensions of the SM (the third reference class). It is important to note that in the framing of Wilson, part of the novelty of the result comes from the process by which it was established that the signature could not be easily accommodated, or made "conventional".
Wilson continued, claiming: "… in general, if we have the means to look for something, and we I think we should look and not be afraid I mean, it's in a way, is what we call a signature-based search, not model-oriented search. So sometimes we really go after models … and we tailor searches to test those models. And sometimes we just look at things which can be plausible …" The diphoton signature searches that observed the 750 GeV excess in CMS and ATLAS were both broadly motivated by BSM models, so called Randall Sundrum (RS) extradimensional models which predict spin 2 resonances (CMS, 2015) and Higgs like spin 0 resonances with masses greater than 200 GeV (ATLAS, 2015), and each of these classes of models predict a resonance that will decay to produce a diphoton signature. Despite the broad BSM motivation, the analysis strategy of both ATLAS and CMS was to search for an unspecified local excess in the m gg spectrum (ATLAS, 2015;CMS, 2015). Here we can further disambiguate between predicted novelty and unpredicted novelty through the expectations identified of different experimental processes. Model based searches can generate a class of precisely expected results, a result expected through a prediction of BSM model. In contrast, a signature search could generate a result that is more generically expected in that it might fit into the framework of classes of BSM models, or a result that is possible but may not be described by an existing BSM model. Here we see that some of the attribution of novelty of the 750 GeV resonance was derived from both the experimental process and the determination that the result was inconsistent with precise BSM model predictions.
Each of the notions of novelty identified here is closely related but by identifying the three different possible expressions of the SM that serve as comparisons, it is possible to see the reasoning behind how the 750 GeV excess was described as potentially more novel than the Higgs. Each reference class, in the presented order, identifies a notion of novelty that is a step further away from what is expected from the current experimental and theoretical knowledge. A result that is compared to and different to the verified SM, but is predicted (i.e. the historic Higgs result), is less novel than a result that is compared to and different from both the verified and predicted SM. A result that is compared to and different to the verified SM and the predicted SM, but is predicted by a model based extension of the SM (i.e. a SUSY particle), is less novel than a result that is different from both the verified and predicted SM, as well as the model based extensions of the SM (i.e. the 750 GeV excess). I will say something more about differing values across these different relations in section five, however for now I will explore one further complication in locating novelty in relations to the SM (in addition to the different expressions of the SM).

Additional notions of novelty: properties vs entities
Here I introduce two additional notions of novelty, which I locate from comparisons between each notion: properties and entities, that is, a distinction between novel knowledge of an entity and novel knowledge of the nature of an entity. The distinction between novel entities and novel properties on the surface also seems to be somewhat trivial. 15 However, much of the work that has gone into large collider experiments has focused on experiments that claimed to have found new entities such as quarks (Pickering, 1984) or new interactions such as the neutral current (Galison, 1987). Moreover, whilst measurements of properties feature in these texts, they are not the focus. 16 This section of the paper will explore some interesting features of how properties are framed by the interview participants, especially in the context of the upcoming phases of the LHC. Emphasised by many of the interview participants was, at the time of the interviews, that ATLAS and CMS were both increasingly focusing on precision measurements (the so-called "data-driven era" of ATLAS and CMS). Following an examination of valued differences as examples of novelty, I will explore an example of how measurements of the properties of entities are construed as novel (often as opposed to novel entities). Note that the relation in which the difference is located may be with respect to any of the aforementioned reference classes of novelty: i.e. a property may be different with respect to the verified SM, the predicted SM, or model-based extensions of the SM.
Baker articulated and offered an explanation of what I identify as novel properties when he outlined what he considered to be possible and desirable future outcomes for the upcoming phases of the LHC: "so, in terms of new physics … I mean realistically what we can hope to come out of the LHC now are deviations in the … I mean are strange things essentially in the particles that we know" (participant's emphasis). The novelty, or the expressed valued difference, can be seen through the expression of the relation of what is part of the ontology of particle physics (the existence of certain particles or entities) against what is not yet part of the ontology of particle physics (i.e. the properties that have yet to be measured). The value, in this instance, is not only found in the completion of a measurement: it is also in a class of results that is unexpected, or 'strange', i.e. it is hoped that expectations will be violated.
The comparisons between novel properties and entities were identifiable in the interviews often due to distinctions drawn in terms of processes. For example, according to Buchanan who described changes following the discovery of the 'Higg-like boson': "… essentially we've switched from the mode of trying to find [a Higgs-like boson] to studying the thing that we got and that's a little bit of different mind-set in some ways … So, you change essentially from looking for something that's a bump to … understanding all sorts of details in the measurements … and 15 Unsurprisingly, in practice, there are often cases where a measurement plays a significant role in the claim of a new entity, complicating the distinction between properties and entities. If you consider the historical development of the Higgs boson, in 2012 ATLAS and CMS both announced that they had seen evidence for a boson with a measured mass of approximately 125 GeV. The measurement of the particle and the measurement of the spin both played a significant role in the process of discovery as the SM did not predict the mass of a SM Higgs boson but did predict the spin. Now that the mass has been measured, more measurements are required to see if the Higgs does play the role in the SM that is predicted i.e. confirming it as a SM Higgs boson, or indicating that the entity may not be a SM Higgs boson. In this example, measurements are constitutive of new entity claims in various and often experimentally complex ways. 16 The exceptions being (Beauchemin, 2017;Staley, 2017), and a short section in (Knorr-Cetina, 1999, pp. 52e56, 65).
these things are much more important once you actually want to make a precise measurement than if you just ask the question 'is there some bump'." Here the difference is framed in terms of time, and the evolution of knowledge, in that Buchanan gives a historical perspective on the shift from how claims are made of a novel entity to how claims are made of the properties of the same novel entity. Buchanan draws upon a concept of different modes of research, and this modal framing highlights both a difference in epistemic processes and differences in the ways of thinking. The reference classes invoked here are framed by difference in epistemic practices rather than an explicit difference in ontological status.
Baker also framed the measurement of properties in time, however instead of only looking backwards for a historical perspective, Baker also framed novel properties with an outlook to the future. And it is here that I can further explore the apparently conflicting simultaneous expectations from earlier: that is, the case of an articulated expectation, which is closely linked with hope, for results that will contradict expectations that are motivated by theory: "… to start with the Higgs Boson, so, I mean, that's the, the particle we discovered last and it's the one on which we have larger error bars, okay? So, between those error bars and the predictions from the standard model theory there may be effects that … that would, would give us a direction e that would be evidence essentially of something else … so … yes, there I think is … [reason to hope or expect] I mean, obviously, because … I mean, I keep working on this …" There are several things of note from this extended quote, the first being that the participant draws upon recent results to project into the future. The large error bars on recent results are presented as providing a space for future novelties (with respect to the SM). Furthermore, the value expressed here is that these future results will provide a "direction", i.e. they will guide future research. This assessment of value is also apparent in how Baker frames his own work: he has invested his time, and made choices about the research he has conducted and will conduct, due to what he presents as future fertility. This appraisal that potentially theoretically unexpected results can provide future fertility helps us to begin to understand how results that contradict expectations can be valued.
The focus on the potential of processes to generate fertile results also invites consideration of novelty beyond ontological novelty, into novel ways of knowing. Relations between processes and practices invoked in the interviews also revealed a notion of novelty in terms of ways of knowing. The relations in which I locate novelty here are in comparisons against processes that were at some point in time, or in some configuration of the laboratory, 17 not possible to complete or remained uncompleted. This further establishes the diversity of novelty and helps to foreground how not only it is the case that results are valued, but it is also the case that the ability to produce results and the fertility of practices are also highly valued.

Disruption
In this section, the role of disruption in the various notions of novelty in the interviews will be explored in order to obtain a deeper understanding of differences between these notions of novelty and the role of expectations. This will be done on the basis that something that disrupts is described as something that alters or destroys structures or content of belief, and a disruption is a process described as resulting in a disturbance or problem. Disruption is a term often used pejoratively and is often associated with negative outcomes; consider the 'disruptive child' or the dread of seeing an email indicating that your flight schedule has been disrupted. However, in the interviews of the 750 GeV episode, participants frequently framed disruption positively. The potential result, a resonance at 750 GeV in the diphoton channel, was described as something that would have been significantly disruptive, and this potentiality was especially valued. One aspect of how the 750 GeV result is framed as disruptive is that it was contrary to all expectations. van Lente has argued that expectations do not only function as statements that are evaluated as having true or false content at some point in the future. Expectations, where articulated, are also "performative" in that they have an immediate impact, for example by creating obligations (van Lente, 2012). These insights are important in this case study, where disruption is formed by the violation of expectations, generating an obligation to change the framework that generated the expectations. One interview participant, Gatz, outlined the results they now hope for, following the end of the 750 GeV episode: "We are all for, again, a huge strange thing. Actually, honesty, my hope is that still e and I say hope, I don't know if this will happen or not but, that we see something that no one has foreseen, I mean, …the nice thing about the bump at 750 GeV is that, was that, there was no theory predicting … Interviewer: Is this why you called the 750 GeV e it would have been the discovery of the century, because it was not predicted at all?
"Absolutely, yes. Yes, the Higgs boson could or could not have been there but if it was there, we knew more or less where to look at, I mean, we didn't know the mass but we had some constraints from theory and it was clearly a missing part … It would have taken a lot of time to really place [the 750 GeV] into a framework we understand. And when nature surprises us with phenomena that we don't understand it's always a good start for, you know, a revolution of the mind rather than you know, moving within the frame you know since years …" In this extended quote, there are a number of things of which to take note. We again see appeals to different notions of novelty: where the Higgs was a novel experimental result, the 750 GeV excess is cast as more novel in that it has not been predicted by the SM or any other theory. This difference is again linked to expectations, where the path to find the Higgs is described as being simple in that theoretical constraints drove expectations. Here we can see the impact of expectations: the potential violation of expectations 17 Here I consider the laboratory quite broadly. One could consider the removal of financial, or organisational, pressures as one way to frame how it came about that a different and valued process of producing knowledge came to occur. An example from a different area of physics, which also relies on large collaborative efforts, also highlights this: the Nobel Prize in Physics in 2017 was given to some of the leaders of the LIGO collaboration. The Nobel Prize committee framed the prize as being for both the result (ontological novelty) and the organisational work required for the LIGO detector: "for decisive contributions to the LIGO detector and the observation of gravitational waves" (Nobel Prize Committee quoted in ("Press Release: The Nobel Prize in Physics 2017,"). results in a potential disruption not only to the particle physics ontology but also to wider knowledge frameworks, which include established methods for the interpretation of results.
Gatz also articulated that there were negative expectations on the 750 GeV excess in that events had only been observed in the diphoton channel: "So the weird thing is that it was not happening anywhere else and there was on some sense what called for extreme caution because we can say 'okay, it's more likely to be a fluctuation or something else' and at the same time, extreme excitement because if this proved to be a real then it's something we don't understand and not understanding is always good, is always a good starting point." The expectation for signals in other channels were based on a number of generic predictions (not BSM model dependent), assuming that the excess would be explained by a new resonance, with mass of approximately 750 GeV, decaying into two photons with either spin 0, 2 or higher (the possibility of a spin 1 resonance was excluded because it cannot decay into gg (Landau, 1948;Yang, 1950)). Due to electroweak symmetry a resonance in the diphoton channel indicates that excesses with the same invariant mass should have been observed in other diboson channels (e.g. WW, ZZ) (Strumia, 2016). Here Gatz paints the experimental expectations that the 750 GeV result was unlikely as one of the reasons that the 750 GeV result was valued as a potential result. The value claimed for the 750 GeV is that it would have required significant changes to the 'frame', or the existing experimental and theoretical knowledge structures, as opposed to the Higgs, which is described as being easily accommodated. A potentially disruptive result is valued over an expected result, a confirmation of theoretical prediction.
The value placed here on experimentally driven disruptive developments runs contrary to the numerous models from HPS, which focus on the relationship between experiment and theory confirmation. This one directional notion, where the role of theory is to make predictions and the role of experiment is search for evidence that, ideally, confirms the prediction, clearly does not accommodate accounts of the different notions of novelty in the 750 GeV episode. It is certainly not unique to claim that the role of experimentation needs further attention beyond theory confirmation; the so-called 'New Experimentalism' literature that came out of the experimentalist turn beginning in the 1980s foregrounded the many roles of experiment as opposed to a one directional relationship where theory directed the experimental agenda. 18 A notable contribution to the New Experimentalism literature includes the concept of an 'exploratory experiment' (Elliott, 2007;Steinle, 1997Steinle, , 2002 which sought to expand how the role of experimentation could be understood beyond theory confirmation. Elliot and Steinle showed historical examples of experiments that were not 'driven' by theory. In Steinle's, 2002 paper, he outlines exploratory experimentation as those experiments that do not aim to be "tests of expectations" that instead are characterised by having absence of specific guidelines. When considering the concept of exploratory experimentation and disruption, a great deal rests on the interpretation of a 'test of expectations'. There is a narrow sense of a test of expectations where a precise prediction coming from theory forms the basis of an expectation. This sense clearly does not capture disruptive results that are characterised as violating expectations. However, if you consider a test of expectations more broadly, a test for expectations could be any experiment that generate a result that has some relation to the theory, including that which alters or destroys the theory. An important distinction needs to be made here in terms of how I identify the concept of disruption in the interview content. There are results that are interpreted as disruptive, such as the 750 GeV excess, and there is broader notion of disruption embedded in the experimental processes evaluated as having the potential to generate unexpected or disruptive results. Examples of this are in section 4.1 where Wilson outlines the potential of signature based searches to generate results such as the 750 GeV excess and, as is explored in section 4.2, the potential of precision measurements to generate novel contradictory results. Whilst it is clear that the aim of the experimental processes that aim for disruptive results is, at least partially, not confirmation (in that these experiments allow for both confirming and disruptive results), such experimental processes cannot be characterised as having no theoretical goal in mind in that the disruptive results would alter or destroy some of the existing theoretical knowledge structures. Whilst these results are not determined as confirmations of an existing theory, there is still a relationship to theory: disruption. This is a complementary understanding of experimentation to exploratory experimentation (EE): disruptive exploratory experimentation (DEE). In both cases, EE and DEE, theoretical confirmation is not sought. In the case of exploratory experimentation, according to Stienle and Elliott, it is due to lack of existing theory. In the case of disruptive experimentation, it is due to an aim to deliver a result that in some way contradicts the existing theory. However, DEE is still exploratory in that a precise result is not specified or aimed for but a more general result is aimed for, one that disrupts. That is to say, disruptive experimentation, where successful, is categorised as contradicting expectations generated by the existing knowledge structures.
Karaca has also written about how ATLAS experiments do not fit the category of EE due to the existence of existing well-formed theory. He argues that it is still possible to consider the experiment as being exploratory through a detailed account of the data selection strategy in the trigger software (Karaca, 2017). Karaca argues that a notion of exploratory procedures, as those that seek to "extend the range of possible outcomes", applies to the ATLAS experiment (Karaca, 2017, p. 340). The account in this paper is similar in that it highlights that it is not only confirmation that is sought through the experimental practices. However, it differs in that it highlights the positive appraisal of results, by some experimentalists, that contradict and disrupt through the exploration of how the 750 GeV result was valued and the appraisal of the potential of measurements and signature based searches.

The heuristic appraisal of disruption
In what follows, I will further expand on the positive appraisal of disruption from the case study by showing that the appraisal is an appraisal of future fertility, or heuristic appraisal in that disruption is assessed as having the potential to provide alternative futures. As far back as 1989, Thomas Nickles explored and attempted to argue for the relevance of heuristic appraisal in reflections on scientific practices and investigation. He argued that philosophy of science had only focused on the retrospective assessments of scientists, and had been slow to explore just how often scientists are forward looking in their appraisals. Nickles argued that "novel predictions do more than provide epistemic warrant for the tested theory: they open up new areas of investigation" (Nickles, 1989). Nickles further elaborated more recently, claiming that "[heuristic appraisal] evaluates the promise or potential fertility or feasibility of further work on a problem, research program, theory, hypothesis, model, or technique. [Heuristic appraisal] estimates the likely return on in investment in expensive equipment, in reorganizing a laboratory, or even in adding new members to a research team" (Nickles, 2006, p. 159). Here I will examine heuristic appraisal in an experimental context, to explore assessments of future fertility beyond novel predictions.
It is important here to reflect on the local context of the 750 GeV episode, and the timing of the interviews with the experimentalists. In the lead up to the first collisions at the LHC a number of experimental and theoretical physicists discussed a possibility: what if the predicted Higgs boson was found, and nothing else? Some named this the 'nightmare scenario' because it would leave experimentalists and theorists with a lack of direction as to future results, or how to solve the problems of the SM. Theorist Aldo Deandrea was quoted in Science in 2007 as saying: "If you have just a Higgs that is consistent with the SM, then you probably don't know what to do next … What then?" (Cho, 2007). At the time of the interviews, a SM Higgs had been found, the 750 GeV excess was a statistical fluke, and no other 'novelties' had been claimed. This has led some theorists, such as the current head of theory at CERN, Giudice (2017), to question whether there is currently a crisis in high-energy particle physics. In the assessments of novelty made in this context, disruption is evaluated as both a measure of difference and as something that provides a direction. In the just discussed example, Gatz strongly argues for the value of experimentally driven disruptions in that they that have the potential to provide a new direction, a "new starting point".
I have alluded to the role of heuristic appraisal and forwardlooking aspects of appraisal in the previously discussed examples of how different notions of novelty are framed (where I showed in sections 4.1 and 4.2 that the interview participants frequently focused on expectations and the promise of results). In the following example, the role of heuristic appraisal is particularly strong where the interview participant, Wolfsheim, reflected on the theoretical attempts to explain the ATLAS signal 19 : "… we had a broad signal and a broad signal to diphotons is very hard to explain. It was a really ugly thing actually. So you needed a lot of new physics to explain it, it would be actually fantastic if it would have been true: we would have had a huge amount of new physics. You know, because there's no way to explain it, even just one additional resonance is … so you needed a loop where you had particles in this loop to make it broad, to make it a large cross section and so they were ingenious ideas in this sense." The difficulty in explaining the signal originates from the requirement that a large width is only possible if the potential 750 GeV resonance also decays to other final states than photons. The width could have been caused by a new 'hidden sector' with new particles that only very weakly couple to the SM (Strumia, 2016). Wolfsheim frames the ATLAS signal as novel in that there was no readily apparent way to explain the result; the result is different with respect to the established modes of interpretation of result (i.e. the well-known BSM approaches 20 ). This is also a further example of how disruptive results are framed as positive.
Wolfsheim is effusive in his assessment of the fertility of an "ugly", or disruptive, result in that it would point researchers in the direction of many future results that would be needed to accommodate the disruption. Nickles has called this aspect of heuristic appraisal an "opportunity profile of a claim, technique, proposal etc" (Nickles, 2006, p. 161). The expressions of the utility of disruption, and its measure of novelty, can be understood as positive heuristic appraisal as disruption is assessed as providing future directions.
Returning to the distinction highlighted between entities and properties as different notions of novelty in section 4.2, and the emphasis on the promise of the measurement of properties, it is again worth reflecting on the context of the timing of the interviews. The ATLAS and CMS experiments were entering into the 'data driven' phase, which they are currently in now, where the focus has shifted to precision measurements as opposed to searches. The appraisal that there is more 'promise' in precision measurements than searches is also a heuristic appraisal in that it has been appraised that there is the potential for future results that might be disruptive coming from measurements, as opposed to searches. This also helps explain the positive appraisal of "signature-based" searches over "model-based" searches, as identified by Wilson in section 4.1, that there is value in these searches in that they make possible unexpected results that have the potential to be disruptive.
Camilleri and Ritson have recently examined the role of heuristic appraisal in theoretical high-energy physics through an exploration of the role of heuristic appraisal in divergent assessments of string theory. They found that heuristic appraisal of unsolved and solved problems in theoretical approaches to quantum gravity were considered in the context of the research program as a whole (including the history) "rather than units that are constitutive of progress" (Camilleri & Ritson, 2015). This insight can be applied when attempting to explore disruption framed as a gauge of novelty in the previously discussed context of a notion of crisis in the high-energy physics programme, or a lack of direction from past guiding principles. Disruption is valued, in part, in that it might find a way to depart from the existing ontology and ways of knowing. In that context this value is positive heuristic appraisal, which is future focused as disruption provides different directions and creates different potential futures. This picture of where future progress is derived from is startlingly different from notions of progress dependent on the successful confirmation of theoretical predictions. It is instead a notion of progress based on hoped for contradictions to the existing theoretical and experimental structures, in the expectation that a contradictory result (the more disruptive the better) will have to be retrospectively explained and will require some level of destruction to existing theoretical structures. These expectations are formed from heuristic appraisals of disruption as offering alternative fertile futures.
Whilst the notion of futures driven from experimental disruption does contradict various models of scientific progress from HPS based on an epistemic value to confirmation, it is important to be careful with claims of uniqueness of the current period as a historically unique period in the history of particle physics. Such claims depend on at least a somewhat static understanding, or single direction, of progress historically, and this understanding does not reflect the diversity of theoretical and experimental practices explored in STS and HPS literature, especially following the insights from the experimentalist turn and the insights of Steinle into the history of physics (1997,2002). Similarly, it is important to be careful with claims of a crisis in high-energy particle physics. This notion, whilst it may come from a broadly Kuhnian perspective, is, as it stands, an actor's category employed in various ways by certain physicists. A more precise understanding of 19 CMS indicated that their result favoured a narrow width (CMS, 2015). 20 The theoretical response to the 750 GeV resonance also demonstrates this point from a different perspective. Many of the 500 þ theoretical papers published attempted to incorporate the signal (with either a broad or narrow width) into various BSM approaches, such as composite Higgs or dark matter models, however, were not easily able to do so. See for example (Franceschini et al., 2015). Reflecting on this in a summary talk theoretical physicist Strumia claimed that "the gg excess is either the biggest statistical fluctuation since decades, or the main discovery" (Strumia, 2016). both the notion of crisis and an argument for uniqueness is needed before there can be further development on these questions. As was outlined in the opening quotation, and emphasised throughout this paper, a notion of something being sufficiently different from the past as to command attention is difficult to articulate.

Conclusion
The aim of this paper was to explore how novelty is framed by the experimentalist interview participants that were involved in the 750 GeV episode, and to disambiguate between various uses of the concept. The aim was not to develop a comprehensive taxonomy of novelty, but instead to understand different aspects of the concept of novelty in the context of the 750 GeV episode and in the context of current epistemic environment of particle physics. The approach that was taken, to identify novelty, was to consider novelty as a valued difference. This approach was born out of consideration of lessons from the formal and detailed approaches in HPS literature to offer a conceptually driven analysis of the interview materials. The advantage of this approach was that it was not as broad as approaching novelty as only 'newness', but still allowed for the exploration of diverse results. Of course, this approach suffers from the same difficulties in that valued differences are also a broad category, but this approach also foregrounded the role of appraisal in how novelty is framed through the exploration of the attribution of value. The approach resulted in a relational approach to novelty, in that through the exploration of relations, invoked by interview participants, various expressions of novelty were analysed.
This relational approach revealed different reference classes of relations in which novelty can be identified. Property novelty and entity novelty were identified and distinguished in different descriptions by interview participants of the modes of research and epistemic practices in the context of the current phase of highenergy particle physics. The potential for novelty in measurements of properties was emphasised over searches. Furthermore, differing scales of difference, and therefore novelty, were identified through the exploration of the role of theoretical expectations. Where the less a result was expected, the more novel the result was conceived to be. That is to say, the more different and (positively) valued the result was conceived to be.
Understanding how results that differed from expectations could be construed as more novel came from a further exploration of the positive heuristic appraisal of results that require changes to the existing structures. This was developed from the observation that the positive appraisals are future focused and by drawing upon Nickles' notion of heuristic appraisal, modified for a perspective from experiment. On this modified account, heuristic appraisals of experiments or results are based on assessments of future fertility such as the likelihood of generating progress, or of opening up new areas for inquiry. This paper argues that it is disruption rather than theoretical confirmation that is most valued, in that it is judged as being able potentially to provide alternate future directions in a phase characterised as lacking direction from traditional constraints. This paper also introduced a notion of disruptive exploratory experimentation, which aims for disruptive novelty, as a complementary category of experimentation to exploratory experimentation (Elliott, 2007;Steinle, 1997Steinle, , 2002.
The approach taken in the paper is to consider novelty as a valued difference, an approach which also permits consideration of differences which have negative value. The perspective coming from the experimentalists interviewed for the paper is one of a positive value for differences, even epistemic disruption, needs to be understood in the context from which the claims were made. This paper, however, stops short of following some of the physicists in their portrayal of high-energy particle physics as experiencing a crisis or of labelling some of the current thinking and modes of appraisal of experimentalists guided by the positive heuristic appraisal of disruption as unique. It is unlikely that nothing similar to this can be found in the history of high-energy particle physics, as that would rest on assumption of an unchanging mode of experimental particle physics research. Whilst it is important to be careful of in how far we consider the current phase as 'exceptional', as that assumes a view that is not historically well informed, it is also important to reflect on some of the interesting aspects that this case study highlights. Disruption is very often considered pejoratively, so expressions of positive appraisals of the utility of disruption contribute to understandings of scientific appraisal, especially those based on heuristic appraisal. This exploration of the role of disruption also contributes to a more nuanced view of the iterative relationship between theory and experiment in highenergy physics.
Some questions that remain open include to what extent are there disagreements within the experimental collaborations in appraisals of novelty or the future fertility of novel ways of knowing. It is also not clear precisely how heuristic appraisal can play a role in the positive appraisal of novel ways of knowing. Clearly, facilitating potential novel results in the future forms part of the assessment, but how such assessments are made, and how disagreements resolved, for yet to be completed experiments (both within the LHC experiments and more broadly, such as in considerations of future generations of particle accelerators) remains to be seen. Finally, theoretical perspectives on disruption are unclear from this account, the timing and number of publications on the initial excess suggest a certain attribution of value for some, however the controversy surrounding these publications (Garisto, 2016) also suggests a complex picture.