Predictive physiological anticipation preceding seemingly unpredictable stimuli: An update of Mossbridge <i>et al</i>’s meta-analysis

Michael Duggan; Patrizio E. Tressoldi

doi:10.12688/f1000research.14330.1

Home Browse Predictive physiological anticipation preceding seemingly unpredictable...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Systematic Review

Predictive physiological anticipation preceding seemingly unpredictable stimuli: An update of Mossbridge et al’s meta-analysis

[version 1; peer review: 2 approved with reservations]

Michael Duggan¹, Patrizio E. Tressoldi ²

PUBLISHED 28 Mar 2018

Author details Author details

¹ Independent researcher, Birmingham, UK
² Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

Michael Duggan
Roles: Conceptualization, Data Curation, Investigation, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Patrizio E. Tressoldi
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: This is an update of the Mossbridge et al’s meta-analysis related to the physiological anticipation preceding seemingly unpredictable stimuli. The overall effect size observed was 0.21; 95% Confidence Intervals: 0.13 - 0.29
Methods: Eighteen new peer and non-peer reviewed studies completed from January 2008 to October 2017 were retrieved describing a total of 26 experiments and 34 associated effect sizes.
Results: The overall weighted effect size, estimated with a frequentist multilevel random model, was: 0.29; 95% Confidence Intervals: 0.19-0.38; the overall weighted effect size, estimated with a multilevel Bayesian model, was: 0.29; 95% Credible Intervals: 0.18-0.39. Effect sizes of peer reviewed studies were slightly higher: 0.38; Confidence Intervals: 0.27-0.48 than non-peer reviewed articles: 0.22; Confidence Intervals: 0.05-0.39. The statistical estimation of the publication bias by using the Copas model suggest that the main findings are not contaminated by publication bias.
Conclusions: In summary, with this update, the main findings reported in Mossbridge et al’s meta-analysis, are confirmed.

Keywords

pre-stimulus activity, anticipatory physiology, temporal processing, psychophysiology, presentiment

Corresponding author: Patrizio E. Tressoldi

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2018 Duggan M and Tressoldi PE. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Duggan M and Tressoldi PE. Predictive physiological anticipation preceding seemingly unpredictable stimuli: An update of Mossbridge et al’s meta-analysis [version 1; peer review: 2 approved with reservations]. F1000Research 2018, 7:407 (https://doi.org/10.12688/f1000research.14330.1) First published: 28 Mar 2018, 7:407 (https://doi.org/10.12688/f1000research.14330.1) Latest published: 17 Jul 2018, 7:407 (https://doi.org/10.12688/f1000research.14330.2)

Introduction

The human ability to predict future events has been crucial in our evolutionary development and proliferation over epochs of time, both from a species perspective, but also, on an individual level. Our day-to-day survival is predicated on a successful marriage of experience (e.g., memory) and sensory processing (e.g., perceptual cues); for example, on a very humid heavily overcast night, our perceptions and memories inform us that a thunder storm is possible and it might be intelligent to find shelter. Such behaviour is highly adaptive as it fosters survival based strategies and is perfectly explicable in terms of current theories of biological causality. Now imagine if such prognosticating ability was possible without any sensory or other inferential cues. Such seemingly inexplicable ability would definitely hold survival advantage, if they existed. For millennia people have been reporting strange feelings of foreboding that later transpired to have significance. Over the last 36 years these phenomena have been scrutinized in the laboratory in which a subject’s physiology is monitored before a randomly presented stimulus that is designed to evoke a significant post-stimulus response. Disturbingly, moments before the stimulus is presented there are murmurings of activity, as if the body is predicting moments ahead of time. This effect is termed presentiment, or more recently, Predictive Anticipatory Activity (Mossbridge et al., 2014). By 2012 a good number of these studies had been completed and it was deemed worthwhile to conduct a meta-analysis of the extant literature at the time. Mossbridge, Tressoldi and Utts located 42 studies published from 1978 to 2010, testing the presentiment hypothesis, out of which 26 enabled a true comparison between pre and post-stimulus epochs (Mossbridge et al., 2012), that is the pre-stimulus physiological responses mirrored even if to a lesser degree, the post-stimulus responses.

Here two paradigms were used: either a randomly ordered presentation of arousing vs. neutral stimuli or guessing tasks in which the stimulus is the feedback about the participant’s guess (correct vs. incorrect). In both of these approaches it is difficult to envision mundane strategies that might explain the anomalous pre-stimulus effects observed, and indeed, Mossbridge et al, went to significant lengths in refuting the leading candidate – expectancy effects, both in the 2012 meta-analysis and in post-review exchanges with sceptical psychologists and physiologists. Regardless of the paradigm, a broad range of physiological measures were employed from skin conductance, heart rate, blood volume, respiration, electroencephalographic (EEG) activity, pupil dilation, blink rate, and/or blood oxygenation level dependent (BOLD) responses. These are recorded throughout the session, with a pre-determined anticipatory period of between 4 to 10 seconds, in which the any pre-stimulus effect is captured. The presentiment hypothesis calls for a difference between arousing and neutral pre-stimulus responses and this is calculated across sessions. Mossbridge et al. found substantive evidence in favour of a presentiment effect concatenated to over 6 sigma – extreme statistical significance. Additionally, they also found evidence of presentiment effects from mainstream research programs – something that is becoming increasingly important as these effects become more widely known.

Because of the high profile nature of Mossbridge et al, (over 93,000 views as of January 2018) there has been a good number of replications in the few years since publication. We located an additional 26 studies describing 34 effect sizes from a dozen laboratories. The most striking aspect of this fresh database is the sheer variation in experimental approaches as researchers seek to tackle more process oriented questions rather than continuing the proof-oriented work found in the earlier meta-analysis. Because expectancy effects have been forwarded to explain at least some of the presentiment effect, it is noteworthy that several experiments in this fresh cohort of studies tackle this head on by only analysing the first trial of a run. These single-trial presentiment studies are expectancy free and are becoming more dominant in this research domain. Another interesting question that is probed in these new studies is the idea of utilizing pre-stimulus physiological activity to predict future events. This provides a second objective measure of the validity of the presentiment effect. There are several studies that utilize this approach and they are discussed later on. Additionally, we also found increasing evidence of presentiment research piggybacking onto mainstream psychology programs, even informing aspects of the conventional research. Also of note we found several PhD theses describing presentiment research and a greater geographical spread than in 2012, both evidence of the increasing attention such research is garnering. Lastly, we found increasing dialogue between presentiment researchers and physicists interested in retrocausality – the idea that effects can precede their cause. This is witnessed in the recent AAAS retrocausality symposium in which several researchers participated and in which some of those papers made their way into this meta-analysis (Sheehan, 2017).

Methods

The whole procedure followed both the APA Meta-Analysis Reporting Standards (APA Publications and Communications Board Working Group on Journal Article Reporting Standards, 2008), the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (Moher et al., 2015) and the reporting standards for literature searches and report inclusion (Atkinson et al., 2015). A completed PRISMA checklist can be found in Supplementary File 1.

Study eligibility criteria

Study inclusion criteria were the analysis of both psychophysiological or neurophysiological signals before the random presentation of whichever type of stimulus, e.g. pictures, sounds etc. Randomization could be performed by using pseudo-random algorithms e.g. like those implemented in MatLab or E-Prime® or true random sources of random digits, e.g. TrueRNG.

It is important to point out that these eligibility criteria are different from those used by Mossbridge et al. Those authors selected only studies were the anticipatory signals mirrored the post-stimulus ones. Differently we included all studies that used anticipatory signals to predict future events independently of the presence of post-stimulus physiological signals. For example, some authors, e.g. Mossbridge (2015) used heart rate variability to predict winning i.e. $4, versus losing outcomes. Our inclusion criteria are consequently more comprehensive than those used by Mossbridge et al.

Studies retrieval procedure

Both co-authors who are experts in this type of investigations, searched for studies through Google Scholar and PubMed by using the keywords: “presentiment” OR “anticipation” OR “precognition”. Furthermore, we emailed a request of the data of completed studies to all authors we knew were involved in this type of investigations. Even if Mossbridge et al. included all studies available up to 2010, we also searched studies that could have been missed in that meta-analysis. We searched all completed studies, both peer reviewed and non-peer reviewed, e.g. Ph.D dissertations, from January 2008 to October 2017.

Study selection

Study selection is illustrated in the flow-diagram presented in Figure 1

Figure 1. Flow-diagram of study selection.

Excluded records were studies were the psychophysiological variables were analysed only after and not before the stimuli presentations (Jin et al., 2013) and with an unusual procedure (Tressoldi et al., 2015), i.e. using heart rate feedback to inform a voluntary decision to predict random positive or negative events.

Records excluded after the screening were studies where authors did not agree to share their data for different reasons (Baumgart et al., 2017; Modestino et al., 2011). Excluded studies revealed either statistically significant or trending evidence in support of the anticipation effect in most cases, thus reducing the concerns surrounding biased removal.

The references of the included studies are reported in Supplementary File 2.

Coding procedure

The two co-authors agreed on the following coding variables: Authors; year of publication; participant selection: yes = selected according to specific criteria; no = selected without specific criteria; number of participants; number of trials; stimuli type; type of randomisation: pseudo or true random; psychophysiological signals, e.g. EEG, Heart Rate, etc.; anticipatory period; type of statistics; value of statistics and independently extracted them from the eligible studies. After the comparison, they discussed how to solve the inter-coder’ differences.

On the database we have added a note for each effect size, describing where we extracted the corresponding statistics in the original papers. The database along with all 18 papers are available from Tressoldi (2017). A summary of the selected studies along with their corresponding effect sizes, variance and standard error, is reported on Table S1 in the Supplementary File 3.

Moderator variables

Apart from the overall effect, we chose to compare the following moderator variables, peer review (PeerRev, yes vs no) as a control of study quality. Given the low number of studies no further moderator analyses were carried out.

Statistical methods

The standardized effect size d of each dependent variable, was estimated from the descriptive statistics (means, standard deviation and number of participants) when available. In all other cases, it was estimated by using the available summary statistics, i.e. paired t-test; Stouffer’s Z; etc. by using Lakens’ software (Lakens, 2013) and the function escalc () of the R package metaphor (Viechtbauer, 2017).

All effect sizes were then converted into the Hedges’ g and the corresponding variance by using the formulae suggested by Borenstein et al. (2009) estimating an average correlation of 0.5 between the dependent variables.

Given our choice of keeping (not averaging) all effect sizes when multiple dependent variables were analysed, we estimated the overall random model weighted effect size by using the robumeta package (Fischer et al., 2017) which implement a Robust Variance Estimation method when there are dependent effect sizes (Tanner-Smith & Tipton, 2014).

In order to control the reliability of the results, a second analysis was carried out by using a multilevel approach as suggested by (Assink & Wibbelink, 2016) implemented with the metafor package (Viechtbauer, 2010) and reported in the Table S2 in the Supplementary File 3.

The Bayesian meta-analysis was implemented with the brms package (Bürkner, 2017).

A copy of the syntax is available here: https://doi.org/10.6084/m9.figshare.5661070.v1 (Tressoldi, 2017)

Even if with our search activity we are quite sure to have reduced to a minimum the problem of publication bias, we performed a statistical estimation by using the Copas selection model which is recommended by Jin et al. (2015).

Results

Descriptive statistics

Studies: Peer review papers: 8; Non -Peer review papers:10. Number of experiments: 26 contributed by 13 authors. Number of effect sizes: 34. Average number of participants: 97.5. Average anticipatory period: 3.5 seconds. Four studies were preregistered (see database).

The group analyses for males and females reported in three papers (Mossbridge, 2014; Mossbridge, 2015; Singh, 2009), were considered independent effect sizes.

Frequentist multilevel random model

The forest plot is presented in Figure 2. The summary of the frequentist multilevel random model analysis is presented in Table 1 compared with the results obtained by Mossbridge et al., whereas the summary of the Bayesian multilevel random model meta-analysis is presented in Table 2.

Figure 2. Forest plot of the frequentist multilevel random model analysis.

Table 1. Results of the frequentist multilevel random model analysis.

	n	ES	95% Conf. Int.	p	I²	τ²
Mossbridge *et al.*	26	0.21	0.13 – 0.29	5.7×10^-8	27.4	0.012
Overall	26	0.29	0.19 – 0.38	8×10^-6	82.5	0.049
Peer Review	12	0.38	0.27 – 0.48	1×10^-5	43.5	0.012
No Peer Review	14	0.22	0.05 – 0.39	0.014	85.2	0.048

n= number of experiments; ES= estimated effect size with corresponding 95% confidence intervals, p values; I²: effect sizes heterogeneity; τ²: effect size variance heterogeneity.

Table 2. Results of the Bayesian Multilevel Random Model.

	n	Effect size	95% CI	Rhat
Overall	26	0.29	0.18 – 0.39	1
Peer Review	12	0.36	0.23 – 0.48	1
No Peer Review	14	0.23	0.04 – 0.42	1

Rhat = ratio of the average variance of samples within each chain to the variance of the pooled samples across chains. CI – Credible Intervals.

Sensitivity analysis of the overall effect size, didn’t reveal any change from Rho 0 to Rho 1, suggesting that the degree of correlations among the dependent effect sizes don’t affect its magnitude.

Another “sensitivity analysis” was carried out excluding the Mossbridge and the Tressoldi studies in order to control whether different authors could obtain similar results. The main results of this analysis by using the same frequentist multilevel random model, is reported in Table 3.

Table 3. Results of the frequentist multilevel random model without Mossbridge and Tressoldi studies.

	n	Effect size	95% CI	p	I²	τ²
Overall	19	0.23	0.05 – 0.40	0.017	82.7	0.065

I² = percentage of variation across studies that due to heterogeneity; τ² = Tau², variance of the true effect sizes. CI – Confidence Interval.

Both the frequentist and the Bayesian analyses support the evidence of an overall main effect of approximately .29, and a small difference between the peer and non-peer reviewed studies. These findings will be commented further in the discussion of the comparison with Mossbridge et al.

Publication bias

The search method used and the small number of people interested in this research field, guarantee that from an empirical point of view, any publication bias is almost absent.

Unfortunately, there is no consensus about what tests are statistically more valid (Carter et al., 2017).

All the traditional tests, like the Fail-Safe, the Trim-and-Fill, the Funnel Plot have been criticized for their limitations (Jin et al., 2015; Rothstein, 2008). We hence applied the Copas selection model which is recommended by Jin et al. (2015).

The Copas selection model was implemented using the metasens package (Schwarzer et al., 2016). The results are presented in the Table 4. With this statistic, it emerges that there is no apparent statistical publication bias.

Table 4. Estimated effect size and corresponding 95% Confidence Intervals (CI) of the Copas Model.

	Effect size	95% CI
Copas Model adjusted	0.28	0.20 – 0.37

Discussion

This update of the Mossbridge et al. (2012) meta-analysis related to the so called predictive anticipatory activity (PAA) responses to future random stimuli, covers the years 2008- October 2017. Overall, we found 18 new studies describing a total of 34 effect sizes. Differently from the statistical approach of Mossbridge et al., in this meta-analysis we used a frequentist and a Bayesian multilevel model which allows an analysis of all effect sizes reported within a single study instead of averaging them.

Both the frequentist and the Bayesian analyses converged on similar results, making our findings quite robust. The overall effect size 0.29, 95% CI = 0.18 - 0.39, overlaps to that reported in the original paper: 0.21, 95% CI = 0.13–0.29, even if the heterogeneity is substantially higher: I²= 80.5 vs 27.4.

The high level of heterogeneity is expected considering the varieties of experimental protocols and the diversity of dependent variables, from heart rate to pupil dilation.

Furthermore, we did not find substantial differences between peer and not-peer reviewed papers as in the original paper.

We found very interesting evidence of presentiment distilled from the conventional post-stimulus psychological research of Jolij and Bierman, who have performed a long series of experiments using a face detection paradigm. Additionally, the work of Kittenis found prestimulus effects from a conventional research program and pre-registered single-trial work of Mossbridge represent an important conceptual replication in countering both the use of questionable research practices and expectancy effects arguments.

A promising development of this line of research is the development of paradigms that use software in real-time to predict meaningful future outcomes before they occur, e.g. (Franklin et al., 2014)

Conclusion

This update confirms the main results reported in Mossbridge et al. (2012) original meta-analysis and gives further support to the hypothesis of predictive physiological anticipation of future random events.

The limitations of the present meta-analysis are similar to most meta-analyses which include non pre-registered studies that cannot be controlled for the degree of freedoms in the methodology and data analysis in the course of their implementations, making them prone, for example, to the so-called “questionable research practices” (John et al., 2012).

The solution is that of prospective meta-analyses (Watt & Kennedy, 2017), based on preregistered studies where the methods and data analyses have been declared and made public beforehand.

Data availability

Underlying data for this meta-analysis is available from FigShare: https://doi.org/10.6084/m9.figshare.5661070.v1 (Tressoldi, 2017) under a CC BY 4.0 licence

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Supplementary material

Supplementary File 1 – Completed PRISMA checklist.

Click here to access the data.

Supplementary File 2 – List of references used in this analysis.

Click here to access the data.

Supplementary File 3 – contains Table S1: Summary of the selected studies along with their corresponding effect sizes, variance and standard error. Table S2: results obtained with the multilevel approach suggested by Assink & Wibbelink, 2016.

Click here to access the data.

Faculty Opinions recommended

References

APA Publications and Communications Board Working Group on Journal Article Reporting Standards.Reporting standards for research in psychology: why do we need them? What might they be? Am Psychol. 2008; 63(9): 839–51. PubMed Abstract | Publisher Full Text | Free Full Text
Assink M, Wibbelink CJ: Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology. 2016; 12(3): 154–174. Publisher Full Text
Atkinson KM, Koenka AC, Sanchez CE, et al.: Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015; 6(1): 87–95. PubMed Abstract | Publisher Full Text
Baumgart SL, Franklin MS, Jimbo HK, et al.: Prediction of truly random future events using analysis of prestimulus electroencephalographic data. AIP Conference Proceedings. AIP Publishing. 2017; 1841(1). Publisher Full Text
Borenstein M, Hedges LV, Higgins JP, et al.: Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons, Ltd. 2009. Publisher Full Text
Bürkner PC: brms: An R Package for Bayesian Multilevel Models Using Stan. J Stat Softw. 2017; 80(1): 1–28. Publisher Full Text
Carter EC, Schönbrodt FD, Gervais W, et al.: Correcting for bias in psychology: A comparison of meta-analytic methods. 2017. Publisher Full Text
Fischer Z, Tipton E, Zhipeng H: Package robumeta. Retrieved 3 August 2017, 2017. Reference Source
Franklin MS, Baumgart SL, Schooler JW, et al.: Future directions in precognition research: more research can bridge the gap between skeptics and proponents. Front Psychol. 2014; 5: 907. PubMed Abstract | Publisher Full Text | Free Full Text
Jin Y, Yan K, Zhang Y, et al.: Gender differences in detecting unanticipated stimuli: an ERP study. Neurosci Lett. 2013; 538: 38–42. PubMed Abstract | Publisher Full Text
Jin ZC, Zhou XH, He J: Statistical methods for dealing with publication bias in meta-analysis. Stat Med. 2015; 34(2): 343–360. PubMed Abstract | Publisher Full Text
John LK, Loewenstein G, Prelec D: Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychol Sci. 2012; 23(5): 524–532. PubMed Abstract | Publisher Full Text
Lakens D: Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013; 4: 863. PubMed Abstract | Publisher Full Text | Free Full Text
Modestino EJ, Kelly EF, Ross Dunseath WJ, et al.: Anomalous physiological responses to local and remote emotive stimulation. Parapsychological Association Annual Convention, Curitiba. 2011.
Moher D, Stewart L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text
Mossbridge JA: Single-trial presentiment experiment. KPU Registry ID N 1005. 2014. Reference Source
Mossbridge JA: Single-trial confirmatory presentiment experiment. KPU Registry ID NV 1018. 2015. Reference Source
Mossbridge JA, Tressoldi P, Utts J, et al.: Predicting the unpredictable: Critical analysis and practical implications of predictive anticipatory activity. Front Hum Neurosci. 2014; 8: 146. PubMed Abstract | Publisher Full Text | Free Full Text
Mossbridge J, Tressoldi P, Utts J: Predictive physiological anticipation preceding seemingly unpredictable stimuli: A meta-analysis. Front Psychol. 2012; 3: 390. PubMed Abstract | Publisher Full Text | Free Full Text
Rothstein HR: Publication bias as a threat to the validity of meta-analytic results. J Exp Criminol. 2008; 4(1): 61–81. Publisher Full Text
Schwarzer G, Carpenter J, Rücker G: Package metasens. 2016. Reference Source
Sheehan DP: Preface and Acknowledgements: Quantum Retrocausation III. 2017; 1841(1): 10001. Publisher Full Text
Singh PK: Personality correlates to electrophysiological measures of prestimulus response. Dissertation presented to Institute of Transpersonal Psychology, Palo Alto, California. 2009. Reference Source
Tanner-Smith EE, Tipton E: Robust variance estimation with dependent effect sizes: practical considerations including a software tutorial in Stata and spss. Res Synth Methods. 2014; 5(1): 13–30. PubMed Abstract | Publisher Full Text
Tressoldi PE, Martinelli M, Torre J, et al.: CardioAlert: A Heart Rate based decision support system for improving choices related to negative or positive future events. 2015. Publisher Full Text
Tressoldi P: Mossbridge's et al. 2012 meta-analysis update. Figshare. 2017. Publisher Full Text
Viechtbauer W: Conducting Meta-Analyses in R with the metafor Package. J Stat Softw. 2010; 36(3). Publisher Full Text
Viechtbauer W: The metafor Package. Retrieved 3 August 2017. Reference Source
Watt CA, Kennedy JE: Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis. Front Psychol. 2017; 7: 2030. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 28 Mar 2018

Author details Author details

¹ Independent researcher, Birmingham, UK
² Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

Michael Duggan
Roles: Conceptualization, Data Curation, Investigation, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Update

Published: 17 Jul 2018, 7:407

https://doi.org/10.12688/f1000research.14330.2

version 1

Published: 28 Mar 2018, 7:407

https://doi.org/10.12688/f1000research.14330.1

© 2018 Duggan M and Tressoldi PE. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Duggan M and Tressoldi PE. Predictive physiological anticipation preceding seemingly unpredictable stimuli: An update of Mossbridge et al’s meta-analysis [version 1; peer review: 2 approved with reservations] F1000Research 2018, 7:407 (https://doi.org/10.12688/f1000research.14330.1)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 28 Mar 2018

Views

Reviewer Report 05 Jul 2018

Stephen Baumgart, Department of Psychology and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.15593.r35197

Addressing Major Criticisms

This is a controversial topic and careful consideration of objections is needed in a meta-analysis. Presentiment or Predictive Physiological Anticipation Studies (PAA) are typically criticized on these grounds (see, for example, Wagenmakers, Wetzels, Borsboom, ... Continue reading

Physical impossibility
File-drawer effect
Biases due to multiple comparisons or p-hacking

As for the first criticism, discussion of physical plausibility is beyond the scope of this meta-analysis and is left to the discretion of the authors. Nevertheless, apparent violations of our intuitions of time are found at the quantum level, such as the Wheeler Delayed-Choice experiment. It is not impossible that such effects may scale up to a macroscopic level in a not-yet-understood emergent process. I think the final two sentences of the introduction satisfy considerations of the physical impossibility objection and no changes are needed.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

By far the most serious concern is the third, that of multiple comparisons or p-hacking, which I do not believe is adequately addressed by either the discussion or conclusion sections. Two sentences in the conclusion are not sufficient to address this serious concern. I have included recommendations later in this review. I am aware the authors already know the following but by doing multiple analyses and only reporting a sub-sample of them a believer or supporter of a hypothesis could bias effect sizes up while a skeptic or opponent could bias effect sizes down (and none of these biases are necessarily intentional or even conscious).

In the context of PAA, serious sources of p-hacking concern are establishing baselines for electrophysiological data, deciding time regions for analysis, and methodologies for rejecting bad data and artifacts. For some physiological measurements, the problem is even worse. In Electroencephalography (EEG) studies, for example, a researcher could either study event-related potentials (ERPs), the spectral power densities of various oscillations, or the phases of such oscillations, or a host of other possible analyses. Considering oscillations, the frequency range of an analysis can also be freely selected. Additionally, a researcher could select different bandpass filters to use or even which section of the head is included in the analysis. This is in addition to the concerns with artifact rejection, time region, and baselining already discussed. With so many free parameters, a non-preplanned study is practically useless as hard evidence for an effect unless the statistical significance of the effect is high enough that it becomes implausible that the effect in question can be generated by tweaking free parameters. Even if the statistical significance is high, the effect size is still untrustworthy because an analyst could be tweaking parameters in an effort to improve the analysis or fix problems but is only homing in on statistical fluctuations. These concerns are one reason why I refused to include exploratory EEG research from my own lab in this meta-analysis.

The solution to the multiple analysis problem is to separate research into exploratory studies where adjustments can be made in analysis and pre-planned confirmatory studies. Some of the studies included in the meta-analysis are pre-planned confirmatory studies, which should be considered the only truly reliable results for estimates of effect size due to the concerns laid out in this review (even for confirmatory studies, mistakes by researchers could distort effect sizes but these mistakes may average out in the long run).

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section
Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two
For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)
Show whether multiple comparison corrections were made for exploratory studies

Exploratory studies are necessary for advancing the field. But a meta-analysis should not include them without major caveats due to potential distortions of the effect size.

I am aware the extra attention given to p-hacking risks in this research is not precedented by other fields but the small effect sizes and the major implications to our understanding of physics, psychology, and neuroscience PAA research engenders may justify additional caution be used. My colleagues and I discuss this further in Schooler, Baumgart, & Franklin, 2018.

Other Comments

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).
Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Are the rationale for, and objectives of, the Systematic Review clearly stated?

Yes
Are sufficient details of the methods and analysis provided to allow replication by others?

Yes
Is the statistical analysis and its interpretation appropriate?

Partly
Are the conclusions drawn adequately supported by the results presented in the review?

Partly

References

1. Galak J, Leboeuf RA, Nelson LD, Simmons JP: Correcting the past: failures to replicate ψ.J Pers Soc Psychol. 2012; 103 (6): 933-48 PubMed Abstract | Publisher Full Text
2. Schooler J, Baumgart S, Franklin M: Entertaining without endorsing: The case for the scientific investigation of anomalous cognition.Psychology of Consciousness: Theory, Research, and Practice. 2018; 5 (1): 63-77 Publisher Full Text
3. Simmons JP, Nelson LD, Simonsohn U: False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant.Psychol Sci. 2011; 22 (11): 1359-66 PubMed Abstract | Publisher Full Text
4. Wagenmakers EJ, Wetzels R, Borsboom D, Kievit RA, et al.: A Skeptical Eye on Psi. In E. C. May and S. B. Marwaha (Eds.), Extrasensory Perception: Support, Skepticism, and Science. Santa Barbara,CA: ABC-CLIO. 2015. 285-316

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

17 Jul 2018

Author Response
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results ... Continue reading
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

Reply: we think we have a sufficient expertise in dealing with this problem. Furthermore we consulted with R.C.M. van Aert who is an expert on this topic.

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section

Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two

Reply: we have added a direct comparison between preregistered and no-preregistered studies, see Table 4 and the paragraph “Preregistered vs No-preregistered studies”

For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)

Reply: Unfortunately no one study checked this moderating variable, but our sensitivity analysis reported in Table 3, suggests that the experimenter expectation did not affect considerably the overall results.

Show whether multiple comparison corrections were made for exploratory studies

Reply: our choice to use multivariate analyses, partly reduce the impact of this procedure.

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).

Reply: revised as “The presentiment hypothesis calls for a difference between the pre-stimulus responses of the two stimulus categories..”

Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Reply: we expanded our conclusion as suggested.
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

Reply: we think we have a sufficient expertise in dealing with this problem. Furthermore we consulted with R.C.M. van Aert who is an expert on this topic.

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section

Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two

Reply: we have added a direct comparison between preregistered and no-preregistered studies, see Table 4 and the paragraph “Preregistered vs No-preregistered studies”

For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)

Reply: Unfortunately no one study checked this moderating variable, but our sensitivity analysis reported in Table 3, suggests that the experimenter expectation did not affect considerably the overall results.

Show whether multiple comparison corrections were made for exploratory studies

Reply: our choice to use multivariate analyses, partly reduce the impact of this procedure.

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).

Reply: revised as “The presentiment hypothesis calls for a difference between the pre-stimulus responses of the two stimulus categories..”

Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Reply: we expanded our conclusion as suggested.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

17 Jul 2018

Author Response
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results ... Continue reading
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

Reply: we think we have a sufficient expertise in dealing with this problem. Furthermore we consulted with R.C.M. van Aert who is an expert on this topic.

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section

Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two

Reply: we have added a direct comparison between preregistered and no-preregistered studies, see Table 4 and the paragraph “Preregistered vs No-preregistered studies”

For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)

Reply: Unfortunately no one study checked this moderating variable, but our sensitivity analysis reported in Table 3, suggests that the experimenter expectation did not affect considerably the overall results.

Show whether multiple comparison corrections were made for exploratory studies

Reply: our choice to use multivariate analyses, partly reduce the impact of this procedure.

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).

Reply: revised as “The presentiment hypothesis calls for a difference between the pre-stimulus responses of the two stimulus categories..”

Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Reply: we expanded our conclusion as suggested.
Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

Reply: we think we have a sufficient expertise in dealing with this problem. Furthermore we consulted with R.C.M. van Aert who is an expert on this topic.

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section

Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two

Reply: we have added a direct comparison between preregistered and no-preregistered studies, see Table 4 and the paragraph “Preregistered vs No-preregistered studies”

For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)

Reply: Unfortunately no one study checked this moderating variable, but our sensitivity analysis reported in Table 3, suggests that the experimenter expectation did not affect considerably the overall results.

Show whether multiple comparison corrections were made for exploratory studies

Reply: our choice to use multivariate analyses, partly reduce the impact of this procedure.

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).

Reply: revised as “The presentiment hypothesis calls for a difference between the pre-stimulus responses of the two stimulus categories..”

Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Reply: we expanded our conclusion as suggested.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 10 Apr 2018

David Vernon, Canterbury Christ Church University (CCCU), Kent, UK

Approved with Reservations

https://doi.org/10.5256/f1000research.15593.r32577

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.
P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.
Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’
P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.
P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.
P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Results
Keep tense to past ie peer reviewed not review.

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

I don’t see the moderation results for PeerRev reported here?

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

The point relating to the work of Jolij and Bierman is again vague and unclear. What evidence precisely are you referring to here and how/why is this similar to the ‘psychological research’ [and what does this refer to?]
What is the ‘conventional research program’ of Kittenis?
How, exactly, does the single trial work of Mossbridge counter QRP?

Are the rationale for, and objectives of, the Systematic Review clearly stated?

Yes
Are sufficient details of the methods and analysis provided to allow replication by others?

No
Is the statistical analysis and its interpretation appropriate?

Partly
Are the conclusions drawn adequately supported by the results presented in the review?

Yes

Competing Interests: No competing interests were disclosed.

CITE

Report a concern

Author Response 17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

17 Jul 2018

Author Response

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting ... Continue reading Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’

P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
Reply: we changed with “‘physiological changes ahead of time”.

P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Reply: No

Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
Reply: No

P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
Reply: Added reference

P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
Reply: replaced with "proposed as a potential mechanism".

P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.

Reply: changed as “another way..”

P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Reply: deleted this paragraph

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.
Reply: added

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’
Reply: changed accordingly

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

Reply: We prefer the term more comprehensive because some experimental designs, e.g. hit guessing, don’t allow a post-stimulus physiological measure. However, all experimental designs tied the differential anticipatory physiological activity to two different outcomes, e.g. hits or misses.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.

Reply: fixed.

Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

Reply: after Mossbridge et al publication, we discovered that Singh, P.K. (2009), was missed.

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’

Reply: fixed.

P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.

Reply: the reasons for such decisions are confidential.

P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.

Reply: fixed

P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
Reply: fixed

P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Reply: fixed

Results
Keep tense to past ie peer reviewed not review.
Reply: fixed

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

Reply: we clarified that the Mossbridge and Tressoldi's studies were those included in this update.

I don’t see the moderation results for PeerRev reported here?

Reply: we think this analysis redundant given the data reported on Tables 1 and 2

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Reply: we clarified that the means are different, but their precision estimate, i.e. confidence intervals, overlap.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Reply: fixed

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

Reply: we changed the discussion and the conclusion to include our evaluation of the status of art and the future of this phenomenon.

What is the ‘conventional research program’ of Kittenis?
Reply: omitted

How, exactly, does the single trial work of Mossbridge counter QRP?

Reply: we wrote “pre-registered single-trial work”. Preregistration of data analyses constraints the use of post-hoc data analysis flexibility.

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’

P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
Reply: we changed with “‘physiological changes ahead of time”.

P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Reply: No

Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
Reply: No

P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
Reply: Added reference

P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
Reply: replaced with "proposed as a potential mechanism".

P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.

Reply: changed as “another way..”

P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Reply: deleted this paragraph

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.
Reply: added

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’
Reply: changed accordingly

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

Reply: We prefer the term more comprehensive because some experimental designs, e.g. hit guessing, don’t allow a post-stimulus physiological measure. However, all experimental designs tied the differential anticipatory physiological activity to two different outcomes, e.g. hits or misses.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.

Reply: fixed.

Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

Reply: after Mossbridge et al publication, we discovered that Singh, P.K. (2009), was missed.

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’

Reply: fixed.

P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.

Reply: the reasons for such decisions are confidential.

P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.

Reply: fixed

P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
Reply: fixed

P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Reply: fixed

Results
Keep tense to past ie peer reviewed not review.
Reply: fixed

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

Reply: we clarified that the Mossbridge and Tressoldi's studies were those included in this update.

I don’t see the moderation results for PeerRev reported here?

Reply: we think this analysis redundant given the data reported on Tables 1 and 2

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Reply: we clarified that the means are different, but their precision estimate, i.e. confidence intervals, overlap.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Reply: fixed

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

Reply: we changed the discussion and the conclusion to include our evaluation of the status of art and the future of this phenomenon.

What is the ‘conventional research program’ of Kittenis?
Reply: omitted

How, exactly, does the single trial work of Mossbridge counter QRP?

Reply: we wrote “pre-registered single-trial work”. Preregistration of data analyses constraints the use of post-hoc data analysis flexibility.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

17 Jul 2018

Author Response

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting ... Continue reading Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’

P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
Reply: we changed with “‘physiological changes ahead of time”.

P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Reply: No

Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
Reply: No

P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
Reply: Added reference

P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
Reply: replaced with "proposed as a potential mechanism".

P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.

Reply: changed as “another way..”

P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Reply: deleted this paragraph

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.
Reply: added

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’
Reply: changed accordingly

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

Reply: We prefer the term more comprehensive because some experimental designs, e.g. hit guessing, don’t allow a post-stimulus physiological measure. However, all experimental designs tied the differential anticipatory physiological activity to two different outcomes, e.g. hits or misses.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.

Reply: fixed.

Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

Reply: after Mossbridge et al publication, we discovered that Singh, P.K. (2009), was missed.

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’

Reply: fixed.

P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.

Reply: the reasons for such decisions are confidential.

P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.

Reply: fixed

P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
Reply: fixed

P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Reply: fixed

Results
Keep tense to past ie peer reviewed not review.
Reply: fixed

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

Reply: we clarified that the Mossbridge and Tressoldi's studies were those included in this update.

I don’t see the moderation results for PeerRev reported here?

Reply: we think this analysis redundant given the data reported on Tables 1 and 2

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Reply: we clarified that the means are different, but their precision estimate, i.e. confidence intervals, overlap.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Reply: fixed

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

Reply: we changed the discussion and the conclusion to include our evaluation of the status of art and the future of this phenomenon.

What is the ‘conventional research program’ of Kittenis?
Reply: omitted

How, exactly, does the single trial work of Mossbridge counter QRP?

Reply: we wrote “pre-registered single-trial work”. Preregistration of data analyses constraints the use of post-hoc data analysis flexibility.

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’

P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
Reply: we changed with “‘physiological changes ahead of time”.

P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Reply: No

Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
Reply: No

P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
Reply: Added reference

P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
Reply: replaced with "proposed as a potential mechanism".

P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.

Reply: changed as “another way..”

P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Reply: deleted this paragraph

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.
Reply: added

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’
Reply: changed accordingly

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

Reply: We prefer the term more comprehensive because some experimental designs, e.g. hit guessing, don’t allow a post-stimulus physiological measure. However, all experimental designs tied the differential anticipatory physiological activity to two different outcomes, e.g. hits or misses.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.

Reply: fixed.

Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

Reply: after Mossbridge et al publication, we discovered that Singh, P.K. (2009), was missed.

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’

Reply: fixed.

P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.

Reply: the reasons for such decisions are confidential.

P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.

Reply: fixed

P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
Reply: fixed

P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Reply: fixed

Results
Keep tense to past ie peer reviewed not review.
Reply: fixed

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

Reply: we clarified that the Mossbridge and Tressoldi's studies were those included in this update.

I don’t see the moderation results for PeerRev reported here?

Reply: we think this analysis redundant given the data reported on Tables 1 and 2

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Reply: we clarified that the means are different, but their precision estimate, i.e. confidence intervals, overlap.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Reply: fixed

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

Reply: we changed the discussion and the conclusion to include our evaluation of the status of art and the future of this phenomenon.

What is the ‘conventional research program’ of Kittenis?
Reply: omitted

How, exactly, does the single trial work of Mossbridge counter QRP?

Reply: we wrote “pre-registered single-trial work”. Preregistration of data analyses constraints the use of post-hoc data analysis flexibility.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 28 Mar 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (update) 17 Jul 18	read	read
Version 1 28 Mar 18	read	read

David Vernon, Canterbury Christ Church University (CCCU), Kent, UK
Stephen Baumgart, University of California, Santa Barbara, Santa Barbara, USA

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

15 Views

04 Sep 2018 | for Version 2

David Vernon, Canterbury Christ Church University (CCCU), Kent, UK

15 Views Cite this report Responses(0)

Approved

I'm happy that the authors have addressed most of my prior comments - however I would still liked to have seen a more fuller discussion of what these results entail and their implications.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

14 Views

01 Aug 2018 | for Version 2

Stephen Baumgart, Department of Psychology and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, USA

14 Views Cite this report Responses(0)

Approved

I think the addition of the "Preregistered versus No-pregistered" section, analysis, and table adequately satisfies the serious concerns that I have about p-hacking. I switch my status to "Approved". Even though I'm still concerned about including non-preregistered studies at all, I realize that such a restriction does not have strong precedent. But I would still encourage authors to focus on meta-analysis of preregistered studies in the future.

One minor comment which I have on the writing is that "No-pregistered" should be "Non-preregistered".

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

15 Views

05 Jul 2018 | for Version 1

Stephen Baumgart, Department of Psychology and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, USA

15 Views Cite this report Responses(1)

Approved With Reservations

Physical impossibility
File-drawer effect
Biases due to multiple comparisons or p-hacking

More discussion of the risks of p-hacking in biasing results in the discussion section
Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two
For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)
Show whether multiple comparison corrections were made for exploratory studies

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).
Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Are the rationale for, and objectives of, the Systematic Review clearly stated?

Yes
Are sufficient details of the methods and analysis provided to allow replication by others?

Yes
Is the statistical analysis and its interpretation appropriate?

Partly
Are the conclusions drawn adequately supported by the results presented in the review?

Partly

References

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Though file-drawer effects are frequently cited as a serious concern, the results section adequately discusses this issue. However, expert review is needed for this area (my response to "Is the statistical analysis and its interpretation appropriate? " should really be a combination of "Partly" and "A qualified statistician is needed".) I agree with the first sentence of the "Publication Bias" subsection that publication bias is not that serious of a concern because of the limited number of researchers and available funding.

Reply: we think we have a sufficient expertise in dealing with this problem. Furthermore we consulted with R.C.M. van Aert who is an expert on this topic.

My recommended solutions for this paper are:

More discussion of the risks of p-hacking in biasing results in the discussion section
Separated analyses of pre-registered confirmatory studies and exploratory studies and discussion comparing the two

Reply: we have added a direct comparison between preregistered and no-preregistered studies, see Table 4 and the paragraph “Preregistered vs No-preregistered studies”

For exploratory studies in the study tables, include the experimenter expectation of whether the hypothesis will be verified (such as in Galak, LeBoeuf, Nelson, & Simmons, 2012)

Reply: Unfortunately no one study checked this moderating variable, but our sensitivity analysis reported in Table 3, suggests that the experimenter expectation did not affect considerably the overall results.

Show whether multiple comparison corrections were made for exploratory studies

Reply: our choice to use multivariate analyses, partly reduce the impact of this procedure.

“The presentiment hypothesis calls for a difference between arousing and neural pre-stimulus response and this is calculated across sessions” is not always true. For example, the hypothesis could also cover the difference between two different types of arousing stimulus (for example, auditory versus visual stimulus or two different types of visual stimulus).

Reply: revised as “The presentiment hypothesis calls for a difference between the pre-stimulus responses of the two stimulus categories..”

Further discussion should be included for the observations mentioned of the second-to-last paragraph of the discussion; otherwise, it may be unclear why these studies are interesting as the paper asserts.

Reply: we expanded our conclusion as suggested.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

18 Views

10 Apr 2018 | for Version 1

David Vernon, Canterbury Christ Church University (CCCU), Kent, UK

18 Views Cite this report Responses(1)

Approved With Reservations

Are the rationale for, and objectives of, the Systematic Review clearly stated?

Yes
Are sufficient details of the methods and analysis provided to allow replication by others?

No
Is the statistical analysis and its interpretation appropriate?

Partly
Are the conclusions drawn adequately supported by the results presented in the review?

Yes

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

17 Jul 2018

Patrizio Tressoldi, Dipartimento di Psicologia Generale, Universita di Padova, Padova, Italy

Thank you for your detailed and constructive comments.
Here it follows our replies to your main comments.

Introduction
P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’

P.2, Line 21: Not sure I would agree with ‘body predicting moments ahead of time’ as this suggests understanding – try ‘reacting ahead of time’ or simply ‘physiological changes ahead…’
Reply: we changed with “‘physiological changes ahead of time”.

P.2: Para 2: the authors note that two paradigms were used, presentation of arousing/neutral stimuli or guessing tasks. Were any clear differences in PAA effects reported between these tasks?
Reply: No

Also, given the ‘broad range of physiological measures’ used to assess such changes were there any key differences here?
Reply: No

P.2, Para 2, final sentence: the ‘evidence from mainstream research’ – what specifically does this refer to? Behavioural effects? Ie changes in accuracy and/or response times and if so could do with a clear reference.
Reply: Added reference

P. 2, Para 3, line 9: ‘forwarded’ doesn’t make sense. Do you mean ‘proposed as a potential framework/theory’?
Reply: replaced with "proposed as a potential mechanism".

P. 2, Para 3: Not sure I’d agree that using physiological markers to ‘predict’ future events is a ‘second objective’ measure. It is simply another way to view the same procedure.

Reply: changed as “another way..”

P. 2, Para 3: the vague references to ‘presentiment piggybacking onto mainstream research’ needs clarifying and supporting with references.

Reply: deleted this paragraph

Methods
P.2, Para 1: need to identify the acronym ‘PRISMA’ after it is outlined.
Reply: added

P. 2, Para 3, line 3: change ‘were’ to ‘where’
……………….., line 4: change ‘Differently’ to ‘In addition,’
Reply: changed accordingly

Also, what is the rationale for utilising a distinct eligibility criterion? It seems that prior research focused on testing for a pre-stim signal that would match the post-stim presentation. By not using this method you open yourself up to the criticism of widening the scope and also of looking for ‘any physiological change’ as opposed to one that would be specifically linked to the presentation of the target. The authors claim this is ‘more comprehensive’ but it could just as easily be seen as less conservative.

Reply: We prefer the term more comprehensive because some experimental designs, e.g. hit guessing, don’t allow a post-stimulus physiological measure. However, all experimental designs tied the differential anticipatory physiological activity to two different outcomes, e.g. hits or misses.

P.3, line 4: change to ‘this type of investigation’
Line 8: change ‘investigations’ to ‘research’.

Reply: fixed.

Line 8: The point about studies possibly ‘missed’ by Mossbridge et al is not clear. What makes you think any studies were ‘missed’ and why did you then include the same time period – ie from 2008 to 2010 – if you are ‘adding’ to the data it would make sense to begin your inclusion time from 2010 unless you have evidence that some studies were ‘missed’?

Reply: after Mossbridge et al publication, we discovered that Singh, P.K. (2009), was missed.

P.3, Para 4: line 1: change ‘were studies were’ to ‘were studies where’

Reply: fixed.

P.3 – is it possible to say a bit more about why some authors did not agree to share their data – looks distinctly odd.

Reply: the reasons for such decisions are confidential.

P.4, Para 6: sentence referring to ‘Assink’ doesn’t make sense – unless you move the ref out of parenthesis and into the sentence.

Reply: fixed

P.4: Change ‘The Bayesian’ to ‘A Bayesian’. And pull the sentence with syntax to the same paragraph.
Reply: fixed

P. 4: Change: ‘Even if with our search activity we are quite….’ To ‘The robust search is likely to have reduced the probability of a publication bias occurring. Nevertheless, to test this a statistical estimation was conducted using the Copas selection model, as recommended by Jin et al’

Reply: fixed

Results
Keep tense to past ie peer reviewed not review.
Reply: fixed

It doesn’t make sense to compare data from the current review to Mossbridge et al ‘if’ both sets of data contain the same studies – as this would lead to obvious similarities etc. To an extent this seems to be addressed by the data in Table 3 but not made clearly – ie why not simply state that when X studies were excluded due to Y reasons the overall effect was still significant?

Reply: we clarified that the Mossbridge and Tressoldi's studies were those included in this update.

I don’t see the moderation results for PeerRev reported here?

Reply: we think this analysis redundant given the data reported on Tables 1 and 2

The reported ‘small difference between the peer reviewed and non-peer reviewed’ is vague and unhelpful. State clearly what was found – ie, are they ‘significantly different’ if not then they are not ‘different’ in any meaningful way.

Reply: we clarified that the means are different, but their precision estimate, i.e. confidence intervals, overlap.

Under ‘Publication bias’ I think para 2, 3 and 4 (which appears on P.6) should be joined as one single paragraph.

Reply: fixed

Discussion
This is rather poor and reads like a list of points. There needs to be some discussion here not simply a repetition of the data. Ie – given this effect size how would the authors attempt to account for it? what are the implications of such a finding? Is there any scope for teasing out of the data any factors that may/may not influence the outcome – e.g., a possible relationship between the PAA and the various DV measures used?

Reply: we changed the discussion and the conclusion to include our evaluation of the status of art and the future of this phenomenon.

What is the ‘conventional research program’ of Kittenis?
Reply: omitted

How, exactly, does the single trial work of Mossbridge counter QRP?

Reply: we wrote “pre-registered single-trial work”. Preregistration of data analyses constraints the use of post-hoc data analysis flexibility.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] APA Publications and Communications Board Working Group on Journal Article Reporting Standards.Reporting standards for research in psychology: why do we need them? What might they be? Am Psychol. 2008; 63(9): 839–51. PubMed Abstract | Publisher Full Text | Free Full Text

[2] Assink M, Wibbelink CJ: Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology. 2016; 12(3): 154–174. Publisher Full Text

[3] Atkinson KM, Koenka AC, Sanchez CE, et al.: Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015; 6(1): 87–95. PubMed Abstract | Publisher Full Text

[4] Baumgart SL, Franklin MS, Jimbo HK, et al.: Prediction of truly random future events using analysis of prestimulus electroencephalographic data. AIP Conference Proceedings. AIP Publishing. 2017; 1841(1). Publisher Full Text

[5] Borenstein M, Hedges LV, Higgins JP, et al.: Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons, Ltd. 2009. Publisher Full Text

[6] Bürkner PC: brms: An R Package for Bayesian Multilevel Models Using Stan. J Stat Softw. 2017; 80(1): 1–28. Publisher Full Text

[7] Carter EC, Schönbrodt FD, Gervais W, et al.: Correcting for bias in psychology: A comparison of meta-analytic methods. 2017. Publisher Full Text

[8] Fischer Z, Tipton E, Zhipeng H: Package robumeta. Retrieved 3 August 2017, 2017. Reference Source

[9] Franklin MS, Baumgart SL, Schooler JW, et al.: Future directions in precognition research: more research can bridge the gap between skeptics and proponents. Front Psychol. 2014; 5: 907. PubMed Abstract | Publisher Full Text | Free Full Text

[10] Jin Y, Yan K, Zhang Y, et al.: Gender differences in detecting unanticipated stimuli: an ERP study. Neurosci Lett. 2013; 538: 38–42. PubMed Abstract | Publisher Full Text

[11] Jin ZC, Zhou XH, He J: Statistical methods for dealing with publication bias in meta-analysis. Stat Med. 2015; 34(2): 343–360. PubMed Abstract | Publisher Full Text

[12] John LK, Loewenstein G, Prelec D: Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychol Sci. 2012; 23(5): 524–532. PubMed Abstract | Publisher Full Text

[13] Lakens D: Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013; 4: 863. PubMed Abstract | Publisher Full Text | Free Full Text

[14] Modestino EJ, Kelly EF, Ross Dunseath WJ, et al.: Anomalous physiological responses to local and remote emotive stimulation. Parapsychological Association Annual Convention, Curitiba. 2011.

[15] Moher D, Stewart L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text

[16] Mossbridge JA: Single-trial presentiment experiment. KPU Registry ID N 1005. 2014. Reference Source

[17] Mossbridge JA: Single-trial confirmatory presentiment experiment. KPU Registry ID NV 1018. 2015. Reference Source

[18] Mossbridge JA, Tressoldi P, Utts J, et al.: Predicting the unpredictable: Critical analysis and practical implications of predictive anticipatory activity. Front Hum Neurosci. 2014; 8: 146. PubMed Abstract | Publisher Full Text | Free Full Text

[19] Mossbridge J, Tressoldi P, Utts J: Predictive physiological anticipation preceding seemingly unpredictable stimuli: A meta-analysis. Front Psychol. 2012; 3: 390. PubMed Abstract | Publisher Full Text | Free Full Text

[20] Rothstein HR: Publication bias as a threat to the validity of meta-analytic results. J Exp Criminol. 2008; 4(1): 61–81. Publisher Full Text

[21] Schwarzer G, Carpenter J, Rücker G: Package metasens. 2016. Reference Source

[22] Sheehan DP: Preface and Acknowledgements: Quantum Retrocausation III. 2017; 1841(1): 10001. Publisher Full Text

[23] Singh PK: Personality correlates to electrophysiological measures of prestimulus response. Dissertation presented to Institute of Transpersonal Psychology, Palo Alto, California. 2009. Reference Source

[24] Tanner-Smith EE, Tipton E: Robust variance estimation with dependent effect sizes: practical considerations including a software tutorial in Stata and spss. Res Synth Methods. 2014; 5(1): 13–30. PubMed Abstract | Publisher Full Text

[25] Tressoldi PE, Martinelli M, Torre J, et al.: CardioAlert: A Heart Rate based decision support system for improving choices related to negative or positive future events. 2015. Publisher Full Text

[26] Tressoldi P: Mossbridge's et al. 2012 meta-analysis update. Figshare. 2017. Publisher Full Text

[27] Viechtbauer W: Conducting Meta-Analyses in R with the metafor Package. J Stat Softw. 2010; 36(3). Publisher Full Text

[28] Viechtbauer W: The metafor Package. Retrieved 3 August 2017. Reference Source

[29] Watt CA, Kennedy JE: Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis. Front Psychol. 2017; 7: 2030. PubMed Abstract | Publisher Full Text | Free Full Text

Predictive physiological anticipation preceding seemingly unpredictable stimuli: An update of Mossbridge et al’s meta-analysis

Abstract

Keywords

Introduction

Methods

Study eligibility criteria

Studies retrieval procedure

Study selection

Figure 1. Flow-diagram of study selection.

Coding procedure

Moderator variables

Statistical methods

Results

Descriptive statistics

Frequentist multilevel random model

Figure 2. Forest plot of the frequentist multilevel random model analysis.

Table 1. Results of the frequentist multilevel random model analysis.

Table 2. Results of the Bayesian Multilevel Random Model.

Table 3. Results of the frequentist multilevel random model without Mossbridge and Tressoldi studies.

Publication bias

Table 4. Estimated effect size and corresponding 95% Confidence Intervals (CI) of the Copas Model.

Discussion

Conclusion

Data availability

Competing interests

Grant information

Supplementary material

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated