Methodological approaches for developing and reporting living evidence synthesis: a study protocol

Background: Living evidence (LE) refers to the methodological processes that permit new research findings to be continually incorporated into evidence synthesis. This approach is of great value in the resolution of relevant and rapidly changing clinical questions. To date, the methods to carry out this type of synthesis are not completely defined, and great variability is observed in the approaches used by different groups of authors. Objective: To identify, evaluate and summarise the current methods used for living evidence synthesis Methods: We will conduct a methodological study based on a systematic literature search to identify any type of evidence synthesis such as systematic reviews, network metanalyses and overviews that used “living evidence synthesis” as part of their methods. The search will be conducted in Medline (via PubMed) and Epistemonikos databases. Additionally, we will search websites of the organisations publishing any living evidence synthesis retrieved in the two databases, in order to identify unpublished subsequent reports. Two reviewers will independently assess each article against the selection criteria, extract data on methods and procedures, and assess the methodological quality of each publication. Data will be analysed descriptively.


Introduction
The exponential proliferation of scientific studies and their dispersion through a multitude of scientific journals, some of them of limited access for readers, poses a significant challenge to health care professionals, given their limited time to keep permanently updated in their respective disciplines.Therefore, systematic reviews (SR) and other derived evidence synthesis products (e.g., overwies of reviews, network meta-analysis) are valuable informational tools that try to bring scientific evidence closer to the end-user, facilitating its interpretation and application 1 .However, SRs often do not have the expected impact [2][3][4][5] ; among various reasons that could be mentioned, the rapid outdating of their conclusions is the one that most limits its use and potential impact 6 , making the enormous efforts of developing them largely sterile.
There are areas of high scientific productivity and controversy in which using outdated information poses a major challenge to clinical practice, and solutions are urgently needed.The methodological approach known as "living evidence" (LE) has emerged as a tool with the potential to address this need.It has been applied mainly to SRs, but currently, it is not uncommon to find other types of evidence synthesis such as overviews and network meta-analysis presented as "living".
A "living systematic review" is a continually updated SR, which incorporates new relevant evidence as it becomes available.In practice, this means continual surveillance for new research evidence through on going or frequent searches, and the inclusion of new information into the review in a timely manner so that the findings and conclusions of the SR remain current 7 .When the LE approach is applied to the resolution of relevant and rapidly changing clinical questions, it ensures a rapid update of evidence synthesis that informs on the effects of controversial health interventions, and/or where there are uncertainties 8 .
The potential for LE to reduce the time and cost of updating evidence-based products such as clinical practice guidelines, health technology evaluations and evidence summaries aimed to inform health decisions (e.g. policy briefs) and making it available to inform practice is enormous; nevertheless, it is necessary to weigh the potential benefits with the overload of time and resources that living synthesis entails.
The foundations of the LE model for SRs were initially stated in 2014 7 and have been under development over the past few years [8][9][10][11] .These have been used as the basis for the development and application of new strategies to keep the evidence supporting different products up-to-date 12,13 .It has been of particular interest to synthesise and disseminate the exponential research output published during the COVID-19 pandemic 14 .Being a highly attractive methodology to be applied for many questions in health, but equally recent, little is known about the methods followed by the authors, their validity and the efficiency to achieve an adequate integration of emerging studies in the existing evidence synthesis.Despite some advances in this regard 15 , the methods and quality of living systematic reviews and other types of living evidence synthesis have not been empirically evaluated.There is also no clear route regarding the dissemination process, publication or editorial processes that the periodic updates of the living evidence synthesis follow to reach readers and potential users.

Amendments from Version 1
In this new version, we have made the following changes to the manuscript according to the peer reviewer's comments: 1. We recognized there is a range of viewpoints about research designs and differences in the taxonomy for study designs, particularly in the methodological research topic as is the topic of this project.Therefore we have changed the description of our study design from "Methodological review" to "Methodological study" and we have introduced changes accordingly in the methods section.Nevertheless, our research objectives remain the same as the initial protocol (first version) as well as our research products (the kind of knowledge to be generated).

2.
We have reviewed the variables to be collected from each study to be reviewed, and we added some new variables to capture specific information from baseline reports and from updated reports.
3. We have also included two new references of interest, that support the inclusion of some of the new variables.These references are the following:

Any further responses from the reviewers can be found at the end of the article
With this study, we seek to identify, evaluate, and summarise the methods and procedures that different groups are currently using to generate and keep the evidence synthesis "living", and identify those aspects associated with the reliability and maintenance of the methodological quality of that synthesis through its multiple updates.We also aim to identify the different actions carrying out by living evidence synthesis authors to alert users of the evidence changes or periodic updates of the evidence synthesis, particularly when they have implications on clinical decisions and health policies.In this regard, we will explore editorial processes for new publications, informal update alerts, use of web-based repositories, among other actions presented by authors.Finally, we aim to provide suggestions for improving the development and report of living evidence synthesis based on a previous work done by our research team, in which we have reviewed all the methodological and guidance papers published up to July 2021 describing the "living evidence methodology" [7][8][9][10][11] , in order to identify the relevant methodological items that a living evidence synthesis should include.
This study is being conducted as part of a larger knowledge transfer project entitled "Living Evidence to Inform Health Decisions", funded by the European Union's Horizon 2020 research and innovation Marie Skłodowska-Curie Action (grant agreement No. 894990).Results of this assessment will contribute to the development of a living evidence implementation framework for health system organizations to use and incorporate LE methodology in the development of knowledge transfer products.

Study design
This is a cross sectional methodological study in which we will collect data from all living evidence synthesis published reports (articles) identified in the current available literature.

Study population
Our population of interest is any type of evidence synthesis including systematic reviews, meta-analysis, rapid review, network metanalysis, and overviews that have reported to use "living evidence" (LE) methodology (i.e.living evidence synthesis reports) to answer any clinical questions regardless the LE methodology used.
Because our aim is to review the methods used by authors to produce and maintain living evidence synthesis, we will include all "living evidence synthesis" identified, regardless of the type of condition, the participants or populations, the interventions or the exposures, and the alternatives against which the interventions/exposures are being compared.
Each report meeting the previously mentioned characteristics and the subsequent publications of the same review will be included and treated as a single study (i.e.review /evidence synthesis), considering the initial publication as the baseline one and the subsequent publications as its updates.Nevertheless all review reports will be assessed independently in order to collect the data of interest (see data collection).

Identification of living evidence synthesis reports
The identification of the evidence synthesis of interest will be carried out through systematic and exhaustive searches in the biomedical databases 16 .With this aim we will use Epistemonikos database as main source; this is a comprehensive database of systematic reviews and other types of evidence synthesis reports, maintained by screening multiple information sources to identify SRs and their included primary studies, including the Cochrane Database of Systematic Reviews, Pubmed/ MEDLINE, EMBASE, CINAHL, PsycINFO, LILACS, DARE, HTA Database, Campbell database, JBI Database of Systematic Reviews and Implementation Reports and EPPI-Centre Evidence Library 17 .This database has been validated, showing its completeness in collecting the published SRs and other type of evidence syntheses 18 .The searches will cover from the inception date.No publication status or language restriction will be applied to the searches in Epistemonikos.The boolean literature search strategy that will be used is presented in Table 1.
We will update the searches in Epistemonikos prior to the publication of this review, looking for updates to the already identified living evidence synthesis.
Additional searches.In order to identify new articles or update reports that might have been missed in the electronic searches, we will manually search websites of the organizations reporting the living evidence synthesis identified in electronic searches and email the contact authors of all the included reviews to ask for publications of their updates, particularly when there is no a published update identified within a year from the last publication.

Sampling and selection
Results from these searches will be automatically included in the Living Overview of Evidence (L.OVE) platform 19 of the Epistemonikos Foundation.This platform has been validated as a repository for the COVID-19 showing to be a highly comprehensive source of evidence 20 , where the duplicate references will be identified and eliminated by an automated process.A special algorithm comparing unique identifiers (database ID, DOI) and citation details (i.e., author names, journal, year OR evidenc* OR "systematic-review" OR "meta-analysis" OR "meta-analysis" OR "meta-analyses" OR metaanalys* OR "rapid-review" OR "rapid-evidence" OR overview* OR scoping*)) of publication, volume, number, pages, article title and article abstract) will be implemented to ensure updates of the same review will not be identified as duplicates of the original publication.
Two researchers (AA and JB) will independently screen the search results based on the title and abstract and confirm eligibility according to the selection criteria.We will retrieve the full-text article of the references that meet the eligibility criteria or require further analysis, to decide on their inclusion.Disagreements will be solved by reaching a consensus between reviewers.
As it was previously mentioned, each of the included articles and subsequent publications of the same review, arising from the "living evidence" updating process, will be linked and included as a single study (i.e.review /evidence synthesis).As part of this process we will identify the original publication, the subsequent publications and search manually for unpublished updates.
In the case that the original publication was not identified as part of our search (e.g., it was not initially published as "living" evidence synthesis or to be continuously updated), we will run necessary searches to identify whether it is, using related references, authors names, or title words; if necessary, we will perform a manual search and contact the review authors.

Data collection
Five researchers trained in advance will extract data from all the included review reports; each report will be assessed by two professionals who will extract data independently using a specifically designed standardized form to collect the information of interest.This form has been developed on the basis of previous work done by our research team, that reviewed all methodological and guidance papers published up to July 2021 describing a "living evidence" methodology [7][8][9][10][11][12][13]20,21 in order to identify the relevant methodological items to be collected.
Based on this review, we developed a preliminary list of variables to be collected as part of this study that were later validated by the research group experts in the field.
The final list of variables to be collected (see Table 2) includes variables from each review report and other variables specifically from the baseline report and from its updates.There are other set of variables (see Table 3) to be collected from the same review (e.g. the baseline publication of the review and its subsequent updates) to allow for the evaluation of other "living evidence process" related aspects .The form developed to collect all these data was piloted in a sample of five living evidence synthesis reports.
To be able of identifying the methodological quality of the baseline reviews and any variations through its multiple updates, as part of the data collection we will assess the methodological quality of all included evidence synthesis reports using the appropriate instruments (i.e.AMSTAR II checklist for systematic reviews 22 and NCCMT Quality Assessment Tool, for other review articles 23 ).Two independent authors will complete the assessment on the original publication (baseline review) and subsequent updated reports or publications.Disagreements will be solved by consensus.We will pay particular attention to changes in the methodological quality through publications of the same review.Results of this assessment will be presented in descriptive tables.

Data analysis
To present the data's basic features in the evidence synthesis reports, we will perform a descriptive analysis (using frequencies, numbers, and proportions).We will present the results in tables, both for each review report and for the group of publications linked to the same review/evidence synthesis.
Qualitative data (e.g., description of the living methodology) will be presented in tables.
We will check the methodological issues reported by the authors against those in the list of variables that we previously defined based on the review of methodological articles.
The aspects related to the updating of the reports, scientific publications or other forms used by authors to disseminate the review updates, will be collected and presented as qualitative information in summary tables, since this is one of the methodological aspects still to be defined in living evidence processes.
If feasible, we will report the results in subgroups by the type of synthesis (LSR, Living overview, or LNMA).
PROSPERO registration.This protocol is registered under the accession number CRD42021248963

Study ID
A study identification number will be assigned to the set of publications of the same review, authoring by the same group or research team

Publication year
Year of the current article

DOI of this article
Journal/Editorial Group/Publisher Journal where the article has been published

Journal Impact Factor
The impact factor of journal where the systematic living evidence synthesis are being published

Funding
Description of funding supporting the review /evidence synthesis and the evidence surveillance as part of the living evidence approach

Source of identification
To define whether this article/report has been identified from electronic searches (databases), registers or other sources (i.e.organizations website or repositories among others).

Type of evidence synthesis report
To define whether this article/report is a systematic review, an overview, network metanalysis or other type of evidence synthesis.

Type of Publication
To define whether or not it is the original (first) publication of the review/ evidence synthesis or a publication of one of its updates

Is the question to be answered clearly stated?
To define whether or not the review authors explicitly state the question and the subgroup of interest

Are the end points or outcomes of interest clearly defined (as part of the protocol or in the text of the manuscript)?
To define whether or not the review authors explicitly state the outcomes of interest, including characteristics of its measure

Use of a "living evidence" approach
To define whether or not the review authors explicitly state they are applying or will apply a living evidence approach to solve the question

Definition used for the living methodology
To define whether or not the review authors explicitly present a definition to describe the living methodology.

Reference(s) that supports the methodological approach used
[author of the reference/other(define)/NR]

Search strategy reported
To define whether or not the review authors present a detailed search strategy

Evidence sources searched (name of data bases used)
To define whether or not the review authors present the data bases searched/to be searched

Grey literature search performed
To define whether or not authors carried out grey literature search

Searching for ongoing trials in registers
To define whether or not authors carried out searches in trials registers

Who is performing the searches?
To define who is in charge of running searches [authors/specialized technician/ external[contracted]/ Other/NR]

Are the selection criteria clearly defined?
To define whether or not the review authors explicitly state the selection criteria for the studies, including the type of study design and publication type

Included studies reporting on the outcomes of interest
To define whether or not the review authors identified studies reporting on the outcomes of interest* *Information of the outcomes with no evidence will be collected

Evidence synthesis for each outcome
To define whether the review authors conducted a metanalysis or a narrative evidence synthesis for summarising the evidence found for the outcomes of interest

Rational for setting up/or preforming a living evidence synthesis
To define whether or not the review authors clearly state the reasons to conduct a living evidence synthesis (setting up or continue the living process)* *information on the appropriateness of the statements will be collected according to predefined criteria -Information of how often the original question is going to be revisited?

Resources to maintain the living evidence processes
To tefine whether or not the review authors have defined in advance any of the following aspects as part of the evidence monitoring plan: number of authors involved in the living evidence synthesis; or whether there is no information provided -Is there any assigned person/role in charge of creation and maintenance the search strategy?
-Is there any assigned person/role in charge of performing the evidence searches? -

Updates communication and publication
Information on Updates including any of the following: -Dissemination of content through website, repositories or similar -Alerting readers (describe how is communicated to the readers that a new update is available) -

Frequency of communicating updates to readers
Editorial and peer review Define whether or not the review authors provided information on the editorial and peer review process for the baseline report and for the updates.Including: -Is there a Citation/DOI is provided for each publication (i.e.Baseline and updates) -Is there a publication agreement with any journal available -Other

Revisit parameters through the living evidence process
Define whether or not the review authors provided information regarding the parameter revisited during the evidence surveillance as part of the LE process, such as: Ethical considerations.As researchers will not access information that could lead to the identification of an individual participant, obtaining ethical approval was waived.

Data sharing.
All data related to the project will be available.Epistemonikos Foundation will grant access to data.

Conclusions
The way in which authors are currently incorporating the living evidence approach into systematic reviews and other synthesis products is still unknown.This project will permit to identify, evaluate and summarise the methods and procedures that different groups are using to generate and keep the evidence "living".
One of the biggest challenges of setting up a "living evidence" process is to rapidly transfer the results of the updates to end-users, such as clinicians and other decisions makers.
There are some proposals done in this regard so far but not a clear route to reach this goal.This project will identify the current practice and methods used by authors to disseminate the constant updates of the evidence synthesis as well as those aspects that will permit the living evidence authors to maintain informed their final users through its multiple updates.
This study will provide information on the compliance of the current authors of the living evidence synthesis with the methodological standards proposed so far, which will serve to improve the living evidence synthesis reports, increase their transparency, as well as guide their potential evaluation.I believe that this study is not a methodological review.My understanding of a methodological review is a review of studies that assess the effects of a method used in research (sometimes compared to another method), or association of a method with some outcome, and these effects or associations are pooled together across studies if possible (see Cochrane methodological reviews).To me, instead, this paper is a survey of what methods are currently being used.This may explain why on Page 4, the reference to the 'intervention of interest' does not seem to apply.
If you change this to a survey, the PRISMA checklist/statement would not be used when reporting this paper.
Since this study is about the methods for living reviews, I think it should be more broad.The information that will be extracted seems to focus mostly on the search methods, but there are other methods that are or could be affected when conducting a living review.I have mentioned a few below but I think the authors should explore the different methods that are currently being discussed in the literature and expand the data they will collect.
For example, the authors could consider adding how the results of the search and screening are reported as there seems to be some research about different methods of reporting being used.Alternatively, the authors may decide not to gather this data and instead refer to the paper by Kahale 2021.
There is also some literature about the statistical methods that could be used in living reviews (see Simmonds 2017).The Simmonds paper identifies many important methods that could be used.The authors have not indicated that they will collect data about different methods of statistical analyses and interpretation, but could.In addition, there is some question about the criteria to decide when re-analysis and a subsequent publication should occur when doing a living review (e.g., it could be when results or certainty of evidence change, or simply when there is new evidence regardless of whether the effect or certainty changes).Data about if and how the authors of the reviews report these methodological decisions would likely be important to collect.
It's not clear to me why the methodological quality of the reviews are being assessed.How will this information be used?In addition, do the current tools actually address issues relevant to living reviews?Will these tools provide quality information relevant to a living review in particular?
A minor comment: I understand that different types of reviews will be included, but the paper may be easier to follow if the living 'reviews' are not referred to as 'studies'.Is the rationale for, and objectives of, the study clearly described?No

Are sufficient details of the methods provided to allow replication by others? No
Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Systematic reviews and guideline development I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
proposal.F1000Research.2021; 10.Based on the previous work done by other authors, we defined the variables to be collected and exceed those related only to the search methods.This list already includes information about how results of the search and screening are reported, as we presented in tables 2 and 3. Nevertheless, taking into account your comments we have reviewed the list of variables and provided a better explanation of its operational definition (see tables 2 and 3).

Comment 3 Data related statistical methods that could be used in living reviews
There is also some literature about the statistical methods that could be used in living reviews (see Simmonds 2017).The Simmonds paper identifies many important methods that could be used.The authors have not indicated that they will collect data about different methods of statistical analyses and interpretation, but could.In addition, there is some question about the criteria to decide when re-analysis and a subsequent publication should occur when doing a living review (e.g., it could be when results or certainty of evidence change, or simply when there is new evidence regardless of whether the effect or certainty changes).Data about if and how the authors of the reviews report these methodological decisions would likely be important to collect.
Reply to comment 3 Even though the publication by Simmonds 2017 was included in our review of prior work done, we agree that still some variables related to statistical methods are needed.We have added new variables to collect specific information related to the statistical methods inspired in Simmonds 2017 work.See: "Data to be collected from the baseline report" in table 2-Planning for evidence monitoring /surveillance, and "Data to be collected from the updates" in table 2-Information about the integration of new evidence (NEI)) Comment 4 The methodological quality of the reviews are being assessed It's not clear to me why the methodological qualities of the reviews are being assessed.How will this information be used?In addition, do the current tools actually address issues relevant to living reviews?Will these tools provide quality information relevant to a living review in particular?
Reply to comment 4 A living systematic review should derive from a reliable systematic review, the "baseline review".Our aim in conducting the methodological quality assessment of both, the baseline report and their subsequent updates, is to assess if the methodological quality is maintained, improved, or decreased through its multiple updates.We agree with you, the AMSTAR II checklist, as well as the other instruments we have proposed to use, are not addressing the relevant issues of living evidence synthesis or reviews.Even though we will use them for this assessment because they do address important quality issues that any systematic evidence synthesis should meet.We have improved the explanation of this procedure in the text: Lines 217-225.To be able of identifying the methodological quality of the baseline reviews and any variations through its multiple updates, as part of the data collection we will assess the methodological quality of all included evidence synthesis reports using the appropriate instruments (i.e.AMSTAR II checklist for systematic reviews [21] and NCCMT Quality Assessment Tool-for other review articles [22]).Two independent authors will complete the assessment on the original publication (baseline review) and subsequent updated reports or publications.Disagreements will solve by consensus.We will pay particular attention to changes in the

Search strategies
The draft reads as follows "An additional search using a highly sensitive search strategy will be performed on PubMed/MEDLINE, in order to compare results and be able to identify additional evidence synthesis reports not obtained from searches in Epistemonikos".
This is intended to validate Epistemonikos as a database for searches on living reviews, including systematic reviews, network meta-analyses and living overviews.Since the main database for searching is Epistemonikos, it would be interesting to elaborate a little more about its quality and previous validation.Many readers may not be familiar to this database.

Assessment of the methodological quality
Authors will evaluate the quality of all included studies (e.g.SRs, overviews, or network metaanalysis) using the AMSTAR II checklist and other appropriate instruments according to the type of study (e.g. the NCCMT Quality Assessment Tool-Review articles).If not possible, this should be stated in the protocol as a limitation of the LE systematic review.

Reply to comment 3
The goal of our analysis is to evaluate the methodological quality of the included SRs, from both the baseline one and their subsequent publications, but not the validity of the conclusions.We will then assess quality over time through the subsequent updates.
Comment 4. Additional information on Table 2.One of the variables included in Table 2 is the "Journal/Editorial Group/Publisher".I would also include the "impact factor" in order to showcase the "visibility" of living evidence.Likewise, the "Journal/Editorial Group/Publisher" and "impact factor" of the last update could also be included in Table 3.

Reply to comment 4
We have added the additional information regarding the "impact factor" on journal publication Living evidence products in table 2. We believe this variable will help to summarize the scope of the journal and add value to our descriptive analysis.
Competing Interests: No competing interests were disclosed.

Data to be collected from the baseline report Variable Description/operational definition
Planning for evidence monitoring /surveillance To define whether or not the review authors defined in advance (in protocol or in the article text) any of the following aspects as part of evidence surveillance or monitoring plan:-Type of studies to be identified as part of evidence surveillance -Frequency of electronic searches during evidence monitoring -Frequency of electronic searches in registers during evidence monitoring -Frequency of screening and selection of the new evidence identified by the searches during evidence monitoring -Type of publications included as part of the during evidence monitoring -Anticipate duration of the evidence monitoring -The statistical methods used for updating the metanalyses when new evidence becomes available-integrating new data-?-Statistical considerations regarding repeated analysis of accumulating primary trial data -Considerations and rules to stop the evidence monitoring for the given question of interest (e.g.improvement of evidence quality for main outcomes; question no longer relevant)

3 .
there any assigned person/role in charge of screening search results during evidence monitoring -Is there any assigned person/role in charge of performed the Risk of Bias (RoB) assessment task?-Is there any assigned person/role in charge of performed extracting the new study data?-Is there any description of technological enablers supporting searches?*-Is there any description of technological enablers supporting evidence screening?** Description of enablers used for searches and/or for screening will be collected Data to be collected from the updates publication or reports Variable Description/operational definition Changes in methodology from the baseline report Define whether or not the review authors provided information on changes from the baseline or the last report in any of the following: -Evidence sources (i.e.new databases or registers searched) -Selection criteria used for eligible studies -Type of studies to be identified as part of evidence surveillance -Frequency of electronic searches during evidence monitoring -Frequency of electronic searches in registers during evidence monitoring -Frequency of screening and selection of the new evidence identified by the searches during evidence monitoring -Type of publications included as part of the during evidence monitoringTable Variables and other measures across publications of the same review.Variable Description Number of publications and/ or update reports identified Number of publications (i.e.baseline report and update) Scientif publications identified Number of peer reviewed articles published from the first publication Other type of update reports identified Number of updates identified from other sources than scientific publications (e.g.Website)

-
Changes in the quality of the report (AMSTAR II assessment)Define whether or not the review reports quality changed from the updates though its updates.There are no changes identified in quality of the reportsData to be collected from the updates publication or reportsVariableDescription/operational definitionInformation to the continuous update of the PRISMA flowchartDefine whether or not the review authors provided information on number of evidence surveillance results from the last update, considering the following:-Informationof the number of studies identified by searches since last report?-Independent information of the number of peer reviewed articles and preprints identified since last report?-Information of the number of ongoing studies identified in registers since last report?-Information of the number of studies screened since last report?-Number of eligible studies since last report -Number of new studies included in the analysis Information about the integration of new evidence (NEI) Define whether or not the review authors provided information on the integration of new evidence, including: -Description of the new study(ies) -RoB assessment of new studies -Outcomes reporting (information of outcomes informed by new evidence) -Statistical methods and considerations taking into account for updating metanalyses integrating new studies data

©
2021 Santesso N.This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Nancy Santesso 1 Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada 2 Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton,CanadaMethods for living reviews are evolving and a study to determine the current methods being used by review authors will be important as we move forward, in particular if compared to the variety of methods being proposed today.Below I have included a few suggestions that may improve the paper.

the rationale for, and objectives of, the study clearly described? Yes Is the study design appropriate for the research question? Yes Are sufficient details of the methods provided to allow replication by others? Yes Are the datasets clearly presented in a useable and accessible format?
Reference 21 in the draft is that of a previous version of the AMSTAR tool (published in 2007).Probably it would be better to swap the current reference 21 for the following: This is really poor approach to the RoB assessment of individual studies included in the reviews which is a key aspect for the reliability of published results.At present, the Cochrane collaboration is piloting its RoB2 tool.Ideally, RoB in the individual studies of the reviews should be re-assessed with the help of better tools like RoB2.If not possible, this should be stated in the protocol as a limitation of the LE systematic review.
○ lack of blinding of patients and assessors when assessing outcomes (unnecessary for objective outcomes such as all cause mortality) ○ For Yes, must also have assessed RoB from: allocation sequence that was not truly random, and ○ selection of the reported result from among multiple measurements or analyses of a specified outcome ○ Likewise, the NCCMT Quality Assessment Tool-Review provides a poor approach to the RoB assessment of individual studies in the review (described on Question 6 of the tool).

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
We have decided not to carried out additional searches because Epistemonikos has been already validated for identification of any type of evidence syntheses.We have changed the paragraph and added a sentence with specific reference (reference 17).Lines 161-168: With this aim we will use Epistemonikos database as the main source; this is a comprehensive database of systematic reviews and other types of evidence synthesis reports, maintained by screening multiple information sources to identify SRs and their included primary studies, including the Cochrane Database of Systematic Reviews, Pubmed/MEDLINE, EMBASE, CINAHL, PsycINFO, LILACS, DARE, HTA Database, Campbell database, JBI Database of Systematic Reviews and Implementation Reports, EPPI-Centre Evidence Library [16].This database has been validated, showing its completeness in collecting the published SRs and other type of evidence syntheses [17]*.*Reference 17: Rada, G., Pérez, D., Araya-Quintanilla, F. et al.Epistemonikos: a comprehensive database of systematic reviews for health decision-making.BMC Med Res Methodol 20, 286 (2020).https://doi.org/10.1186/s12874-020-01157-xComment 2 Assessment of the methodological quality Authors will evaluate the quality of all included studies (e.g.SRs, overviews, or network meta-analysis) using the AMSTAR II checklist and other appropriate instruments according to the type of study (e.g. the NCCMT Quality Assessment Tool-Review articles).Reference 21 in the draft is that of a previous version of the AMSTAR tool (published in 2007).Probably it would be better to swap the current reference 21 for the following: (1) Shea B J, Reeves B C, Wells G, Thuku M, Hamel C, Moran J et al.AMSTAR 2: a critical appraisal tool for systematic reviews that include randomized or non-randomised studies of healthcare interventions, or both BMJ 2017; 358 :j4008 doi:10.1136/bmj.j4008Reply to comment 2 We have updated the reference in the protocol according to your suggestion: Reference 22. Shea B J, Reeves B C, Wells G, Thuku M, Hamel C, Moran J et al.AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or nonrandomised studies of healthcare interventions, or both BMJ 2017; 358 :j4008 doi:10.1136/bmj.j4008Comment 3 Assessment of the methodological quality An important limitation of the AMSTAR II tool is that the quality of individual studies in the meta-analysis is poorly evaluated.Domain 9 reads as follows: "Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?"There are two possible answers to this question, namely: For Partial Yes, must have assessed RoB from unconcealed allocation, and Likewise, the NCCMT Quality Assessment Tool-Review provides a poor approach to the RoB assessment of individual studies in the review (described on Question 6 of the tool).Furthermore, in both AMSTAR II and NCCMT tools, the score in Domain 9 and Q6, respectively, is obtained simply if information on allocation, blinding, etc., has been addressed, regardless the resulting quality of individual studies included.This is really poor approach to the RoB assessment of individual studies included in the reviews which is a key aspect for the reliability of published results.At present, the Cochrane collaboration is piloting its RoB2 tool.Ideally, RoB in the individual studies of the reviews should be re-assessed with the help of better tools like RoB2.
○ lack of blinding of patients and assessors when assessing outcomes (unnecessary for objective outcomes such as all-cause mortality)○ For Yes, must also have assessed RoB from: allocation sequence that was not truly random, and ○ selection of the reported result from among multiple measurements or analyses of a specified outcome ○