Every ROSE has its thorns

In their study, Haddaway et al. [1] present new reporting guidelines, RepOrting standards for Systematic Evidence Syntheses (ROSES) with a thoughtful breakdown of the issues when applying Preferred Reporting Items for Systematic review and Meta-Analyses (PRISMA) in the field of conservation and environmental management. We believe the expanding use of systematic reviews and systematic maps in non-medical fields illustrates the growing importance of evidence synthesis and recognizes the importance of transparency and reporting standards in research. However, we have some concerns regarding the methodology as well as the communication and dissemination plan for the new reporting guidelines.

The development of the ROSES reporting guidelines was limited to semi-structured questionnaires or in-depth interviews, which differs from the recommended guidance for developers of health research reporting guidelines that suggests the following steps: initial steps, pre-meeting activities, the face-to-face consensus meeting itself, post meeting activities and post publication activities [2]. Despite this different approach, the authors provide a detailed point-by-point breakdown of the problems applying the reporting guideline PRISMA to their field (Table 1), and highlight the key differences between the two reporting guidelines (Table 2) [1]. Presentation of the thorough rationale for changes for new reporting guidelines or extensions is not common and other new guidelines can use this structure. However, this breakdown does not make it clear why they chose to develop a separate but parallel tool to PRISMA.

Given that PRISMA is already known and widely used, ROSES could have been better framed as an extension (akin to PRISMA for network meta-analyses [3]). We recognize that the intention to include more methodological guidance than PRISMA is valuable as it recognizes the need for additional education. However, this intention is not reason enough for it to be distinctly separate from PRISMA.

While the authors provide detailed rationale for their individual changes, the relationship of the ROSES guidance within the existing domain of reporting guidelines is not stated. Although they mention that ROSES should be used in conjunction with the Collaboration for Environmental Evidence (CEE) guidelines for methodological guidance, the link to PRISMA is vague. Using multiple guidelines simultaneously can be a difficult task for authors and practicality and ease of use concerns should be more clearly addressed. We praise the authors for their innovative approach in making their guidelines user-friendly in the form of editable and downloadable formats.

Uptake of reporting guidelines is generally low across many fields, therefore we believe that further elaboration on plans for dissemination and communication is essential to the success of this initiative [4]. For example, while the creation of the ROSES website is valuable and in accordance with Moher et al.’s guidance for reporting guideline developers, an appropriate dissemination plan is needed to ensure widespread uptake [2]. Moreover, additional future plans to gather user experiences and feedback, including evaluating its impact on completeness of reporting, will be of great importance to further advance the checklist and flow diagram.