Participants’ perspectives and preferences on clinical trial result dissemination: The TRUST Thyroid Trial experience

Background: While there is an increasing consensus that clinical trial results should be shared with trial participants, there is a lack of evidence on the most appropriate methods. The aim of this Study Within A Trial (SWAT) is to use a patient and public involvement (PPI) approach to identify, develop and evaluate a patient-based approach to receiving trial results for participants in the Thyroid Hormone Replacement for Subclinical Hypo-Thyroidism Trial (TRUST), a trial of thyroxine versus placebo in people aged 65 years and older. Methods: Mixed methods study with three consecutive phases. Phase 1 iteratively developed a patient-based approach using semi-structured focus groups and a consensus-orientated-decision model, a PPI group to refine the method and adult literacy review for plain English assessment. Phase 2 was a single-blind parallel group trial. Irish TRUST participants were randomised to the intervention (patient-based approach) and control group (standard approach developed by lead study site). Phase 3 used a patient understanding questionnaire to compare patient understanding of results between the two groups. Results: Participants want to receive results of clinical trials, with qualitative findings indicating three key themes including ‘acknowledgement of individual contribution’, ‘contributing for a collective benefit’ and ‘receiving accessible and easy to understand results’. Building on these findings, the patient-based approachwas developed. TRUST participants (n=101) were randomised to the intervention (n=51) or control group (n=50). The questionnaire response rate was 74% for the intervention group and 62% for the control group. There were no differences in patient understanding between the two approaches. Conclusions: We have demonstrated that it is feasible to involve trial participants in the development of result dissemination materials. Although, in this study PPI did not influence patients’ understanding of results, it documents the process of conducting PPI within the clinical trial setting.

sufficient regardless of any pragmatic impact. However, PPI costs time and money, therefore pragmatic claims need scrutiny 8 . More substantive evidence is needed to evaluate the potential impact of PPI on the conduct and outcomes of research 5,9 . In 2001, the need to establish if PPI leads to actual, rather than merely perceived benefits for research processes and output was identified. Over fifteen years later, this need remains.
In clinical research, the results of clinical trials have not traditionally been shared with clinical trial participants. A recent survey carried out on a large registry of health research participants, found that while 95.6% of respondents said researchers should always or sometimes offer the results to participants, only 33% of respondents actually received the results of studies in which they had participated 10 . An upcoming European Union Clinical Trial Regulation requires sponsors to provide summary results of clinical trials in a format understandable to laypersons, including participants 11 . However, there is a lack of evidence on the most appropriate methods of sharing results with participants. Uncertainty persists around what information should be shared, how results should be shared and who should be responsible for sharing the results. Since the findings of clinical research often exist in a complex context of scientific exchange and debate, it is important that the information shared is accessible and relevant to participants 12 . The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials.
The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols 13 , the Consolodated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials 14 and the AllTRials iniative calls for all past and present triasl to be registered and their full methods and summary results reported 15 . Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community 13 .
The Thyroid Hormone Replacement for Subclinical Hypothyroidism Trial (TRUST) was a multi-centre, double blind, placebo controlled, phase III clinical trial testing the efficacy of thyroxine replacement in subclinical hypothyroidism in older community dwelling adults 16 . The results of the TRUST trial were published in the New England Journal of Medicine on 3 rd of April, 2017 16 . This Study Within A Trial (SWAT) was conducted at the Irish TRUST trial site prior to and after publication of results. The aim of this SWAT was to investigate methods of disseminating trial findings to participants by using a PPI approach to identify, develop and evaluate a patient-based approach of receiving trial results.

Study design
This was a sequential mixed methods study with three phases. In this study, methods were combined for complementarity, where

Amendments from Version 1
This improved version contains some minor revisions as suggested by peer-reviewers.
Throughout the manuscript, the following changes have been made: • "patient-preferred" has been changed to "patient-based".
• "Standard method" has been changed to "standard approach".
Within the Abstract, the aim of the study has been re-worded to clarify that all TRUST participants were aged 65 and over.
Within the Introduction section, additional background information has been provided on the need to evaluate the impact of PPI. This serves as a rationale for doing the study. We have also introduced the recent movement towards transparency in trials including references to the SPIRIT, CONSORT and AllTrials initiatives.
Within the Methods section, additional details have been provided on the PPI group and how PPI partners were identified and recruited. Further information has also been provided on the Consensus Oriented Decision Making (CODM) model and how the model was specifically used in this study. We have also provided a clear distinction between adult literacy and health literacy.

Introduction
Patient and public involvement (PPI) is increasingly recognised as an essential component of clinical research. In the UK, the national advisory group supporting active public involvement in health services, public health and social care research (INVOLVE) defines PPI as 'research being carried out 'with' or 'by' members of the public rather than 'to' 'about' or 'for' them' 1 .
In clinical trials, PPI has been defined as experimenting with participants instead of experimenting on participants 2 . PPI may occur at any stage during the research process from priority setting and drafting study protocols right through to conducting the study, interpreting the end results and communicating and disseminating research findings 3,4 . Research funders increasingly expect that PPI is prioritised and resourced within studies. This increasing expectation has heightened the risk of researchers carrying out 'tick-box' PPI rather than 'meaningful' involvement 5 . There are many moral and ethical arguments being made for PPI. Many believe that as citizens and taxpayers, members of the public have a right to influence research that is being funded by public money 6 . PPI researchers are also making pragmatic arguments for PPI and providing anecdotal accounts about how PPI can make research more relevant, accessible and acceptable to participants 7 . The ethical arguments are often seen as each method addressed a different aspect of the study aim 17 . The first phase used a qualitative approach to identify and develop a patient-based approach to disseminating the results, the second phase used a SWAT intervention to compare the dissemination approaches and the third phase used a quantitative patient understanding questionnaire to evaluate the patient-based approach. The full study protocol has been published elsewhere 18 , but a summary follows here.

Setting
The study sites for the TRUST trial were the University of Glasgow, Scotland (lead site); Leiden Academy on Vitality and Ageing, The Netherlands; Leiden University Medical Centre, The Netherlands; University of Berne, Switzerland; and University College Cork, Ireland. A total of 738 participants with subclinical hypothyroidism were recruited to the trial over a three-and-ahalf year period from 2013-2017 16 . The trial completed recruitment in November 2016 and the results were published in April 2017 16 .
This SWAT was conducted at the Irish TRUST site. The hub centre for the Irish TRUST site was located at the Mercy University Hospital, Cork where 38 participants were recruited. A further 77 participants were recruited from five satellite sites.

Population
As this SWAT was embedded in an ongoing clinical trial the study sample was determined by the TRUST Thyroid trial. There were 115 TRUST participants recruited in the Irish site, 11 of these participants withdrew over the course of the trial. Our study sample included all remaining TRUST participants (n=104).

Phase One: Identification and development of patient-based approach (qualitative and PPI phase)
The first phase of the study used a qualitative approach to iteratively identify and develop a patient-based approach to disseminate the results of TRUST trial. This was done in three separate stages: qualitative focus groups, a PPI group and an adult literacy review.

Focus groups
Three semi-structured focus groups were conducted with four to eight TRUST trial participants per group. All Cork-based patients (n = 38) were contacted via letter and invited to participate. A €20 shopping voucher was given to all participants to cover travel expenses. Each session was led by trained qualitative researchers (WHS, ER, CH). A topic guide was used to guide the focus groups. The topic guide was reviewed and refined by all members of the SWAT research team (see Supplementary File 1: Focus group topic guide).
The Consensus-Oriented-Decision-Making (CODM) model was used to guide the group to reach a consensus 19 . The CODM model is accepted as a flexible model for reaching decisions 19 .
In this study some of the steps were initiated by the focus group facilitator and others occurred naturally as a follow on from the previous step. Below is an outline of each of the seven steps of the CODM model and how they were used in this study: 1. Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants.
2. Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial.
3. Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used.
4. Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language.
5. Choosing a direction: This step occurred naturally as part of the previous step.
6. Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback. 7. Closure: This step occurred naturally as part of the previous step.
Analysis. Focus group recordings were transcribed verbatim and entered into NVivo Version 11 for data management during thematic analysis. Braun and Clarke guidelines 20 for conducting thematic analysis were followed. Initial focus group transcripts were analysed independently by two researchers (ER and AC). Each transcript was read multiple times (data familiarisation) and initial codes were identified. These codes were then used to identify emerging themes. Both researchers discussed emerging themes and conducted further refinement. The refined themes were then discussed and agreed upon with other members of the research team (ER, CH, AC, KMS). Researchers (ER, CH, AC) then used the focus group findings to develop an initial draft of a patient-based approach for the dissemination of results (see Supplementary File 2: Draft one of patient-based result letter).

PPI group
A PPI group was established to develop and refine the content of the patient-based appproach for the dissemination of results. During the focus groups, three TRUST trial participants volunteered to take part in the PPI group. In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This individual was keen to learn more about research and expressed an interest in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner. Originally, we intended to conduct these sessions in a group format, due to difficulties with PPI partners' schedule commitments, one-to-one sessions were conducted. At the one-one session, a researcher (ER) and the PPI partner discussed the layout, content and language of the initial draft of the result method. Researchers and PPI partners worked together to edit different sections of the document. These discussions were not audio recorded but comprehensive field notes were taken by the researcher (ER). These notes were then collated by the researcher and used to further ensure that the results letter reflected PPI partners' perspectives and preferences.
Adult literacy review While the PPI group had significant input into the format and language used in the patient-based approach, the research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national "Plain English" standards. These standards ensured that the information presented to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented (health literacy) 21 . This review was an iterative process with several drafts exchanged for editing. Although the review was taken as an additional step to the published protocol for the study, the research team felt it was helpful to further ensure that the document was accessible and easy to understand.
At the end of the first phase of the study, a final draft of the patient-based result letter was approved by researchers, PPI group and adult literacy experts (see Supplementary File 3: Final draft of patient-based result letter).

Phase Two: Dissemination of trial results (intervention phase)
The second phase of the study used a SWAT intervention to disseminate the results of the TRUST Thyroid Trial to trial participants. This was done using a prospective, randomised, single blind, parallel trial design. It is important to note that when the term randomisation is used, it refers to the allocation of patients to intervention/control within the SWAT and not the TRUST Thyroid trial. Irish TRUST participants were randomised to intervention or control groups using an online random number generator. The intervention group received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter). Participants were blinded to their intervention group. One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial. As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way.

Phase Three: Evaluation of patient -based approach (quantitative phase)
The third phase of the study used a quantitative patient understanding questionnaire to evaluate the patient-based approach to disseminating trial results. The questionnaire was developed in consultation with experts in the area of subclinical hypothyroidism and scale (questionnaire) development (PK and KMS The questionnaire was sent to all Irish TRUST participants (intervention and control group) one week after they received the results of the trial. A reminder questionnaire was sent to non-responders 3 weeks later.
Analysis. The primary outcome was the difference in levels of patient understanding between the intervention and control groups. This measured the impact of PPI on patient understanding of end of trial results. The psychometric properties and construct validity of the questionnaire were examined with exploratory factor analysis. Principal component analysis (PCA) was conducted on the six LIKERT scale items. Internal consistency of the questionnaire was investigated using Cronbach's alpha coefficient. Completed questionnaires were entered into SPSS software (version 24) and analysed using descriptive and inferential (Chi-square test and Fishers Exact) statistics. The researcher carrying out data input and analysis was blinded to the participants' allocation status.

Costs of conducting PPI
The lead researcher (ER) kept a detailed account of all direct costs associated with conducting PPI for the purpose of this study. These costs included researcher salary, travel and expenses for PPI participants, adult literacy review and printing and postage costs.
This paper has been written in adherence to the Guidance for Reporting Involvement of Patients and Public 2 (GRIPP 2) 23 .
The GRIPP 2 checklist is a tool, developed to improve the reporting of patient and public involvement in research and guide the development of a transparent, consistent and high-quality PPI evidence base. The Good Reporting of a Mixed Methods Study (GRAMMS) framework was also used to inform the reporting of the findings 24 .

Results
Characteristics of the trial participants stratified by participation in the different stages of the study are presented in Table 1.
Phase One: Identification and development of patient-based approach (qualitative and PPI phase) Focus groups Three focus groups were held with 19 out of 38 participants accepting an invitation to join. Participants who attended the focus groups were similar in age, gender, education level to those who did not attend.
Focus group findings indicate that participants want to receive the results of the trial in which they are taking part. Three main themes emerged in relation to participants' perspectives of and preferences for receiving trial results: 'acknowledgement of individual contribution', 'contributing for a collective benefit' and 'receiving accessible and easy to understand results'.

Acknowledgement of individual contribution
Many participants reported feeling they had made an individual contribution to the trial in terms of their time and personal information while attending the trial study visits. As such, participants felt that receiving the results of the trial would provide an acknowledgement of this individual contribution: 'Yes,Imeanit'skindofinstinctive…whenyougointoa [clinicaltrial] Receiving accessible and easy to understand results Participants expressed a clear need to receive the results of the trial in an accessible and easy to understand way. This preference applied to the format, language and content of the patient-based approach. The majority of participants said they would like to receive the results in a letter format posted to them directly from the TRUST trial. Participants felt that this method would be accessible to them as they could read the results 'in text' (FG3 P4) and keep a 'hardcopy' (FG P1). While participants wanted an official statement of the results in a letter format, they also felt it was important to add a personal element to the letter. They suggested this could be done by offering participants a phone number that they could call if they wished to discuss any further issues or concerns with the TRUST study team: 'Could you attach a helpline on to it? If you know, somebody had some kind of serious medical question or that they thought was a bit personal element or whatever. That they'd like to talk to a medical person or whatever. Instead of just talking to your GP, maybe that would add another dimensionofcarearoundtheTRUST' (FG2 P3) Participants agreed that the format, content and language of the results letter needed to be easy to read and understand. All participants wanted the letter to be no longer than 2-3 pages and presented in a question and answer format. Participants believed the content of the results letter should include 'pertinent information' (FG1 P7) relating to the trial itself, the study drug (including side effects) and the results of the trial. They stressed the importance that this information needed to be informed by medical experts and 'from a good authoritative source' (FG2 P2) but it should be presented to them in a language that fits their current context and could be easily understood by those who do not have scientific or medical backgrounds.
'Just in ordinary language that we can understand ourselves, you know that we don't want big and long explanation or that, just that we can pick it up straight away that it's without any huge number of pages. Just the bare, to me anyway,answerstothequestions.' (FG3-P2) It was evident from the focus groups that participants want to receive the results of the trial both to acknowledge their individual contribution to the trial and also help them to feel that they had contributed to a greater good. Participants expressed a clear preference to receive the results in an accessible and easy to understand way. These results were used by the researcher (ER) to develop an initial draft of the results letter (see Supplementary File 2: Draft one patient-based result letter).

PPI group
The initial draft of the results letter was then further iteratively developed by the PPI group. There were four PPI partners in total (three trial participants and one older adult) Each partner toook part in one-to-one session. Each session contained an open discussion between the researcher (ER) and PPI partners on the layout, content and language of the document. Researchers and PPI partners worked together to write, re-write, edit and change different sections of the document.

Health literacy review
This draft was then iteratively reviewed and approved by health literacy experts from the NALA (see Supplementary File 3: Final version patient-based results letter).

Phase Two: Dissemination of trial results (intervention phase)
There were a total of 101 Irish TRUST participants randomised to the SWAT intervention. Trial participants from the PPI group (n = 3) were excluded from randomisation as they reviewed the content of the intervention method prior to the intervention. The intervention group (n=51) received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group (n=50) received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter).

Phase Three: Evaluation of patient-based approach (quantitative phase)
The overall response rate for the patient understanding questionnaire was 68% (69/101). The response rate for the intervention group was 74% (38/51) and the response rate for the control group was 62% (31/50). There were no significant differences in age, gender and education between those who returned the questionnaire and those who did not.
Post hoc power calculations showed that the study was underpowered to detect an effect. Power for each of the patient understanding components ranged from .01 to. 58.  Figure 1 shows patients' actual understanding of the primary aim, side effect and results of the TRUST Thyroid Trial. Almost 82% (n=31) of the intervention group and 65% (n=20) of the control group correctly understood the primary aim of the TRUST trial (p=0.108). Almost 40% (n=15) of the intervention group and 36% (n=9) of the control group correctly understood the associated side effects of the active drug (p=0.734). In total 50% of the intervention group (n=19) and 58% of the control group correctly understood the results of the trial (p=0.504). There were no differences in patient understanding of trial results between the intervention and control groups.
In terms of patient understanding of hypothetical patient case studies, 43% (n=13) of the intervention group gave the correct answer to Vignette A; this was lower than the control group (62.1%, n=18, p=0.15). In total 77% (n=23) of the intervention group gave the correct answer to Vignette B, this was higher than the control group (66%, n=19, p=0.344).

Psychometric testing
An exploratory principal components analysis (PCA) was conducted on the patient understanding questionnaire to determine  its usefulness as a measure of perceived understanding. The Kaiser-Meyer-Olkin (KMO) measure verified the sampling adequacy for the analysis, KMO= .83. Bartlett's test of sphericity indicated that the correlation matrix was significantly different from an identity matrix, X2 (.852) = 283.92, p<.001. An examination of eigenvalues greater than Kaiser's criterion of one, suggested the extraction of one factor; this was supported by inspection of Cattell's scree plot. An examination of the constituent items for this factor structure also indicated that items loaded most highly on a single factor. This single factor represents a measure of perceived understanding of trial results. PCA was then conducted using an oblique (direct oblimin) rotation, specifying the extraction of one factor. This model explained a combined 69.58% of the variance in patients understanding of the TRUST thyroid trial.

Cost of conducting PPI
The total cost of this study amounted to €8,049 (see Supplementary File 6: Costs of conducting PPI).

Discussion
While PPI is increasingly recognised as an important element of clinical research, evidence on optimal methods and potential impact is lacking 4,9 . Previous research conducted on the impact of PPI has largely focused on the experiences of participants and researchers 25 and on the research process in broad terms 26 . In this study, our primary outcome was specific: a quantitative measure of patient understanding of trial results between those who received the patient-based approach and the standard approach. To our knowledge there has been no previous research conducted on the impact of PPI on patient understanding of trial results.
The involvement of clinical trial participants in this study offered insightful perspectives on the information needs of the study population in terms of receiving end of trial results. Study findings show that trial participants want to receive the results of the clinical trial in which they had participated. This is supported by much of the available literature on patients' preferences of receiving results, with up to 90% of participants in previous studies reporting a desire to receive results 27 . Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an 'ethical imperative or 'moral obligation'. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism 12,28 .
Unsurprisingly, findings also show that participants want to receive results that are accessible and easy to understand. In this study, the preferred format of receiving results was a letter posted to them directly from the TRUST trial. This preference is also consistent with the literature on patient preferences of receiving results. A previous study investigating prefrences of individuals taking part in a cardiac rehabilitation trial found that 80% of trial participants preferred to receive the results by post 29 . The patient-based approach identified in this study was feasible for researchers to develop with significant involvement from trial participants and adult literacy experts.
Previous studies exploring participants' reactions found that sharing trial results with participants can cause some negative impacts such as anxiety, anger, guilt, upset and confusion 30-32 . As far as researchers in this study are aware, providing results did not cause any negative impacts. This may have been due to the fact that the TRUST trial had a low risk of morbidity or mortality compared to some of the other studies citing negative impacts. Both result methods contained the telephone number, email address and postal address of the research team and participants were urged to contact should they have any questions or concerns relating to the study. The research team did not receive any queries.
Previous systematic reviews highlight the lack of evidence on economic analysis of PPI and call for researchers to consider the costs of its implementation 26,33 . As discussed previously research funders are increasingly demanding that PPI be carried out in research. However, the costs of PPI are often underestimated and can cause a significant financial burden on research project budgets 26,33-35 . It is extremely important that researchers plan PPI at the grant proposal stage and estimate the costs appropriately. If these costs are not correctly estimated during the initial stages of developing research proposals, they may cause a financial burden on PPI partners.
Participants in this study were not paid for their time but were provided with a €20 voucher to cover travel expenses. When PPI is not the primary focus of a study, researchers do not consider the cost implications at the beginning of the study and are often tied with limited resources to carry out PPI 34-36 . INVOLVE, the national advisory group supporting active public involvement in health services, public health and social care research in the UK, have recommended that PPI partners should be paid for their involvement 37 . Despite this, existing research suggests that institutional difficulties make negotiating the mechanisms of paying participants very difficult 34 . One study reported that in order for participants to be remunerated for their efforts, they needed to be registered as employees, a process that incurred much paperwork and time delays 34 . This study outlines the cost of conducting PPI and includes a full breakdown of costs (see Supplementary File 6: Costs of conducting PPI). This breakdown provides a template to other researchers who plan to carry out and evaluate PPI as part of their research. It is important to note that not all costs associated with carrying out the study were included in this amount. For example, the only salary costed was that of the research assistant. The expertise provided by other members of the study team were not included in the total cost as they were being paid by the University or other research grants. The total cost of conducting this study was €8,049 which is not insignificant but should be considered in the context of the cost of large-scale trials.

Strengths and Limitations of the study
While this study provides important insights into patients' preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that 'to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population' 3 . All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed to long-term follow up. This is a strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial 38 , thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention. Previous research suggests that people that actively choose to engage in research either as research participants or involvement partners are more likely to be middle-class and highly educated 39,40 . In this study, those that attended the focus groups and PPI group were similar in education level to those that did not attend. This is not surprising considering the entire study sample had already actively volunteered to take part in the TRUST trial.
Secondly, the results of the patient understanding questionnaire show that the levels of patient understanding were similar between the two groups. However, this study was underpowered to detect an effect. As this was a Study Within A Trial (SWAT), the power was limited by the sample size that was available to us from the trial (n=115). Furthermore, validation of the patient understanding questionnaire was limited by the sample size in this study. While validation of the questionnaire was limited, exploratory factor analysis provided some evidence that the questionnaire is a useful tool for measuring patient understanding of trial results. The developed questionnaire can be tailored for use in other trials in future examinations of patients understanding of trial results. This would provide insight into patient understanding and provide further validation data.
Thirdly, all SWATparticipants were aged 65 and over. The layout, format and language of this patient-based approach which was identified and developed may only be relevant for this study population. Other trial populations may prefer to receive the results via email, online or in person from a member of the study team 12 . The evidence on patient preferences of receiving trial results is limited, therefore further research is needed to explore patient preferences of receiving trial results amongst different study populations.
It is also important to point out that the control group in this study received a copy of the trial results in a press release format. Most trial participants do not receive this. While this control method was a step further than normal procedure, the researchers in this study felt this was appropriate. The information presented in the press release was similar to that of the patient-based approach. However, the format and layout of the press release was different. Information was writtern in four long paragraphs separated by individual headings. It was also much shorter (1 page in total) that the patient-based approach (3 pages in total). Given the fact that press releases are written by public relations professionals with a view to communicating effectively and efficiently, this may have potentially minimised differences between the intervention and control conditions. The primary outcome of this study was assessing the impact of PPI on patient understanding of results, however, this was not the only potential impact. In hindsight, we adopted a limited approach to PPI in this study as we did not involve our PPI partners from the outset of the SWAT. Involving PPI partners in the development of core outcome sets for this SWAT could have identified other more appropriate primary outcome measures 41 .
The aim of this SWAT was to investigate methods of disseminating trial findings to trial participants by using a PPI approach to identify, develop and evaluate a patient-based method of receiving trial results. The PPI approach actively involved focus group participants in making decisions about the result method and worked with PPI partners to co-develop the result letter. However, PPI partners were not involved in other aspects of the research process such as research design, data collection or analysis. This is partly due to the fact that PPI is a relatively new concept in clinical trials. As the majority of the literature has only been published in the last 12 months, there is little evidence available on the impact of PPI and no gold standard or comprehensive guidelines for researchers to follow 29 . Thornton 2 suggests that in order for PPI to develop it is important to record its social and cultural history by collecting comprehensive databases and undertaking ongoing reviews of the impact of PPI. This paper along with the study protocol have been written in adherence with the Guidelines for Reporting Involvement of Patients and the Public 23 , thus providing templates for involving patients and the public in clinical trial design and development. This study is an important step forwards in documenting the process of conducting PPI as part of a SWAT and evaluating its impact. Future research is needed to further develop PPI in clinical trial settings. As there is currently no gold standard or comprehensive guidelines for researchers to follow when evaluating the impact of PPI, further research is needed. This research should involve PPI partners in the development of core outcome sets for evaluating PPI impact. These would significantly enhance the literature in the area.

Conclusion
Patient and Public Involvement (PPI) is advocated for every step of the trial process. We have demonstrated that it is feasible to involve PPI partners in the development of dissemination materials. Sharing clinical trial results with participants in a format understandable to laypersons will soon be a legal requirement 11 . However, there is a significant lack of evidence as to the most appropriate methods of sharing results with participants. The study identified and developed a patient-based approach to disseminating clinical trial results for trial participants. Although, in this study PPI did not influence patients' final understanding of results, it documents the process of conducting PPI within the clinical trial setting. This process may be useful for other trialists interested in conducting and evaluating the impact of PPI in clinical trials.

Ethics approval and consent to participate
The research was approved in Ireland by the Clinical Research Ethics Committee of the Cork Teaching Hospitals, UCC, Ref ECM 4 (t).
All participants provided signed informed consent to take part in the study.

Data availability
The raw data from this study cannot be sufficiently de-identified, and therefore are not publicly available. However, the data from the current study are available for further (collaborative) research purposes on reasonable request. Available datasets include transcripts from focus groups, field notes from PPI sessions and responses from the patient understanding questionnaire.
To access the data, please contact the corresponding author (emmy.racine@ucc.ie) or the Principal Investigator (patricia. kearney@ucc.ie). Researchers must provide a written proposal on how the data will be used in research before access is granted. The paper is an important contribution to enhance patient and public involvement in trial research. A very worthy SWAT, one of a few SWATs conducted within the TRUST trial. Please explain SWAT at the outset.

Some suggestions:
While the authors refer to the PPI definition by INVOLVE it would have been helpful to get a clearer understanding of the authors' specific understanding and conceptualisation of PPI in the context of their study. The approach of involving patients in shaping the dissemination of results is laudable but admittedly quite limiting and appears to be an afterthought rather than an integral element of the trial. The paper very much resides in the traditional experimental (albeit using a preference-based design), researcher-led paradigm (clearly expressed in the 'randomisation' to determine effectiveness of modes of dissemination -which somewhat contravenes the notion of engaging with a diverse public and patient population for bespoke dissemination) with limited shift of control over the research process to patients or the public. For example, how were study participants involved at the intersection of analysis, dissemination and knowledge transfer (selection of outcomes that are relevant? Decision on subgroup analysis? Involvement in interpretation of analysis findings? Decisions about outlets for dissemination (audiences, events, settings, format and content?).
It would be helpful to see a more detailed description of the PPI Group. How was this group selected, what is its composition, how frequently did it meet and what was its immediate influence in the research process throughout. Did members of the PPI group participate as co-researchers at any stage of contribute to the facilitation, analysis etc of the focus group results?
Paper mentions qualitative aspect and discusses the need to ensure accurate and relevant information is being shared with participants. Wonder if benefit could be derived from drawing upon the notion of "translational statistics", where additional statistical information is obtained, analysed and accurately translated to clinically meaningful findings -perhaps with the authors referring to the tension of statistical significance versus clinical relevance and the importance of minimising statistical jargon when presenting trial results. Great to see contribution of NALA -but did the authors experience this barrier of translating statistical results into findings that were clinically relevant, most notably in the context of PPI in understanding this? The suggestion therefore is that including PPI here helps to minimise both statistical and clinical jargon, which can often be challenging, for a concise and accurate report.
Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), "This document highlights the information that trial participants should understand after reading their trial summary." The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.
It would be important to distinguish between 'literacy' and 'health literacy'. They represent related but substantially different concepts.
In line with other members of the trial community, the paper could benefit from the word 'transparency'paragraph 2 introduction -to put it in context with other papers discussing sharing of trial findings and results. The authors could also consider making reference to the AllTrials initiative in paragraph 2 (not directly relevant to PPI, but important nonetheless in the movement towards transparency in trials).
It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.
Methods section of Phase 2 -" One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial." -what other parts of the study was this member involved in? Did their unblinding have any impact on any other aspects of the study?
The paper could benefit from briefly mentioning the 'moral obligation' trialists have to share results -in this sense, I mean results to be the trial findings -(results in the context of data sharing is subject to new EU GDPR, which is something the authors could consider to mention but unlikely to have a place for discussion in this article; justification for not being able to share the data is already briefly mentioned in the "data availability" paragraph). So the term 'moral obligation' of trialists would complement the qualitative quotes the authors have in the results section about patients feeling they contributed to research.
Phase 1 results part: "This feeling of contributing for a collective benefit was further reinforced when participants discussed their desire to understand how the results of the trial will be implemented by medical experts and ultimately how it will affect others who have the condition:" à could mention one aspect, particularly this being a drug trial, of how statistics can explain why one treatment works on some patients and not others etc and that this may be something public and patients would like to know, therefore is important to accurately portray.
"education" -demographic not discussed in detail in other sections of paper-only mentioned in terms of not being statistically significantly different -need more? Relevance to PPI is not explained thoroughly and should be strengthened Great mention of CT regulation and checklist when reporting PPI -but could also benefit from mentioning the fundamental reporting guidance in trials, such as putting SPIRIT into context and relevance, if any, to layman's terms and similar to using CONSORT as a reference and explaining this to PPI? Also refer to reporting section for public www.clinicaltrials.gov Clearly, it is a strength of the SWAT having participants already from the trial -see the trial through from beginning to end so their help is important in terms of accurate context/background when interpreting and disseminating findings Generally, well-written, well laid out, concise and important paper.

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate? I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility? No source data required

Are the conclusions drawn adequately supported by the results? Partly
No competing interests were disclosed.

Competing Interests:
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Author Response 11 Jan 2019 , University College Cork, Cork, Ireland Emmy Racine The paper is an important contribution to enhance patient and public involvement in trial research. A very worthy SWAT, one of a few SWATs conducted within the TRUST trial. Please explain SWAT at the outset.
Firstly, thank you for your comprehensive comments.
To address this, we have inserted the following sentence into the abstract: "The aim of this Study Within A Trial (SWAT) is to use…." (Abstract) While the authors refer to the PPI definition by INVOLVE it would have been helpful to get a clearer understanding of the authors' specific understanding and conceptualisation of PPI in the context of their study. The approach of involving patients in shaping the dissemination of results is laudable but admittedly quite limiting and appears to be an afterthought rather than an integral element of the trial. The paper very much resides in the traditional experimental (albeit using a preference-based design), researcher-led paradigm (clearly expressed in the 'randomisation' to determine effectiveness of modes of dissemination -which somewhat contravenes the notion of engaging with a diverse public and patient population for bespoke dissemination) with limited shift of control over the research process to patients or the public. For example, how were study participants involved at the intersection of analysis, dissemination and knowledge transfer (selection of outcomes that are relevant? Decision on subgroup analysis? Involvement in interpretation of analysis findings? Decisions about outlets for dissemination (audiences, events, settings, format and content?).
We have reflected on our approach to PPI in this Study Within a Trial and have added the following text to the manuscript to address this comment: The aim of this SWAT was to investigate methods of disseminating trial findings to trial participants by using a PPI approach to identify, develop and evaluate a patient-based method of receiving trial results. The PPI approach actively involved focus group participants in making decisions about the result method and working with PPI partners to co-develop the result letter. However, PPI partners were not involved in other aspects of the research process such as research design, data collection or analysis. This is partly due to the fact that PPI is a relatively new concept in clinical trials. As the majority of the literature has only been published in the last 12 months, there is little evidence available on the impact of PPI and no gold standard or comprehensive guidelines for researchers to follow (30). (Discussion: Strengths and Limitations, Paragraph 5)

It would be helpful to see a more detailed description of the PPI Group. How was this group selected, what is its composition, how frequently did it meet and what was its immediate influence in the research process throughout. Did members of the PPI group participate as co-researchers at any stage of contribute to the facilitation, analysis etc of the focus group results?
A more detailed description of the PPI group has been added to the manuscript to address this comment: A PPI group was established to develop and refine the content of the patient-based approach for the dissemination of results. During the focus groups, three TRUST trial participants volunteered to take part in the PPI group. In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This individual was keen to learn more about research and expressed an interested in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner.(Methods: PPI group) There were four. PPI partners in total (three trial participants and one older adult). Each partner took part in one one-to-one session (Results: PPI group) Members of the PPI group did not participate as co-researchers in the facilitation, analysis etc. of focus group results. We have addressed our approach to PPI in the previous comment and have made necessary additions to the main manuscript.
Paper mentions qualitative aspect and discusses the need to ensure accurate and relevant information is being shared with participants. Wonder if benefit could be derived from drawing upon the notion of "translational statistics", where additional statistical information is obtained, analysed and accurately translated to clinically meaningful findings -perhaps with the authors referring to the tension of statistical significance versus clinical relevance and the importance of minimising statistical jargon when presenting trial results. Great to see contribution of NALA -but did the authors experience this barrier of translating statistical results into findings that were clinically relevant, most notably in the context of PPI in understanding this? The suggestion therefore is that including PPI here helps to minimise both statistical and clinical jargon, which can often be challenging, for a concise and accurate report.
While we believe this is an important point, we believe it was not a major contributor in our study. As there were no statistically significant or clinically relevant results, the results of the trial were simple and straightforward (ie. no effect), and so we did not have this challenge.
Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), "This document highlights the information that trial participants should understand after reading their trial summary." The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.
The authors do not mean to sound prescriptive in their approach. This sentence has been changed The authors do not mean to sound prescriptive in their approach. This sentence has been changed to the following: 'This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial'. (Methods: Evaluation) It would be important to distinguish between 'literacy' and 'health literacy'. They represent related but substantially different concepts.
Both concepts were important in this study. We collaborated with the National Adult Literacy Agency to ensure that the material in the results letter to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented to them (health literacy).
To ensure accuracy, we have changed the wording throughout the document to 'adult literacy' and described our approach as follows: Adult literacy review While the PPI group had significant input into the format and language used in the patient-based approach. The research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national "Plain English" standards. These standards ensured that the information presented to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented (health literacy) (15). This review was an iterative process with several drafts exchanged for editing. (L-P-) In line with other members of the trial community, the paper could benefit from the word 'transparency' -paragraph 2 introduction -to put it in context with other papers discussing sharing of trial findings and results. The authors could also consider making reference to the AllTrials initiative in paragraph 2 (not directly relevant to PPI, but important nonetheless in the movement towards transparency in trials).
Yes we agree, this paragraph would benefit from a reference to the recent movement towards transparency in trials. The following text has been added: The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols (11), the Consolidated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials (12) and the AllTrials initiative calls for all past and present trials to be registered and their full methods and summary results reported (13). Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community (11). (Introduction, Paragraph 2).
It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.
The following text has been added to the manuscript to describe how the CODM model was specifically used in this study: The Consensus-Oriented-Decision-Making (CODM) model was used to guide the group to reach a consensus. The CODM model is accepted as a flexible model for reaching decisions (14). In this study some of the steps were initiated by the focus group facilitator and others occurred naturally as a follow on from the previous step. Below is an outline of each of the seven steps of the CODM model and how they were used in this study: 1. Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants. 2. Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial. 3. Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used. 4. Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language. 5. Choosing a direction: This step occurred naturally as part of the previous step. 6. Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback. 7. Closure: This step occurred naturally as part of the previous step. (Methods: Phase 1)

Methods section of Phase 2 -" One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial." -what other parts of the study was this member involved in? Did their unblinding have any impact on any other aspects of the study?
This member of the research team (AC) was involved in the design of the study, preparing study materials, performing data collection and developing the result material. After they were un-blinded in order to perform the randomisation and distribute the results, they were not involved in the data analysis or interpretation in any way. The following sentence has been added to the manuscript to provide clarity on this: "As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way." (Methods: Phase 2) The paper could benefit from briefly mentioning the 'moral obligation' trialists have to share results -in this sense, I mean results to be the trial findings -(results in the context of data sharing is subject to new EU GDPR, which is something the authors could consider to mention but unlikely to have a place for discussion in this article; justification for not being able to share the data is already briefly mentioned in the "data availability" paragraph). So the term 'moral obligation' of trialists would complement the qualitative quotes the authors have in the results section about patients feeling they contributed to research.
We have inserted the following text into the discussion to address this: "Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an 'ethical imperative or 'moral obligation'. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism (10, 23)." (Discussion, Paragraph 2) We have not mentioned the EU GDPR in this paper as we feel it is not directly relevant to sharing trial results with participants. The upcoming EU Clinical Trial regulation is mentioned in the introduction which will be the main EU regulation which requires investigators to share clinical trial results with participants in a format understandable to laypersons.
Phase 1 results part: "This feeling of contributing for a collective benefit was further reinforced when participants discussed their desire to understand how the results of the trial will be implemented by medical experts and ultimately how it will affect others who have the condition:" à could mention one aspect, particularly this being a drug trial, of how statistics can explain why one treatment works on some patients and not others etc and that this may be something public and patients would like to know, therefore is important to accurately portray.
We acknowledge that understanding how statistics can explain why one treatment works on some patients and not others is an important aspect and may be something that patients would like to know. However, this aspect was not brought up by the trial participants in this study (either during the focus groups or PPI sessions). As this paper was about listening to trial participants and giving them the information they wanted to know, we feel it would be inappropriate to add it in to the paper at this late stage and presume that it is something that they wanted to know. Therefore, while it is an important aspect, we have not made any changes to address it in our paper.

"education" -demographic not discussed in detail in other sections of paper-only mentioned in terms of not being statistically significantly different -need more? Relevance to PPI is not explained thoroughly and should be strengthened
We have added the following text to the discussion to address this: Previous research suggests that people that actively choose to engage in research either as research participants or involvement partners are more likely to be middle-class and highly educated (34,35). In this study, those that attended the focus groups and PPI group were similar in education level to those that did not attend. This is not surprising considering the entire study sample had already actively volunteered to take part in the TRUST trial. (Discussion: Strengths and Limitations, Paragraph 1)

Great mention of CT regulation and checklist when reporting PPI -but could also benefit from mentioning the fundamental reporting guidance in trials, such as putting SPIRIT into context and relevance, if any, to layman's terms and similar to using CONSORT as a reference and explaining this to PPI? Also refer to reporting section for public
The following text has been inserted into the introduction to address this comment: The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols (11), the Consolidated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials (12) and the AllTrials initiative calls for all past and present trials to be registered and their full methods and summary results reported (13). Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community (11). (Introduction, Paragraph 2)

Clearly, it is a strength of the SWAT having participants already from the trial -see the trial through from beginning to end so their help is important in terms of accurate context/background when interpreting and disseminating findings
We have made the following changes to the manuscript to address this: This section of the discussion has been reworded from 'Limitations of the study' to 'Strengths and limitations of the study' (Discussion) This paragraph in question has been rephrased as a highlight of the SWAT and moved to the first paragraph in the 'Strengths and limitations of the study' section. It now reads as following: "While this study has provided important insights into patients' preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that 'to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population' (3). All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed for long-term follow up. This is a major strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial (31), thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention." Discussion: Strengths and Limitations, Paragraph 1)

Generally, well-written, well laid out, concise and important paper.
No competing interests were disclosed.

Competing Interests:
Author Response 11 Jan 2019 , University College Cork, Cork, Ireland Emmy Racine

The paper is an important contribution to enhance patient and public involvement in trial research. A very worthy SWAT, one of a few SWATs conducted within the TRUST trial. Please explain SWAT at the outset.
Firstly, thank you for your comprehensive comments.
To address this, we have inserted the following sentence into the abstract: "The aim of this Study Within A Trial (SWAT) is to use…." (Abstract) To address this, we have inserted the following sentence into the abstract: "The aim of this Study Within A Trial (SWAT) is to use…." (Abstract)

While the authors refer to the PPI definition by INVOLVE it would have been helpful to get a clearer understanding of the authors' specific understanding and conceptualisation of
PPI in the context of their study. The approach of involving patients in shaping the dissemination of results is laudable but admittedly quite limiting and appears to be an afterthought rather than an integral element of the trial. The paper very much resides in the traditional experimental (albeit using a preference-based design), researcher-led paradigm (clearly expressed in the 'randomisation' to determine effectiveness of modes of dissemination -which somewhat contravenes the notion of engaging with a diverse public and patient population for bespoke dissemination) with limited shift of control over the research process to patients or the public. For example, how were study participants involved at the intersection of analysis, dissemination and knowledge transfer (selection of outcomes that are relevant? Decision on subgroup analysis? Involvement in interpretation of analysis findings? Decisions about outlets for dissemination (audiences, events, settings, format and content?).
We agree that our approach to PPI in this study was limited. We involved trial participants in shaping the dissemination of results and did not involve them in other stages of the research process. We agree that there was a limited shift of control over the research process to PPI partners. We have reflected on our limited approach to PPI and have added the following text to the manuscript to address this comment: The aim of this SWAT was to investigate methods of disseminating trial findings to trial participants by using a PPI approach to identify, develop and evaluate a patient-based method of receiving trial results. The PPI approach actively involved focus group participants in making decisions about the result method and working with PPI partners to co-develop the result letter. However, PPI partners were not involved in other aspects of the research process such as research design, data collection or analysis. This is partly due to the fact that PPI is a relatively new concept in clinical trials. As the majority of the literature has only been published in the last 12 months, there is little evidence available on the impact of PPI and no gold standard or comprehensive guidelines for researchers to follow (30). (Discussion: Strengths and Limitations, Paragraph 5) This study is an important step forwards in documenting the process of conducting PPI as part of a SWAT and evaluating its impact. (Discussion: Strengths and Limitations, Paragraph 5)

It would be helpful to see a more detailed description of the PPI Group. How was this group selected, what is its composition, how frequently did it meet and what was its immediate influence in the research process throughout. Did members of the PPI group participate as co-researchers at any stage of contribute to the facilitation, analysis etc of the focus group results?
A more detailed description of the PPI group has been added to the manuscript to address this comment: A PPI group was established to develop and refine the content of the patient-based approach for the dissemination of results. During the focus groups, three TRUST trial participants volunteered to take part in the PPI group. In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This identified from a previous qualitative research study undertaken by the research team. This individual was keen to learn more about research and expressed an interested in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner. (Methods: PPI group).
There were four. PPI partners in total (three trial participants and one older adult). Each partner took part in one one-to-one session (Results: PPI group).
Members of the PPI group did not participate as co-researchers in the facilitation, analysis etc. of focus group results. We have addressed our limited approach to PPI in the previous comment and have made necessary additions to the main manuscript.
Paper mentions qualitative aspect and discusses the need to ensure accurate and relevant information is being shared with participants. Wonder if benefit could be derived from drawing upon the notion of "translational statistics", where additional statistical information is obtained, analysed and accurately translated to clinically meaningful findings -perhaps with the authors referring to the tension of statistical significance versus clinical relevance and the importance of minimising statistical jargon when presenting trial results. Great to see contribution of NALA -but did the authors experience this barrier of translating statistical results into findings that were clinically relevant, most notably in the context of PPI in understanding this? The suggestion therefore is that including PPI here helps to minimise both statistical and clinical jargon, which can often be challenging, for a concise and accurate report.
While we believe this is an important point, we believe it was not a major contributor in our study. As there were no statistically significant or clinically relevant results, the results of the trial were simple and straightforward (ie. no effect), and so we did not have this challenge.
Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), "This document highlights the information that trial participants should understand after reading their trial summary." The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.
The authors do not mean to sound prescriptive in their approach. This sentence has been changed to the following: 'This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial'. (Methods: Evaluation)

It would be important to distinguish between 'literacy' and 'health literacy'. They represent related but substantially different concepts.
Both concepts were important in this study. We collaborated with the National Adult Literacy Agency to ensure that the material in the results letter to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented to them (health literacy). 1.

2.
To ensure accuracy, we have changed the wording throughout the document to 'adult literacy' and described our approach as follows:

Adult literacy review
While the PPI group had significant input into the format and language used in the patient-based approach. The research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national "Plain English" standards. These standards ensured that the information presented to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented (health literacy) (15). This review was an iterative process with several drafts exchanged for editing.
In line with other members of the trial community, the paper could benefit from the word 'transparency' -paragraph 2 introduction -to put it in context with other papers discussing sharing of trial findings and results. The authors could also consider making reference to the AllTrials initiative in paragraph 2 (not directly relevant to PPI, but important nonetheless in the movement towards transparency in trials).
Yes we agree, this paragraph would benefit from a reference to the recent movement towards transparency in trials. The following text has been added: The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols (11), the Consolidated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials (12) and the AllTrials initiative calls for all past and present trials to be registered and their full methods and summary results reported (13). Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community (11). (Introduction, Paragraph 2).
It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.
The following text has been added to the manuscript to describe how the CODM model was specifically used in this study: The Consensus-Oriented-Decision-Making (CODM) model was used to guide the group to reach a consensus. The CODM model is accepted as a flexible model for reaching decisions (14). In this study some of the steps were initiated by the focus group facilitator and others occurred naturally as a follow on from the previous step. Below is an outline of each of the seven steps of the CODM model and how they were used in this study: Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants.
Open discussion: The facilitator asked the group whether or not they think results should be 2. 3.

7.
Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial. Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used. Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language. Choosing a direction: This step occurred naturally as part of the previous step. Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback. Closure: This step occurred naturally as part of the previous step. (Methods: Phase 1)

Methods section of Phase 2 -" One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial." -what other parts of the study was this member involved in? Did their unblinding have any impact on any other aspects of the study?
This member of the research team (AC) was involved in the design of the study, preparing study materials, performing data collection and developing the result material. After they were un-blinded in order to perform the randomisation and distribute the results, they were not involved in the data analysis or interpretation in any way. The following sentence has been added to the manuscript to provide clarity on this: "As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way." (Methods: Phase 2) The paper could benefit from briefly mentioning the 'moral obligation' trialists have to share results -in this sense, I mean results to be the trial findings -(results in the context of data sharing is subject to new EU GDPR, which is something the authors could consider to mention but unlikely to have a place for discussion in this article; justification for not being able to share the data is already briefly mentioned in the "data availability" paragraph). So the term 'moral obligation' of trialists would complement the qualitative quotes the authors have in the results section about patients feeling they contributed to research.
We have inserted the following text into the discussion to address this: "Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an 'ethical imperative or 'moral obligation'. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism (10, 23)." (Discussion, Paragraph 2) We have not mentioned the EU GDPR in this paper as we feel it is not directly relevant to sharing trial results with participants. The upcoming EU Clinical Trial regulation is mentioned in the introduction which will be the main EU regulation which requires investigators to share clinical trial results with participants in a format understandable to laypersons.

Phase 1 results part: "This feeling of contributing for a collective benefit was further
Phase 1 results part: "This feeling of contributing for a collective benefit was further reinforced when participants discussed their desire to understand how the results of the trial will be implemented by medical experts and ultimately how it will affect others who have the condition:" à could mention one aspect, particularly this being a drug trial, of how statistics can explain why one treatment works on some patients and not others etc and that this may be something public and patients would like to know, therefore is important to accurately portray.
We acknowledge that understanding how statistics can explain why one treatment works on some patients and not others is an important aspect and may be something that patients would like to know. However, this aspect was not brought up by the trial participants in this study (either during the focus groups or PPI sessions). As this paper was about listening to trial participants and giving them the information they wanted to know, we feel it would be inappropriate to add it in to the paper at this late stage and presume that it is something that they wanted to know. Therefore, while it is an important aspect, we have not made any changes to address it in our paper.

"education" -demographic not discussed in detail in other sections of paper-only mentioned in terms of not being statistically significantly different -need more? Relevance to PPI is not explained thoroughly and should be strengthened
We have added the following text to the discussion to address this: Previous research suggests that people that actively choose to engage in research either as research participants or involvement partners are more likely to be middle-class and highly educated (34,35). In this study, those that attended the focus groups and PPI group were similar in education level to those that did not attend. This is not surprising considering the entire study sample had already actively volunteered to take part in the TRUST trial. (Discussion: Strengths and Limitations, Paragraph 1) Great mention of CT regulation and checklist when reporting PPI -but could also benefit from mentioning the fundamental reporting guidance in trials, such as putting SPIRIT into context and relevance, if any, to layman's terms and similar to using CONSORT as a reference and explaining this to PPI? Also refer to reporting section for public The following text has been inserted into the introduction to address this comment: The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols (11), the Consolidated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials (12) and the AllTrials initiative calls for all past and present trials to be registered and their full methods and summary results reported (13). Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community (11). (Introduction, Paragraph 2)

Clearly, it is a strength of the SWAT having participants already from the trial -see the trial through from beginning to end so their help is important in terms of accurate through from beginning to end so their help is important in terms of accurate context/background when interpreting and disseminating findings
We have made the following changes to the manuscript to address this: This section of the discussion has been reworded from 'Limitations of the study' to 'Strengths and limitations of the study' (Discussion) This paragraph in question has been rephrased as a highlight of the SWAT and moved to the first paragraph in the 'Strengths and limitations of the study' section. It now reads as following: "While this study has provided important insights into patients' preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that 'to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population' (3). All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed for long-term follow up. This is a major strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial (31), thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention." Discussion: Strengths and Limitations, Paragraph 1) Generally, well-written, well laid out, concise and important paper.
No competing interests were disclosed. Competing Interests: materials as provided by the lead TRUST site. The SWAT protocol was published last year.
Although the study is small and has some limitations it is an important piece in the result provision literature for trials, providing the detailed account of a thorough approach to the results provision process in trials.
We have a few comments which should be addressed:

Overall
Some people wonder why it is necessary to quantify at all the contribution public and patients make to a trial given that they are clearly trial stakeholders and we don't do this for other stakeholders such as the statistician and the trial manager. Other people definitely do think it is necessary to quantify the gain from PPI. It would be good if the authors could mention this somewhere in the article. It could be early as a rationale for doing the study, or the Discussion in recognition of the different views.
-Identifying the intervention results as 'patient-preferred method' is at times confusing (it is easily misunderstood as the mode of delivery). Alternatives could be 'patient-based approach', 'patient-generated materials' or 'patient-centred approach'.

Abstract
Background -rather than referring to 'an increasing consensus' the 'increasingly recognised as an essential component' text mentioned in the Introduction in the main text seems easier to agree with.
Results -it would be better rephrase so that it is clear the results are specific to the TRUST trial participants rather than in general. For example 'TRUST patients want to receive…' Results -it is stated that "TRUST participants (n=101) were randomised to the intervention." Unless we have misunderstood something, this must be a mistake as this is the total available sample to be randomised so not all of them can get the intervention. Our guess is that 101 were involved in the SWAT evaluation and that 51 were randomised to the intervention.
Conclusion -please rephrase "conduct PPI" to e.g. 'We have demonstrated that it is feasible to involve PPI members in the development of result dissemination materials.' Conclusion -The reference to age in "The study identified and developed a patient-preferred method of receiving clinical trial results for older adults over 65 years." wasn't part of the aim of the study as far as we can see and the authors should reconsider including it in the conclusion. If developing a strategy for the over 65s was a distinct aim of the study, then the text of study aims needs tweaked.
Conclusion -it would be good if the authors could see if they can improve ",… it provides a record of the process of conducting PPI within the clinical trial setting." We're not entirely clear what is meant.

Main text
Methods -please ensure that the term used to denote the control/comparison group is consistent throughout the paper as currently these terms are used interchangeably (also in tables and figures).
In the results section it is stated that the PPI group consisted of three trial participants and one older adult. It would be good to give a sentence in Methods on how this 'older adult' was identified and recruited.
The published protocol doesn't say that the materials will also reviewed by the National Adult Literacy The published protocol doesn't say that the materials will also reviewed by the National Adult Literacy Agency (NALA). That this is an addition should be mentioned, along with why. It's not a problem, just that it would be good to be clear.

Results
Table 1 -Clarify the table title and include that it is the 'characteristics of the trial participants stratified by participation in the different stages of the study'. Table 1 -Some of the column headings need clarification.
"TRUST participants (n=104)" it should be clear these are the Irish TRUST sample only. It should also be noted that three participants were not eligible for the rest of the study as they took part in the PPI group.
"Attended focus groups (n=19), Total sample n=38 RR=50%" it should be clear only 38 were invited/eligible or that this is a sub-sample of the 104.
Please add an explanation of what RR is short for.
"Randomised into intervention (n=101)", it should be clear n=101 were randomised (delete "into intervention"). See earlier comment about this. It's involved in SWAT evaluation that has n=101 as far as we can see, not randomised to intervention.
Main text relating to figure 1 -please include the denominator in the parentheses as the percentages and n can be misleading on their own when small numbers are concerned.  Any amendments which cannot be accommodated within the journal 15 word limit for the title of tables and figures could be added to a legend.
In relation to the cost -it should also be clear that the NALA review is included in the cost.

Discussion
Paragraph 2 -please clarify the description of the intervention in relation to the literature as it currently confounds mode of delivery with only one type of material that can be posted to participants.
Paragraph 3 -please change 'negative reaction' to 'negative impact' or similar -the reaction might be negative in researchers' view but completely justified. You should also comment on the trial context in relation to this as the TRUST trial is relatively low risk compared to some of the trials cited.
Paragraph 4 relates to the aim of the SWAT and should probably be moved to a bit earlier in the Discussion, paragraph 2 perhaps.
Paragraph 5 and 6 (about cost) -The main message seems to be that PPI needs to be planned and costed appropriately -please be clear about this. We're not sure this sentence is necessary "As the primary focus of this study was to carry out PPI, researchers carefully considered the costs at grant primary focus of this study was to carry out PPI, researchers carefully considered the costs at grant proposal stage and wrote these costs into the budget." -The authors paid a fixed cost for travel costs for PPI participation rather than to cover participants' time. The UK's INVOLVE has recommendations for covering PPI participants' time and it might be nice to link these to the sentences that follow about the institutional difficulties of actually making these payments (which colleagues have experienced at our own institution).
-Please include some consideration of the cost of carrying out the SWAT -is it a lot of money for a trial and is it money well spent? Please also consider that there may be some hidden costs e.g. expert input into questionnaire development an expert input in to the qualitative phase. This kind of expertise will not be available to all trials.
Paragraph 7 (Limitations of the study) -The sample size of the SWAT was limited but to n=104 not n=115 (11 had withdrawn before the SWAT started so were never eligible).
Paragraph 8 (Limitations of the study) -Change "Secondly, all trial participants were aged 65 and over." To 'Secondly, all SWAT participants were aged 65 and over.' While the former is true the SWAT did not include all trial participants.
-Change "Alternative study populations may prefer…" to 'Other trial populations may prefer…' Paragraph 8 (Limitations of the study) -This paragraph actually highlights a strength of the SWAT which is that the PPI members consulted were participants in the trial. We'd suggest highlighting that many SWATs will be underpowered because they are generally unable to change the size of the host trial; they are made for meta-analysis really.
Paragraph 10 -This is an important consideration. In relation to this it would be beneficial to include the research team's reflection on what is the most appropriate primary outcome to measure regarding dissemination of trial results. The use of PPI clearly had an impact in that the TRUST trial would not have posted results to Irish participants at all if it had not been for this SWAT (i.e. the Press release control for the SWAT was more than planned for the trial without the SWAT).
-It would be informative to have the authors comment on the difference in content of the two versions of the results used in the SWAT -was there a notable difference?
-Would the authors in hindsight consider asking their PPI members what they think the primary outcome in the evaluation of the results materials should have been? What would the authors recommend for future evaluations of PPI impact?
Paragraph 11 -It would be good to expand on "While this study provides a record of the process of conducting PPI as part of a SWAT, future research is needed to further develop PPI in clinical trial settings." Any thoughts on what sort of PPI research?

If applicable, is the statistical analysis and its interpretation appropriate? Partly
Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Partly HB is currently working on a related project, Reporting of clinical trial results Competing Interests: appropriately to participants. ST has no conflicts.

Reviewer Expertise: Trial methodology, feedback provision to trial participants
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Author Response 11 Jan 2019 , University College Cork, Cork, Ireland Emmy Racine

This research article reports a Study Within A Trial (SWAT) hosted by the Thyroid
Hormone Replacement for Subclinical Hypo-Thyroidism Trial (TRUST). The TRUST trial is an international trial and the SWAT was only conducted in one of the participating countries, namely Ireland. The purpose of the SWAT was to use a 'Patient and Public Involvement (PPI) approach' to identify and develop results reporting for the TRUST trial to its participants and subsequently evaluate the result materials. Evaluation of the result materials was carried out by assessing recipients' understanding of the trial results compared to result materials as provided by the lead TRUST site. The SWAT protocol was published last year.
Although the study is small and has some limitations it is an important piece in the result provision literature for trials, providing the detailed account of a thorough approach to the results provision process in trials.
We have a few comments which should be addressed: Some people wonder why it is necessary to quantify at all the contribution public and patients make to a trial given that they are clearly trial stakeholders and we don't do this for other stakeholders such as the statistician and the trial manager. Other people definitely do think it is necessary to quantify the gain from PPI. It would be good if the authors could mention this somewhere in the article. It could be early as a rationale for doing the study, or the Discussion in recognition of the different views.
Firstly, thank you for your comprehensive comments.
We agree this is a potentially contentious area. The following text has been added to the We agree this is a potentially contentious area. The following text has been added to the introduction of the manuscript as a rationale for doing the study.
"Research funders increasingly expect that PPI is prioritised and resourced within studies. This increasing expectation has heightened the risk of researchers carrying out 'tick-box' PPI rather than 'meaningful' involvement (6). There are many moral and ethical arguments being made for PPI. Many believe that as citizens and taxpayers, members of the public have a right to influence research that is being funded by public money (7). PPI researchers are also making pragmatic arguments for PPI and providing anecdotal accounts about how PPI can make research more relevant, accessible and acceptable to participants (8). The ethical arguments are often seen as sufficient regardless of any pragmatic impact. However, PPI costs time and money, therefore pragmatic claims need scrutiny (9). More substantive evidence is needed to evaluate the potential impact of PPI on the conduct and outcomes of research (5, 7)". (Introduction, Paragraph 1)

Identifying the intervention results as 'patient-preferred method' is at times confusing (it is easily misunderstood as the mode of delivery). Alternatives could be 'patient-based approach', 'patient-generated materials' or 'patient-centred approach'.
We have made the following changes throughout the manuscript: "patient-preferred" has been changed to "patient-based". "patient-preferred method" has been changed to "patient-based approach". "Standard method" has been changed to "standard approach".

Abstract
Background -rather than referring to 'an increasing consensus' the 'increasingly recognised as an essential component' text mentioned in the Introduction in the main text seems easier to agree with.
There may be a slight misunderstanding here. The 'increasing consensus' phrase in the abstract refers to the sharing of results with participants. The 'increasingly recognised as an essential component' phrase in the introduction refers to PPI in clinical research. These phrases are referring to two different things. We believe it would be confusing to the reader to use the same phrase to refer to two separate issues. Therefore, we have not made any changes to address this comment. We hope this is acceptable.

Results -it would be better rephrase so that it is clear the results are specific to the TRUST trial participants rather than in general. For example 'TRUST patients want to receive…'
This sentence in the abstract has been changed to: "TRUST patients want to receive…" (Abstract) Results -it is stated that "TRUST participants (n=101) were randomised to the intervention." Unless we have misunderstood something, this must be a mistake as this is the total available sample to be randomised so not all of them can get the intervention. Our guess is that 101 were involved in the SWAT evaluation and that 51 were randomised to the intervention.

to the intervention.
The sentence in the abstract has been corrected: "TRUST participants (n=101) were randomised to the either the intervention (n=51) or control group (n=50)." (Abstract) Conclusion -please rephrase "conduct PPI" to e.g. 'We have demonstrated that it is feasible to involve PPI members in the development of result dissemination materials.' We have changed this sentence in the abstract as suggested: "We have demonstrated that it is feasible to involve PPI partners in the development of result dissemination materials." (Abstract) We have also changed the same sentence in the main manuscript as following: "We have demonstrated that it is feasible to involve PPI partners in the development of result dissemination materials." (Conclusion, Paragraph 1) Conclusion -The reference to age in "The study identified and developed a patient-preferred method of receiving clinical trial results for older adults over 65 years." wasn't part of the aim of the study as far as we can see and the authors should reconsider including it in the conclusion. If developing a strategy for the over 65s was a distinct aim of the study, then the text of study aims needs tweaked.
The aim of the study has been changed to clarify that all TRUST participants were aged 65 and over: ""The aim of this Study Within A Trial (SWAT) is to use a patient and public involvement (PPI) approach to identify, develop and evaluate a patient-based approach to receiving trial results for participants in the Thyroid Hormone Replacement for Subclinical Hypo-Thyroidism Trial (TRUST), a trial of thyroxine versus placebo in people aged 65 years and older" (Abstract) The conclusion has been changed to reflect the aim: "The study identified, developed and evaluated a patient-based approach to receiving results for trial participants." (Abstract)

Conclusion -it would be good if the authors could see if they can improve ",… it provides a record of the process of conducting PPI within the clinical trial setting." We're not entirely clear what is meant.
We have made changes to this sentence in both the abstract and the main manuscript to cliarfiy the conclusion: "… it documents the process of conducting PPI within the clinical trial setting. This process may be useful for other trialists interested in conducting and evaluating the impact of PPI in clinical trials." (Abstract, Discussion, Paragraph 11 and Conclusion, Paragraph 1)

Main text
Methods -please ensure that the term used to denote the control/comparison group is consistent throughout the paper as currently these terms are used interchangeably (also in tables and figures).
To ensure consistency, the word 'comparison' has been changed to 'control' throughout the manuscript.
In the results section it is stated that the PPI group consisted of three trial participants and one older adult. It would be good to give a sentence in Methods on how this 'older adult' was identified and recruited.
The following sentence has been added to the manuscript to give more detail on how the older adult was identified: "In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This older adult was keen to learn more about research and expressed an interested in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner". (Methods: PPI group) The published protocol doesn't say that the materials will also reviewed by the National Adult Literacy Agency (NALA). That this is an addition should be mentioned, along with why. It's not a problem, just that it would be good to be clear.
We have inserted two additional sentences into the manuscript to clarify: Sentence 1: While the PPI group had significant input into the format and language used in the patient-based approach, the research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national "Plain English" standards. (Methods: Adult Literacy Review) Sentence 2: Although the review was taken as an additional step to the published protocol for the study, the research team felt it was helpful to further ensure that the document was accessible and easy to understand. (Methods: Adult Literacy Review) "TRUST participants (n=104)" it should be clear these are the Irish TRUST sample only. It "TRUST participants (n=104)" it should be clear these are the Irish TRUST sample only. It should also be noted that three participants were not eligible for the rest of the study as they took part in the PPI group.

Results
The first column heading has been changed to: "Total Irish TRUST participants (n=104)" (Results:

Please add an explanation of what RR is short for.
A footnote has been added to the table to clarify this. ` RR=Response Rate (Results: Table 1) "Randomised into intervention (n=101)", it should be clear n=101 were randomised (delete "into intervention"). See earlier comment about this. It's involved in SWAT evaluation that has n=101 as far as we can see, not randomised to intervention.
Yes, we agree that this needs to be clearer. We have deleted 'to intervention' as you have suggested.
Main text relating to figure 1 -please include the denominator in the parentheses as the percentages and n can be misleading on their own when small numbers are concerned.
The main text relating to figure has now been changed to include the denominator in the parentheses. Patient understanding of primary aim, side effect and trial results was assessed using multiple choice questions." (Results: Figure 1) Table 2 -in the title include that this is presented by group and please also indicate how this was assessed (e.g. Likert scale). The title for Table 2 has been changed in the manuscript to: " Table 2: Patient perceptions of understanding presented by group." (Results: Table 2) A footnote has been added to the title to clarify how perceptions of understandings were assessed. The footnote reads: "Patient perceptions of understanding were assessed using a five point LIKERT scale." (Results: Table 2) Any amendments which cannot be accommodated within the journal 15 word limit for the title of tables and figures could be added to a legend.
Thank you for this suggestion.

In relation to the cost -it should also be clear that the NALA review is included in the cost.
The following sentence has been added to make it clear that the NALA review is included in the cost: These costs included researcher salary, travel and expenses for PPI participants, adult literacy review and printing/postage costs. (Methods: Costs of conducting PPI)

Discussion
Paragraph 2 -please clarify the description of the intervention in relation to the literature as it currently confounds mode of delivery with only one type of material that can be posted to participants.
The following text has been added to this paragraph to clarify the description of the literature cited: "A previous study investigating the preferences of individuals taking part in a cardiac rehabilitation trial found that 80% of trial participants (mean age 68.5 years) preferred to receive the results by post (22)" (Discussion, Paragraph 3) Paragraph 3-please change 'negative reaction' to 'negative impact'-the reaction might be negative in researchers' view but completely justified. You should also comment on the trial context in relation to this as the TRUST trial is relatively low risk compared to some other trials cited.
We have changed the phrasing of 'negative reaction' to 'negative impact' and added the following sentence: "1 "Previous studies exploring participants' reactions found that sharing trial results with participants can cause some negative impacts such as anxiety, anger, guilt, upset and confusion (23-25). As far as researchers in this study are aware, providing results did not cause any negative impacts. This may have been due to the fact that the TRUST trial had a low risk of morbidity or mortality compared to some of the other studies citing negative impacts." (Discussion, Paragraph 4) Paragraph 4 relates to the aim of the SWAT and should probably be moved to a bit earlier in the Discussion, paragraph 2 perhaps.
Paragraph 4 has been moved to the first paragraph of the discussion.
Paragraph 5 and 6 (about cost) -The main message seems to be that PPI needs to be planned and costed appropriately -please be clear about this. We're not sure this sentence is necessary "As the primary focus of this study was to carry out PPI, researchers carefully considered the costs at grant proposal stage and wrote these costs into the budget." Yes this is the main message. We have deleted the sentence as suggested and added the following sentence to the paragraph: "It is extremely important that researchers plan PPI at the grant proposal stage and estimate the costs appropriately." (Discussion, Paragraph 5) The authors paid a fixed cost for travel costs for PPI participation rather than to cover participants' time. The UK's INVOLVE has recommendations for covering PPI participants' time and it might be nice to link these to the sentences that follow about the institutional difficulties of actually making these payments (which colleagues have experienced at our own institution).
We have edited the text in this paragraph to include a reference to INVOLVE's payment policy: "INVOLVE, the national advisory group supporting active public involvement in health services, public health and social care research in the UK, have recommended that PPI partners should be paid for their involvement (30). Despite this, existing research suggests that institutional difficulties make negotiating the mechanisms of paying participants very difficult (32)." (Discussion, Paragraph 6) Please include some consideration of the cost of carrying out the SWAT -is it a lot of money for a trial and is it money well spent? Please also consider that there may be some hidden costs e.g. expert input into questionnaire development an expert input in to the qualitative phase. This kind of expertise will not be available to all trials.
We have inserted the following text into the manuscript to address this: "This breakdown provides a template to other researchers who plan to carry out and evaluate PPI as part of their research. It is important to note that not all costs associated with carrying out the study were included in this amount. For example, the only salary costed was that of the research assistant. The expertise provided by other members of the study team were not included in the total cost as they were being paid by the University or other research grants. The total cost of total cost as they were being paid by the University or other research grants. The total cost of conducting this study was €8,049 which is not insignificant but should be considered in the context of the cost of large scale trials." (Discussion, Paragraph 6) Paragraph 7 (Limitations of the study) -The sample size of the SWAT was limited but to n=104 not n=115 (11 had withdrawn before the SWAT started so were never eligible).
We have changed the sample size to n=104. (Discussion: Strengths and Limitations, Paragraph 2) Paragraph 8 (Limitations of the study) -Change "Secondly, all trial participants were aged 65 and over." To 'Secondly, all SWAT participants were aged 65 and over.' While the former is true the SWAT did not include all trial participants.
We have changed the manuscript as suggested. (Discussion: Strengths and Limitations, Paragraph 3) Change "Alternative study populations may prefer…" to 'Other trial populations may prefer… ' We have changed the manuscript as suggested.
Paragraph 8 (Limitations of the study) -This paragraph actually highlights a strength of the SWAT which is that the PPI members consulted were participants in the trial. We'd suggest highlighting that many SWATs will be underpowered because they are generally unable to change the size of the host trial; they are made for meta-analysis really. This section of the discussion has been reworded from 'Limitations of the study' to 'Strengths and limitations of the study' (Discussion) This paragraph in question has been rephrased as a highlight of the SWAT and moved to the first paragraph in the 'Strengths and limitations of the study' section. It now reads as following: "While this study provides important insights into patients' preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that 'to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population' (3). All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed to long-term follow up. This is a strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial (31), thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention." Paragraph 10 -This is an important consideration. In relation to this it would be beneficial to include the research team's reflection on what is the most appropriate primary outcome to measure regarding dissemination of trial results. The use of PPI clearly had an impact in that the TRUST trial would not have posted results to Irish participants at all if it had not been for this SWAT (i.e. the Press release control for the SWAT was more than planned for the trial without the SWAT).
We agree that PPI had an important role to play in ensuring that participants actually received the We agree that PPI had an important role to play in ensuring that participants actually received the results of the trial and other primary outcome measures could have been used to evaluate the impact of PPI. If we had involved PPI partners from the outset of this study, relevant outcome measures could have been co-developed by researchers and PPI partners. We have added this important point to the discussion as following: The primary outcome of this study was assessing the impact of PPI on patient understanding of results, however, this was not the only potential impact. In hindsight, we adopted a limited approach to PPI in this study as we did not involve our PPI partners from the outset of the SWAT. Involving PPI partners in the design of the SWAT could have identified other appropriate primary outcome measures which would be more relevant to evaluating the impact of PPI in clinical trials "The intervention group (n=51) received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group (n=50) received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter)." (Results: Phase Two) We have added a discussion on the difference in content of the versions of the results: "It is also important to point out that the control group in this study received a copy of the trial results in a press release format. Most trial participants do not receive this. While this control method was a step further than normal procedure, the researchers in this study felt this was appropriate. The information presented in the press release was similar to that of the patient-based approach. However, the format and layout of the press release was different. Information was written in four long paragraphs separated by individual headings. It was also much shorter (1 page in total) than the patient-based approach (3 pages in total). Given the fact that press releases are written by public relations professionals with a view to communicating effectively and efficiently, this again may have potentially minimised differences between the intervention and control conditions." (Discussion: Strengths and Limitations, Paragraph 4)

Would the authors in hindsight consider asking their PPI members what they think the primary outcome in the evaluation of the results materials should have been?
Yes. This is a very important suggestion. We have inserted the following text into the manuscript: The primary outcome of this study was assessing the impact of PPI on patient understanding of results, however, this was not the only potential impact. In hindsight, we adopted a limited approach to PPI in this study as we did not involve our PPI partners from the outset of the SWAT. Involving PPI partners in the development of core outcome sets for this SWAT could have identified other more appropriate primary outcome measures (39). (Discussion: Strengths and Limitations, Paragraph 4) What would the authors recommend for future evaluations of PPI impact?

What would the authors recommend for future evaluations of PPI impact?
We have considered our recommendations for future evaluations of PPI impact and have inserted the following text into the manuscript: Evaluating the impact of PPI can be resource intensive, especially if PPI impact is not the primary aim of a study. As there is currently no gold standard or comprehensive guidelines for researchers to follow when evaluating the impact of PPI, further research is needed. This research should involve PPI partners in the development of core outcome sets for evaluating PPI impact. This would significantly enhance the literature in the area. (Discussion: Strengths and Limitations, Paragraph 5) Paragraph 11 -It would be good to expand on "While this study provides a record of the process of conducting PPI as part of a SWAT, future research is needed to further develop PPI in clinical trial settings." Any thoughts on what sort of PPI research?
We have added our thoughts on the sort of PPI research needed in the discussion: Evaluating the impact of PPI can be resource intensive, especially if PPI impact is not the primary aim of a study. As there is currently no gold standard or comprehensive guidelines for researchers to follow when evaluating the impact of PPI, further research is needed. This research should involve PPIpartners in the development of core outcome sets for evaluating PPI impact. This would significantly enhance the literature in the area. (Discussion: Strengths and Limitations, Paragraph 5) No competing interests were disclosed. Competing Interests: