Towards reproducible and respectful autism research: Combining open and participatory autism research practices

A B S T R A

The increased application of open research practices in the last few years has been changing the ways in which research is being conducted (Azevedo et al., 2019;Syed, 2019;Syed & Kathawalla, 2022), yet this application has been uneven across different research fields (Norris & O'Connor, 2019;Syed & Kathawalla, 2022).In particular, the field of neurodevelopmental research, within which autism falls, has been fairly slow in incorporating open research practices, though this has been urgently called for (e.g., Elsherif et al., 2022;Farran & Scerif,., 2021;Gourdon-Kanhukamwe et al., 2022;Kalandadze & Hart, 2022).Regarding autism research more specifically, there have been excellent examples of open autism research practices (e.g., Crompton et al., 2022;Manning et al., 2022;Wilson & Bishop, 2022), as well as very recent calls for autism research to adopt more open research practices (Hobson et al., 2021;Hobson et al., 2022;Yeung, 2022).However, autism research is arguably a field that could benefit from adopting further open research practices, in the hope that it leads to research becoming not just more rigorous, but also more useful and acceptable to the autistic and broader autism communities.

What is participatory autism research?
In the context of autism, participatory research means "incorporating the views of autistic people and their allies about what research gets done, how it is done and how it is implemented" (Fletcher-Watson et al., 2019, p. 1).There is no one 'right' way to engage in participatory research, with participatory practices needing to be tailored to each individual research study.Irrespective of the precise way in which participatory practices are undertaken, it is fundamental that there is an acknowledgement of the traditional power imbalances between autism researchers and the autistic community (and, to an extent, the broader autism communities too).This power imbalance has often been conceptualised within Arnstein's ladder of citizen participation (see Fig. 1).Within this framework, it has been noted that most autism research takes place at the bottom rungs of the ladder, whereby researchers hold most/ all of the power over community members: the research is done on, about or for participants, as opposed to with them.Autism researchers have sometimes engaged in activities moving towards the middle rungs of the ladder (e.g., informing community members about research, inviting consultation on research findings), but these are described as degrees of tokenism: there may appear to be some elements of community input, but the power still lies very much with academic researchers.A less common occurrence in autism research (despite some strong examples by specific researchers and research teams, e.g., Crane et al., 2019;Lilley et al., 2022;Nicolaidis et al., 2019;Stark et al., 2021) are research activities towards the top rungs of the ladder, which involve equitable power sharing between academic and community partners, or where community partners hold power over the research process.
Recent studies (e.g., den Houting, Higgins, Isaacs, Mahony, & Pellicano, 2021;Pickard et al., 2021) have revealed several perceived barriers to the incorporation of participatory principles within autism research.These barriers include confusion over the inherently flexible nature of participatory research (e.g., with researchers questioning whether their work 'counts' as participatory), challenges with working with diverse groups (e.g., with researchers unsure how to communicate effectively, especially with those who may have limited research experience), and concerns about researchers' capacity to undertake such practices (e.g., issues of time, resource, and funding constraints).Taken together, there seems to be a push towards more participatory practices in the autism research field, but the move in this direction is not a straightforward one.

Principles for open and participatory autism research
To date, there has been a tendency for open and participatory approaches to be considered as two separate frameworks.We suggest that bringing the two together can lead to more respectful, ethical, and scientifically robust research.Next, we outline three principles that we feel could support autism researchers to embed more fully the use of open and participatory practices within their work.We also consider some potential tensions between open and participatory research and suggest how these may be addressed.

Principle onethe need for adequate expertise and infrastructure to facilitate high quality research
Both open and participatory research require knowledge about the principles these practices are based on, and the myriad of specific techniques and approaches to execute them.Both frameworks are comprehensive, involving many different ways to conduct research (Cargo & Mercer, 2008;Kathawalla et al., 2021;Pickard et al., 2022).Therefore, a key principle in open and participatory autism research is ensuring that adequate expertise is developed, by providing opportunities for researchers to learn relevant research practices.
Researchers embracing open research practices need to know, for example, the how, what, and where of preregistering a study or submitting a preprint, and how to check which journals accept submissions that have been previously pre-printed (e.g., using Sherpa Romeo: https://v2.sherpa.ac.uk/romeo/).Also, knowledge of free and open software (such as R for statistics), and different repositories and platforms (such as Open Science Framework or GitHub) is necessary.The technical skills to use these tools are also essential.Increasingly, researchers are also expected to have a thorough understanding of the legalities of data sharing, including what data to share and where to host it.
Developing knowledge and expertise may be even more challenging in relation to participatory practices.In this regard, areas to consider include debates around terminology (e.g., Bottema-Beutel et al., 2021), debates around the conceptualisation of autism (e.g., Botha & Cage, 2022;den Houting, 2019) and debates around approaches and supports for autistic people (e.g., Chapman & Bovell, 2022); all of which can be areas of contention within the field.It is essential that autism researchers (especially those who do not identify as autistic) make efforts to educate themselves in these areas, instead of expecting autistic people to undertake the burden of educating non-autistic researchers on these issues (Botha, 2021).Such understanding undoubtedly requires immersion in autistic culture, yet Milton (2014) questions the extent to which this can be fully realised for non-autistic researchers.Milton also adds that even if it were possible, it is "certainly doubtful whether many established researchers have made the effort" (p.796).
Lack of knowledge appears to underpin a lack of confidence in enacting these practices.For example, researchers report that they often do not know where to start with open and participatory research practices, and lack of access to such information has been identified as a primary reason that researchers do not adopt them (Crüwell et al., 2019;Pickard et al., 2022;Washburn et al., 2018).Starting to apply these practices can be especially difficult for early career researchers (ECRs) who may not have enough confidence in, nor time and funding for, these practices (Allen & Mehler, 2019;Pickard et al., 2021).Further, ECRs might not receive help from senior researchers, either because they have not received training in this area themselves or they are reluctant to change their longstanding ways of conducting research (e.g., den Houting et al., 2021;Pickard et al., 2022).Notably, efforts are starting to be made to address this latter issue (e.g., the three-step guide of Kowalczyk et al., 2022, discussing how senior researchers can support open research practices).However, the onus for change does not, and should not, rest with individual researchers.
We argue that many barriers to applying open and participatory research practices should be addressed through top-down incentives promoting structural and cultural change (Hobson, Sedgewick, Manning, & Fletcher-Watson, 2022;Nuijten, 2019;Orben, 2019).These incentives might include addressing the often-significant time and financial implications associated with both open and participatory research practices (see for example Brett et al., 2014;Blackbourn et al., 2018;Finkel, Eastwick, & Reis, 2015;Reimer et al., 2019;Turoman et al., 2022).For open research, for example, costs can accrue at all stages of a given research programme.At the outset, researchers may need to factor in the time it takes to receive in-principle acceptance of a registered report 6 (i.e., the green light to start data collection); this adds time to the duration of the project.Then, at the data collection stage, meeting power requirements might mean more participants to pay.Finally, at the publishing stage, researchers without access to funds may not have the resources to pay for their work to be available via gold open access (see the related discussions in Else, 2021;Van Noorden, 2021).
While these increased infrastructure requirements for open research may be assumed, there has been little systematic research estimating the actual added financial and time costs of these processes.For example, it is not definitively known that registered reports take longer from study conception to final publication.Indeed, the final write up and publication stage is likely expedited, as researchers have already written a significant proportion of their academic paper and have already decided their data analytic plans.There are also innovations that allow researchers to apply open research principles to their work, at a cheaper cost (e.g., publishing accepted versions of academic papers on institutional repositories).
Like open research, participatory research is considered more resource-intensive than non-participatory research (Brett et al., 2014).Costs may come in the form of expenses, as autistic collaborators should be paid for their time and expertise (although, concerningly, a significant minority appear to receive no compensation at all; den Houting et al., 2021).Participatory practices also require more time and effort than projects that do not engage in these processes, particularly in relation to developing trusting relationships, which take time to build and establish.Further, community members should ideally be consulted at the genesis of a 6 registered reports are a form of publication in which researchers first submit a manuscript that proposes a project, and wait to collect their data after the journal has given in principle acceptance.The researchers then conduct their project and analyse their data in accordance with their registered protocol.
project, to help steer research questions to align with community priorities.However, such decisions often predate the awarding of research funding, meaning that embedding paid autistic involvement becomes challenging.Researchers may therefore need to be creative in terms of negotiating how to 'repay' autistic collaborators in the absence of funding.
As with open research, there are limitations as to what individual researchers can do to resource participatory practices, and systemic change is urgently needed.For instance, funders may lack an appreciation of the requirements of participatory methods, such as the need for initial funding to develop research ideas and subsequent bids.Yet take up of such practices are in funders' interests, as they should ensure that the resulting research better meets the needs of communities, and that it produces methods that are more acceptable to the community.
Until broader, more systematic change is enacted, researchers at all career stages can gain some advice and guidance on embedding open and participatory methods in their work through freely available resources (e.g., Allen & Mehler, 2019;Cook et al., 2021;Crüwell et al., 2019;den Houting et al., 2021;Farran, Silverstein, Ameen, Misheva, & Gilmore, 2020;Kathawalla et al., 2020;Kalandadze & Hart, 2022;Nicolaidis et al., 2019;Pellicano et al., 2017).Importantly, and in line with advice from most of these resources, we emphasise that there is 'no one size fits all' in terms of applying both open and participatory research practices.We recommend that autism researchers aim to start with steps that are most feasible to them at the specific stage of their open/participatory research journey (e.g., Corker, 2018).The 'buffet' metaphor (e.g., shorturl.at/cfhDW),originally developed by Christina Bergmann in relation to open research, is of relevance here: decisions on which practice to use will depend on the specific research project, as well as the researchers' expertise in open (or participatory) research practices, as well as their available resources and timescales.

Second principlethe need for a greater degree of accessibility at all stages of the research process
Our second principle concerns accessibility.Accessibility cuts across multiple stages of a research programme and interacts with both open and participatory research practices.However, the term "accessibility" has not been used to mean the same thing within open research and participatory research.One definition of accessibility concerns whether a person (be it a researcher or a member of the public) can get to a resource, such as a paper or a dataset, or whether a person can see a process taking place or how it was done, such as an analysis or the development of a hypothesis.For example, a person may ask, "Can I see the data this person is writing about myself?".We could think of this form of accessibility as simply "openness".Accessibility is not just about being able to download a dataset, or access a paper, however.It also concerns whether, even if a person can gain access to a resource or process, that resource or process can be understood; that is, "Can I make sense of this dataset as an outsider to the original research team?".We could think of this form of accessibility as "understandability".Accessibility in research also considers being able to be a part of the research process, which concerns practical and social barriers, such as issues of power.We could think of this form of accessibility as "inclusivity".All three are discussed here.
In relation to openness and open research, a central tenet of this approach is to make study information, including data, methodological details, materials and outputs publicly accessible for other researchers (Asendorpf et al., 2013;Munafò et al., 2017).Arguably, there are multiple benefits to this openness: for example, data sharing enables researchers to spot errors, to re-run analyses (or conduct additional analyses to support new research endeavours), and allows for the maximum use of carefully collected data, reducing "research waste" by permitting other teams to run additional analyses on existing datasets rather than having to collect new data.Such information is necessary for others to verify that study findings and conclusions are based on robust research.
Making materials, particularly data, open requires careful consideration.Importantly, making study information physically available does not ensure accessibility in terms of the contentdata may be open, but not understandable.If there is agreement that data can be made suitably open, meta-information around data needs to be such that others can understand the variables that have been collected within the dataset.Otherwise, datasets risk being open but impenetrable to those who wish to use them.
Understandability applies not only to research data and material but also plays a key role in our participant-facing documents, such as information sheets, consent forms, and debrief forms, as well as instructions during data collection.When such documents are not written in an accessible manner, research risks losing its inclusivity.Indeed, while informed consent is a key aspect of ethical research, consent is not informed if it is given in response to information sheets and consent forms that are unclear (Natri, 2021), for example due to the use of complex legal terms and vocabulary that is largely unfamiliar to people outside of academic research.Meeting the demands of one's institution to include specific phrases and terms, whilst ensuring documents are accessible for participants, is an ongoing challenge.Discussion of such issues in relation to autism research specifically can be found in Nicolaidis et al. (2011), who note that their autistic partners have found standard consent forms inaccessible, and they have worked with autistic collaborators to redesign consent processes to make them accessible and respectful to autistic people.Indeed, Nicolaidis et al. (2011) noted examples where potential research partners were discouraged from involvement in projects because the terminology around what participatory frameworks and approaches would be adopted were too confusing.These issues may be particularly challenging for autistic people with co-occurring intellectual disability, for whom consent procedures may need to be even more accessible.If this issue is not addressed, only a subset of autistic participants will be able to take part in autism research, which will lead to bias in study results (see Russell et al., 2019, for discussions on selection bias in autism research).These issues of understandability and inclusion feed back into issues around open research practices, particularly around the issue of data sharing.For data to be made open, researchers must have the consent of participants to share their data this way.However, researchers will then need to explain the process of open data and data sharing, in an accessible and inclusive way to autistic people and their families.There is limited guidance available on how to do this currently.
Accessibility is also about inclusion.Accessibility should be considered from the perspectives of power balances and perspectives within the team: if an autistic partner is on the research team, are they empowered to voice their views and bring their expertise to the H. Hobson et al. table, and are there accessible ways for them to do so?There are indications that perceptions of engagement differ between researchers and community members, as the latter may feel that the research process itself is inaccessible and they are not able to influence this in meaningful ways (den Houting et al., 2021;Pellicano et al., 2014).As a result of such concerns, guidelines have been produced to aid researchers in conducting research with the autistic and broader autism communities (see Gowen et al., 2019).
In addition, it is important to consider a very physical form of accessibility: for example, is research being conducted in an environment that is accessible for autistic participants and research partners?To our knowledge, there is no systematic research on how lab environments might impact on the effects seen in research projects.Further, due to lack of reporting on such variables, such confounds are challenging to examine as a factor affecting research findings (Linden & Hönekopp, 2021).Yet it is reasonable to assume that adverse effects from sensory sensitivities, stimulated by lab environments, could interfere with what experimental effects are demonstrated.Indeed, issues of variability in findings due to differences in samples and settings have already been established during more general replication projects (e.g., Klein et al., 2018).
Finally, it is crucial that researchers reflect on issues of accessibility throughout the lifecycle of their research programmes.Different autistic people will have different accessibility requirements, and requirements will apply differently to different projects: thus, researchers cannot simply reapply what they did in their previous projects, but they will need to actively review and revise their processes for each study, to ensure maximum accessibility.Accessibility in the forms we have described should help to build trust, but making our research accessible, in terms of its openness, understandability and inclusivity, requires trust itself.

Third principle: fostering trusting relationships between the autism and research communities
For autism research to be truly open and participatory, members of the autistic and broader autism communities need to be able to trust autism researchers, which includes being able to trust researchers' intentions.For example, investigations into autistic people's concerns about autism research have highlighted that they have worries about "secret research", in which researchers are dishonest about the real purpose behind a research project, and how the study's findings will be used (Gowen, 2019).These concerns may extend to trust in the methods being undertaken (e.g., do I think this measure will be something I feel safe completing?), and may also extend to open research practices (e.g., data sharing).
The information that researchers feel is important to share may differ to that which is felt to be important to members of the autistic and broader autism communities.Further, members of the research and autistic/autism communities may have opposing views around which aspects of projects are important and ethical to share.As such, data sharing can be considered one example where open and participatory research may appear, at least ostensibly, to be incompatible.This situation may be somewhat nuanced, since Ashworth, Crane, Steward, Bovis, and Pellicano (2021) found that autistic people's acceptance of data sharing depends on the perceived trustworthiness of the researchers involved, with participants noting that they would not feel comfortable sharing their information with all autism researchers.
It is essential to reflect on ways to build trust in autism research, and the bringing together of open and participatory research practices may be particularly salient here.First, there is a need for critical reflection on what "open" means in open research, alongside meaningful involvement of the autistic and broader autism communities via participatory practices.For example, as described above, researchers need to clearly explain to potential participants why open research is needed and what it means for data to be "openly available".An agreement should be reached via input from autistic communities (i.e., participatory practices), addressing any concerns raised in this regard.Mitigations could include consideration of alternatives to fully open data, such as making data available on request, with some agreement reached between autistic representatives and researchers as to the circumstances under which data would be shared (e.g., depending on the aims of the researcher requesting access to the data).Working in a participatory way could therefore help to guide decision making around the use of open data in ways that work for both researchers and for the autistic and broader autism communities.The open research principle of transparency seems particularly pertinent here, since "explaining why sharing certain data is not feasible or ethical also aligns with transparent practice" (Knecht, Meer, Brinkman, Kluijtmans, & Miedema, 2021, p.5).
Second, developing trust in autism research may require listening to the concerns of autistic people by going beyond current data sharing practices and perhaps sharing other aspects of the study (see, for example, Natri, 2021).For example, it may be helpful to make documentation submitted to, and reviewed by, research ethics committees publicly available, since a simple statement of ethical approval is likely to be insufficiently accessible and transparent to lay people, and may provide little reassurance as to the ethical standards being adhered to.Publicly sharing ethics documentation would prevent the situation we see in the context of Spectrum 10K (2021), a large-scale autism genomics study, which caused considerable controversy (Natri, 2021).The concerns that autistic community raised in relation to Spectrum 10 K included a 'lack of transparency in recruitment and engagement, consent issues, the suitability of the principal and co-investigators, and the possibility that the data and results could be used towards eugenics' (p.3).More complete reporting of ethical information may help mitigate this issue and foster trust in the ethical integrity of the research (for a good practice example, see Pellicano et al., 2020).
Third, greater transparency about the researchers themselves could help to build trust.A key barrier in participatory research can be the actual and perceived power imbalance between autism researchers and autistic participants.One means of reducing this imbalance is via researchers sharing information about themselves, such as their approach to autism research and, if the researcher feels comfortable doing so, about their own neurology (i.e., whether they identify as autistic/neurodivergent).This openness can allow participants to feel more comfortable in the research setting with the researcher (for good practice example, see Crane et al., 2020).
In sum, participatory research practices should go some way towards fostering trusting relationships between researchers and community members, levelling the power imbalance, valuing both lived and research experience (Haas et al., 2016;Gowen, 2019), and allowing for trust to be built in open research practices.Equally, open research practices can foster trusting relationships, via promotion of the principles of transparency, openness, and accessibility (e.g., see Haven et al., 2022 for a discussion on how open research and research integrity are intertwined).More fundamentally, however, we suggest that it is an ethical obligation to meaningfully engage with the communities impacted by research, by conducting research in a community-engaged and respectful manner at all times, trusting the community with guiding our research activity.

Concluding reflections
In conclusion, we suggest that open and participatory research practices have much to offer the field of autism research.These two approaches may share common barriers, but they also share common principles, requirements, and opportunities.Bringing these approaches into closer alignment with one other could improve the robustness, reproducibility, and respectfulness of autism research.These approaches have the potential to ensure greater transparency around research, and greater involvement (and influence) of the autistic and broader autism communities in autism research.Further, these approaches can provide a framework for researchers to increase the quality, rigour, and ethics of their research.Such a change is urgently needed given recent controversies in the autism research field (e.g., Natri, 2021).
For change in our field to happen, both bottom-up and top-down approaches need to be enacted (see also Pellicano & den Houting, 2022).In this Orben (2019) notes that: We need all those who care about better research to stay invested, and this will not happen by telling the next generation of scientists to just sit back and hope.Early-career researchers do not need to wait passively for coveted improvements.We can create communities and push for bottom-up change.(p.465).This push from researchers must go hand in hand, however, with more systemic and structural changes.This could include support from autism research journals and funding bodies who support autism research, providing specific outlets (e.g.registered reports, publications solely concerned with community research priorities) and resources for researchers to apply participatory and open scholarship.Underpinning both bottom-up and top-down initiatives should be involvement and leadership from the autistic community.

Declaration of Competing Interest
None.