Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol

Rikke Siersbaek; John Ford; Clíona Ní Cheallaigh; Sara Burke; Steve Thomas

doi:10.12688/hrbopenres.13154.2

Home Browse Making healthcare accessible for single adults with complex needs...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Revised

Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol

[version 2; peer review: 2 approved]

Previously titled: How accessible is healthcare for single adults experiencing long-term homelessness and complex needs? A realist evaluation protocol

Rikke Siersbaek ¹, John Ford², Clíona Ní Cheallaigh³, Sara Burke¹, Steve Thomas¹

Rikke Siersbaek ¹, John Ford², [...] Clíona Ní Cheallaigh³, Sara Burke¹, Steve Thomas¹

PUBLISHED 21 Jan 2021

Author details Author details

¹ Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Dublin, Ireland
² Institute of Public Health, University of Cambridge School of Clinical Medicine, Cambridge, CB2 0SR, UK
³ Department of Clinical Medicine, Trinity College Dublin, Dublin 8, Dublin, D08 NHY1, Ireland

Rikke Siersbaek
Roles: Conceptualization, Methodology, Project Administration, Software, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

John Ford
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Clíona Ní Cheallaigh
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Sara Burke
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Steve Thomas
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Over the last several years, homelessness has increased in Ireland and across Europe. Rates have recently declined since the coronavirus disease 2019 (COVID-19) pandemic, but it is unclear whether emergency housing measures will remain in place permanently. Populations experiencing long-term homelessness face a higher burden of multi-morbidity at an earlier age than housed populations and have poorer health outcomes. However, this population also has more difficulty accessing appropriate health services. A realist review by the authors found that important health system contexts which impact access are resourcing, training, funding cycles, health system fragmentation, health system goals, how care is organised, culture, leadership and flexibility of care delivery. Using a realist evaluation approach, this research will explore and refine key system-level factors, highlighted in our realist review, in a local health care system.
Aim: The aim of this study is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible to populations experiencing homelessness.
Methods: A realist evaluation will be undertaken to explain how funding and health system performance management impact healthcare accessibility for populations experiencing homelessness. Data will be collected using qualitative and realist interview techniques and focus group methodology. Secondary data such as policy documents and budgets will utilised. The analysis will follow Pawson and Tilley’s iterative phases starting with building an Initial programme theory, then data collection, data analysis, synthesis and finally building a refined programme theory.
Conclusion: Building on a realist review conducted by the same research team, this study will further test and refine findings that explain how health system factors impact healthcare accessibility for populations experiencing homelessness. The study has the potential to inform policy makers, health planners and managers of contextual factors that can be modified to increase healthcare accessibility.

Keywords

Homelessness, Homeless adult, Inclusion health, Access to healthcare, Realist evaluation

Corresponding author: Rikke Siersbaek

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board Ireland [SPHeRE/2013/1]
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2021 Siersbaek R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Siersbaek R, Ford J, Ní Cheallaigh C et al. Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol [version 2; peer review: 2 approved]. HRB Open Res 2021, 3:73 (https://doi.org/10.12688/hrbopenres.13154.2) First published: 02 Oct 2020, 3:73 (https://doi.org/10.12688/hrbopenres.13154.1) Latest published: 21 Jan 2021, 3:73 (https://doi.org/10.12688/hrbopenres.13154.2)

Revised Amendments from Version 1

We have responded to helpful reviewer comments to make the protocol clearer with regards to the justification for using realist evaluation and how this work builds on a previous realist review by the same authors. The title was also revised for clarity.

See the authors' detailed response to the review by Barbara Clyne
See the authors' detailed response to the review by Nigel Hewett

Introduction

Homelessness has been on the rise in Ireland and internationally over the last several years. From March 2015 to March 2020, the total number of people experiencing homelessness in the Republic of Ireland (including single adults and families with children) grew from a total of 4,261 people to 9,335 representing an increase of 119 percent¹. The total number of people being reported as homeless has declined since the outbreak of coronavirus disease 2019 (COVID-19) to 8,728 people in July 2020¹. However, it is unclear whether emergency housing measures will be in place permanently².

According to the European Typology of Homelessness and Housing Exclusion (ETHOS) homelessness occurs on a continuum including the following experiences:

Rooflessness (sleeping rough, without any shelter)
Houselessness (having somewhere to sleep but temporary in shelter or institution)
Living in insecure housing (insecure tenancies, threat of eviction, violence)
Living in inadequate housing (overcrowding, unfit housing, caravans on illegal campsites)³

The homelessness statistics in Ireland cover several different populations with somewhat differing needs, including single adults and families. The people included in the statistics are those relying on emergency accommodation and does not include people who are staying with family members, rough sleeping, camping, etc^1,4.

In addition to the physical aspects of homelessness there is an important temporal aspect. Researchers use three temporal categories: crisis homelessness, intermittent homelessness or chronic homelessness. Crisis homelessness occurs for people who experience sudden homelessness after an eviction, divorce or job loss. Intermittent homelessness occurs for people who move in and out of homelessness interrupted by access to adequate housing or institutional care stays. Chronic homelessness is defined as experiencing homelessness lasting more than a year continuously or for individuals with a disabling condition, four episodes of homelessness over a period of two years⁵.

The focus of this research is on the subset of the homeless populations who are single adults experiencing chronic homelessness. This population group experiences lengthy periods of homelessness while moving between any or all types of accommodation outlined in the ETHOS typology, sometimes also with time spent in institutional settings and/or in sufficient accommodation⁵. Such people typically have lived lives marked by trauma from a young age and often have complex health and social care needs^6,7. Many experience tri-morbidity, that is the simultaneous presence of physical and mental ill health and substance use disorders⁸. As a result, populations who experience homelessness and other forms of social exclusion have higher levels of morbidity than other populations and the onset of morbidity and multimorbidity happens at an earlier age^9–11. Homeless people also tend to die at a much younger age than their housed peers^10,11. In Dublin the median age of death for homeless people who die while accessing Dublin Region Homeless Executive supported services is 42 years old (44 years old for males and 37 years old for females)¹². In line with Tudor Hart’s inverse care law, this high-need population generally has more difficulty accessing healthcare than other populations, despite their poorer health and low life-expectancy^7,13–16.

The majority of research examining healthcare access for populations experiencing homelessness is focused on the patient journey from the perspective of the individual seeking to access health services^{13–15,17–21}. While it is important to understand the factors that make healthcare accessible for highly vulnerable population groups, there is a risk that this focus leads to inappropriately placing responsibility for accessing healthcare with populations experiencing homelessness and not with the health system. To remedy that imbalance, this research aims to understand access from a systems perspective.

Health systems are also characterised by a high level of complexity as they consist of a number interacting people, places, policies and actions as described in the WHO’s definition of a ‘health system’: ‘(i) all the activities whose primary purpose is to promote, restore and/or maintain health; (ii) the people, institutions and resources, arranged together in accordance with established policies, to improve the health of the population they serve, while responding to people’s legitimate expectations and protecting them against the cost of ill-health through a variety of activities whose primary intent is to improve health’²². In other words, a health system is an open system with many parts, at times interacting, and moving along non-linear pathways on diverse timelines. At the intersection with patients experiencing homelessness seeking to access healthcare services, health systems contribute to a number of intended and unintended outcomes which emerge as more than the sum of their parts (eg a patient may nominally access healthcare in the emergency department but leave before being seen even through physical access is possible due to a stigmatising culture which makes the environment unwelcoming)^23–27

Realist approaches (realist review and realist evaluation) are designed for understanding complex phenomena. The realist logic of inquiry is based on an understanding of reality as existing independent of our ability to observe it. It views the world as being one where we cannot observe or measure many of the processes that produce outcomes we are interested in²⁸. For example, in seeking healthcare access, a person experiencing homelessness may be met with stigma and feel unwelcomed and as a result decide not to seek care. In this case, we can measure the lack of realised healthcare access in low healthcare utilisation. However, we cannot independently observe and measure stigmatising attitudes nor can we measure the response arising in the person experiencing stigma. In the realist view of the world, we can theorise about what is happening to cause the outcome of poor healthcare access by understanding the social and psychological processes commonly at play in situations where health services prove accessible or inaccessible for populations experiencing homelessness.

In the school of Pawson and Tilley^29,30, realist evaluation seeks to understand the underlying mechanisms that generate a given outcome in a particular context using primary data. It is methods neutral and can employ data sources from a variety of study designs and methods. Realist approaches seek to uncover the conditions in which something works and for whom it works and understanding why rather than merely whether something works or not.

To add to the understanding of health system factors that improve access to healthcare for long-term homeless adults, we carried out a realist review³¹. This realist review produced a number of context-mechanism-outcome configurations (CMOCs) and an overarching programme theory (see Figure 1). This programme theory synthesised the full set of findings from the review showing the interlinking set of factors which must all be in place for health systems to successfully provide healthcare access to populations experiencing homelessness. The programme theory explains that important health system contexts which impact access are resourcing, training, funding cycles, health system fragmentation, health system goals, how care is organised, culture, leadership and flexibility of care delivery. Key mechanisms which arise in these contexts are provider attitudes, provider confidence, staff and provider experience of stability and sustainability, organisation-centred thinking, flexibility, transparency, timeliness, demonstration of respect and empathy, trust, adaptability, and anticipation.

Figure 1. Full programme theory of health system features that contribute to healthcare accessibility for homeless populations³¹.

Conversely, when those system features are partially in place or not in place at all, the patient experience of health services is impacted negatively. Eg when health services are fragmented, care cannot be delivered in a timely manner because each service or speciality operates on their own schedule and each step on a patient pathway depends on a referral from the last.

This study will focus on a subset of the findings of our realist review to evaluate how funding procedures and health system performance management impact service settings, staff, providers and their ability to make themselves accessible to populations experiencing homelessness. We have chosen to further investigate these two areas of study because they are under-researched and they will contribute to the work of planners and policy makers^32–34.

Study design

In this study, we will use realist evaluation, a theory-driven approach well-suited for analysing complex topics and interventions as detailed above.

Data collection will take place in Dublin with the goal of learning lessons particular to how health services make themselves accessible or not to populations experiencing homelessness. The study will be limited to Dublin region homeless healthcare services because the majority of homeless adults in Ireland live in Dublin³⁵ and the majority of healthcare services are provided in Dublin³⁶. Transferrable findings that will be useful internationally will be generated.

The study will employ documentary analysis of policy documents (Health Service Executive (HSE), Department of Health, non-government organisation (NGO) sector) and health service utilisation and budgeting statistics, as well as semi-structured and realist interviews with stakeholders in the health and NGO sectors. This study will also include focus groups with individuals with lived experience of homelessness who will aid in challenging, confirming and further refining study findings.

The study will follow the iterative realist evaluation design as set out by Pawson and Tilley³⁰ in the following phases (Figure 2):

1. Initial programme theory building
2. Data collection
3. Data analysis
4. Synthesis
5. Refined programme theory building

Figure 2. Stages of a realist evaluation.

The phases will repeat as needed. If it becomes clear that more data or different sources are needed, these will be incorporated into the synthesis in order to produce the most complete refined programme theory possible.

Initial programme theory

Theory building is the start and the end of realist work – the work starts with an initial programme theory and ends with a refined one after iterative rounds of data gathering and theory building.

Building on our realist review, the team has selected two context-mechanism-outcome configurations (CMOCs) to further explore and refine. The review indicates what is known about what works in promoting healthcare access in homeless populations from a health systems perspective – see the full programme theory above in Figure 1. Due to time and resource constraints it is not possible to further test the full set of six detailed CMOCs generated in the review.

For this study we will focus on two particular areas of the full programme theory explained in the two CMOCs described below. These relate to:

How health services are funded
How health systems manage performance

We have chosen to further investigate these two areas of study because they are under-researched and they will contribute to the work of planners and policy makers^32–34.

Building on the below CMOCs from our review, we will, where possible, further explain and add to the relationships between the contexts, mechanisms and outcomes identified in the international literature by expanding the analysis with the use of primary data and additional secondary data sources.

CMOC1 (Figure 3) suggests that when funding cycles are short, unreliable and come from multiple sources, eg grant funding to meet a specific need^34,37–39, services lack sustainability and stability^1,34,40–42 and as a result face difficulties hiring and retaining skilled and experienced staff members^34,40–43.

Figure 3. CMOC 1: Funding stability.

CMOC2 (Figure 4) suggests that when the parts of a health system operate in silos with narrowly defined goals^{34,38,39,44,45}, performance management is aligned with meeting those goals and they become the priority of staff even if they are not aligned with the needs of their patients^39,44,45. Performance management practices have tended to inhibit the ability of staff to deliver services in holistic, coordinated and flexible ways. As a result, healthcare is organised around the needs of providers and the system not the person^{15,38,39,44–46}.

Figure 4. CMOC 2 - Health system fragmentation and performance management.

Building on these CMOCs and the understanding of the underlying causative relationships they describe, the analysis will evaluate how funding procedures and health system performance management impact service settings, staff, providers and their ability to make themselves accessible to populations experiencing homelessness.

We will seek to answer the following questions:

How are health services for homeless populations funded?
For whom do funding arrangements work and not work, and why?
How is health system performance management intended to translate into service design and delivery for populations experiencing homelessness?
In what circumstances are health system performance management and organisational targets useful for creating healthcare accessibility for homeless populations and when are they not?
Who (patients, staff, providers, the system) does health system performance management work for, to what extent?

Data collection

Data will include grey literature and secondary sources identified through internet searching and snowballing as well as primary data collected via qualitative and realist interviews and focus groups. Realist interview/ focus group guides are available as extended data⁴⁷. Approximately ten key informant interviews will take place with professionals with significant experience in preventive, primary and secondary healthcare sectors and NGOs that provide services to homeless, and the HSE National Office for Social Inclusion, as well as relevant experts and academics. These will be identified by members of the research team which has broad networks in health and social services for homeless populations in Ireland. Snowball sampling will also be used to identify further interviewees. Interviews will take place via online video conferencing due to the current COVID-19 pandemic or in person, if and when possible. Interviews will be recorded and transcribed.

The research team will also conduct at least one focus group with experts by experience (individuals with lived experience of homelessness). An agreement to engage with individuals who are part of the Depaul peer health worker panel is in place but it remains to be seen if COVID-19 restrictions will allow us to proceed. It is likely that several focus groups conducted with subgroups within the target population would be useful. Women who have experienced homelessness may feel more comfortable participating in a women-only group and the same may be the case for LGBT or migrant people. If possible, given time and COVID-19 constraints, the research team intends to conduct more than one focus group. If holding focus groups is not possible, other methods will be considered. The goal of the focus groups is to add a service user’s perspective to the analysis of factors that impact health accessibility, particularly uncovering ones which may not be evident to individuals who have not personally had the experience of homelessness.

Data collection will be iterative and additional data sources may be collected as needed after the first data analysis phase to substantiate findings from interviews and focus groups. Eg if a particular policy or budget or procedure is discussed by a source, the research team will endeavour to access documentation regarding the item being discussed to add it to the analysis beyond the interviewee’s opinion.

Data analysis

All collected data will be transcribed and then inputted and analysed in NVivo software. For key informant interviews, a set of initial codes will be generated inductively while coding the first several interviews. The same codes will then be used across the rest of the interviews with more added as needed. Data collected from the focus group will likely have some similar codes and some different codes due to the difference in perspective and will be coded separately and inductively. Policy documents and grey literature sources will also be included in the analysis and will be coded separately from the primary data with a set of initial codes being developed inductively from what the team deem to be the likely most useful sources. Subsequent documents will be coded deductively using those codes while adding more as needed. Sources which may be identified over the course of the data collection from interviews and focus groups will be coded using the same procedure.

Synthesis

Each set of data (interviews, focus groups and secondary literature) will be analysed separately and will then be synthesised at the interpretation stage using triangulation⁴⁸ to formally explore the agreement and disagreement between sources and to identify how these contribute to the analysis. The goal is not to only find instances where there is agreement between sources but also to investigate areas of disagreement or where findings only occur in one set of sources. Eg service users may have a perspective on healthcare accessibility that is invisible to planners and providers of services but which is nonetheless important to understanding how services are and are not made accessible.

Following the data triangulation, CMOCs will be built using a combination of data sources. CMOCs will be iteratively refined and further developed as needed based on team discussions. A final programme theory will be constructed from CMOCs at the end of the project.

Ethics

Ethical approval has been granted by the Trinity College Dublin Health Policy and Management and Centre for Global Health Research Ethics Committee (application 22/2019/01).

Given that the population in question is a highly vulnerable one, the team is keenly aware of our responsibility in conducting this research ethically and sensitively. Trusted and known gatekeepers will be used to recruit focus group participants to provide every opportunity for potential participants to say no to participating for any reason. Participants will also be given a thorough oral explanation of the informed consent form along with the written copy of the document to ensure that they understand that participation is voluntary and that they may leave the study at any time. A professional who is known to the individuals in the peer health worker panel and who has significant experience working with the population, will take part in moderating the focus group in order to help provide a familiar and safe environment. Should a need for any support services arise as a result of participation in the focus group, Dr Cliona Ni Cheallaigh, a member of the research team and the clinical lead for the Inclusion Health Team in St. James’s Hospital Dublin will personally ensure that the individual or individuals are given the right care and treatment.

Dissemination of information

The main product of this research will be a research article which will be disseminated via a peer-reviewed journal. It will also be disseminated to research, policy making and practice networks, and through the Centre for Health Policy and Management, Trinity College Dublin website.

Study status

Documentary analysis and data collection via interviews has commenced. Three pilot interviews have been completed. Realist interviews are underway. Data analysis and synthesis is yet to be done.

Discussion

To our knowledge this is the first realist evaluation seeking to explain how funding procedures and health system performance management impact service settings, staff, providers and their ability to make themselves accessible to populations experiencing homelessness.

With the predominance in the literature of studies examining individual level factors that impact on healthcare access, there is a lack of understanding of the impact of system level decisions and priorities on downstream health service delivery for socially excluded populations. This study will add important knowledge to the field by specifically examining health system contexts and mechanisms.

A strength of this study is the use of a realist approach. The study will uncover modifiable contexts which impact upon healthcare accessibility and will be of interest to policy makers and healthcare planners and managers. A potential impact of the study is to inform health system and policy leaders, with the ultimate aim of making healthcare more accessible for populations experiencing homelessness.

Data availability

Underlying data

No underlying data are associated with this article

Extended data

Open Science Framework: Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol. https://doi.org/10.17605/OSF.IO/GCHFD⁴⁷

This project contains the following extended data:

Realist interview-FG guide_Rikke Siersbaek.docx (Interview and focus group guides)

Extended data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Faculty Opinions recommended

References

1. Focus Ireland: Latest Figures on Homelessness in Ireland. 2020. Reference Source
2. Focus Ireland: Homeless numbers drop for a fifth consecutive month to 8,699 as Focus Ireland calls on government to keep vulnerable renters safe in their homes. Focus Ireland. 2020 [cited 2020 Aug 25]. Reference Source
3. FEANTSA: What is ETHOS ? European Typology of Homelessness. 2017; 13–13. Reference Source
4. Long AE, Sheridan S, Gambi L, et al.: Family Homelessness in Dublin: Causes, Housing Histories, and Finding a Home Full Report. Focus Ireland; (Insights into family homelessness). Report No.: 2019; 2(1): 72. Reference Source
5. Fazel S, Geddes JR, Kushel M: The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet. 2014; 384(9953): 1529–40. PubMed Abstract | Publisher Full Text | Free Full Text
6. Hopper EK, Bassuk EL, Olivet J: Shelter from the Storm: Trauma-Informed Care in Homelessness Services Settings. Open Health Serv Policy J. 2010; 3: 80–100. Reference Source
7. Luchenski S, Maguire N, Aldridge RW, et al.: What works in inclusion health: overview of effective interventions for marginalised and excluded populations. Lancet. 2018; 391(10117): 266–80. PubMed Abstract | Publisher Full Text
8. Cornes M, Whiteford M, Manthorpe J, et al.: Improving hospital discharge arrangements for people who are homeless: A realist synthesis of the intermediate care literature. Health Soc Care Community. 2018; 26(3): e345–59. PubMed Abstract | Publisher Full Text
9. de Sousa E, O’Gara R, Power C, et al.: Premature Ageing in the Homeless Population. Dublin: Depaul; 2018. Reference Source
10. O’Reilly F, Barror S, Hannigan A, et al.: Homelessness: An unhealthy State Health Status, Risk Behaviours and Service Utilisation among Homeless People in Two Irish Cities. Partnership for Health Equity. 2015; 97. Reference Source
11. Queen AB, Lowrie R, Richardson J, et al.: Multimorbidity, disadvantage, and patient engagement within a specialist homeless health service in the UK: an in-depth study of general practice data. BJGP Open. 2017; 1(3): bjgpopen17X100941. PubMed Abstract | Publisher Full Text | Free Full Text
12. Ivers JH, Barry J: Mortality amongst the homeless populatin in Dublin. Dublin: Department of Public Health and Primary Care. Institute of Population Health, School of Medicine, Trinity College Dublin; 2018. Reference Source
13. Campbell DJT, O’Neill BG, Gibson K, et al.: Primary healthcare needs and barriers to care among Calgary’s homeless populations. BMC Fam Pract. 2015; 16(1): 139. PubMed Abstract | Publisher Full Text | Free Full Text
14. O’Carroll A, Wainwright D: Making sense of street chaos: an ethnographic exploration of homeless people’s health service utilization. Int J Equity Health. 2019; 18(1): 113. PubMed Abstract | Publisher Full Text | Free Full Text
15. O’Donnell P, Tierney E, O’Carroll A, et al.: Exploring levers and barriers to accessing primary care for marginalised groups and identifying their priorities for primary care provision: a participatory learning and action research study. Int J Equity Health. 2016; 15(1): 197. PubMed Abstract | Publisher Full Text | Free Full Text
16. Hart JT: THE INVERSE CARE LAW. Lancet. 1971; 1(7696): 405–12. PubMed Abstract | Publisher Full Text
17. Argintaru N, Chambers C, Gogosis E, et al.: A cross-sectional observational study of unmet health needs among homeless and vulnerably housed adults in three Canadian cities. BMC Public Health. 2013; 13(1): 577. PubMed Abstract | Publisher Full Text | Free Full Text
18. Baggett TP, O’Connell JJ, et al.: The Unmet Health Care Needs of Homeless Adults: A National Study. Am J Public Health. 2010; 100(7): 1326–33. PubMed Abstract | Publisher Full Text | Free Full Text
19. Booth ML, Bernard D, Quine S, et al.: Access to health care among Australian adolescents young people’s perspectives and their sociodemographic distribution. J Adolesc Health. 2004; 34(1): 97–103. PubMed Abstract | Publisher Full Text
20. Kertesz SG, McNeil W, Cash JJ, et al.: Unmet Need for Medical Care and Safety Net Accessibility among Birmingham’s Homeless. J Urban Health. 2014; 91(1): 33–45. PubMed Abstract | Publisher Full Text | Free Full Text
21. Salem BE, Nyamathi A, Idemundia F, et al.: At a Crossroads: Reentry Challenges and Healthcare Needs Among Homeless Female Ex-Offenders. J Forensic Nurs. 2013; 9(1): 14–22. PubMed Abstract | Publisher Full Text | Free Full Text
22. WHO: Health Systems Strengthening Glossary. WHO. World Health Organization; [cited 2020 Nov 17]. Reference Source
23. Aday LA, Andersen R: A Framework for the Study of Access to Medical Care. Health Serv Res. 1974; 9(3): 208–220. PubMed Abstract | Free Full Text
24. Davidson PL, Andersen RM, Wyn R, et al.: A Framework for Evaluating Safety-Net and other Community-Level Factors on Access for Low-Income Populations. Inquiry. 2004; 41(1): 21–38. PubMed Abstract | Publisher Full Text
25. Dixon-Woods M, Cavers D, Agarwal S, et al.: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006; 6: 35. PubMed Abstract | Publisher Full Text | Free Full Text
26. Evans DB, Hsu J, Boerma T: Universal health coverage and universal access. Bull World Health Organ. 2013; 91(8): 546–546A. PubMed Abstract | Publisher Full Text | Free Full Text
27. Levesque J-F, Harris MF, Russell G: Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013; 12(1): 18. PubMed Abstract | Publisher Full Text | Free Full Text
28. Jagosh J: Retroductive theorizing in Pawson and Tilley’s applied scientific realism. J Crit Realism. 2020; 19(2): 121–30. Publisher Full Text
29. Pawson R: Evidence-based policy: a realist perspective. London ; Thousand Oaks, Calif: SAGE; 2006; 196. Reference Source
30. Pawson R, Tilley N: Realistic Evaluation. Sage Publications Inc; 1997. Reference Source
31. Siersbaek R, Ford J, Burke S, et al.: Contexts and mechanisms that promote access to healthcare for populations experiencing homelessness: a realist review. BMJ Open. Under review. Publisher Full Text
32. Crane M, Cetrano G, Joly L, et al.: Mapping of specialist primary health care services in England for people who are homeless. 2018; 105–105. Reference Source
33. Pleace N, Filipović Hrast M, Benjaminsen L, et al.: The Costs of Homelessness in Europe: An Assessment of the Current Evidence Base. Brussels: FEANTSA; (European Observatory on Homelessness Comparative Studies on Homelessness). 2013. Reference Source
34. The King’s Fund: Delivering health and care for people who sleep rough: Going above and beyond. London; 2020. Reference Source
35. Department of Housing, Planning & Local Government: Homelessness Report May 2020. Dublin; 2020. Reference Source
36. Department of the Environment, Community and Local Government:Independent Review of Homeless Services. 2015. Reference Source
37. The Faculty for Homeless and Inclusion Health: Homeless and Inclusion Health standards for commissioners and service providers. London; 2018; 64. Reference Source
38. Gill P, MacLeod U, Lester H, et al.: Improving access to health care for Gypsies and Travellers, homeless people and sex workers. 2013; 42. Reference Source
39. Page A, Hilbery O: Turning the tide: a vision paper for multiple needs and exclusions. London; 2011. Reference Source
40. Cortis N, Blaxland M: Workforce Issues in Specialist Homelessness Services. Social Policy Research Centre, UNSW Sydney, Sydney; 2017 [cited 2020 Aug 24]. Reference Source
41. Mahon E: More Than a Caring Personality: Factors Affecting Staff Retention in Non Profit Organisations in Ireland? Ir Soc Work. 2016; 8. Reference Source
42. The Queen’s Nursing Institute: Nursing care for people experiencing homelessness: A Survey of the QNI Homeless Health Network. London: The Queen’s Nursing Institute; 2018 [cited 2020 Sep 9]; 20. Reference Source
43. Focus Ireland: Recommendations to Government: Budget 2020. 2019. Reference Source
44. Cabinet Office Social Exclusion Task Force: Inclusion health: Improving the way we meet the primary health care needs of the socially excluded. London; 2010. Reference Source
45. Making Every Adult Matter Coalition: Solutions from the Frontline. London; 2015. Reference Source
46. Homeless Link and St Mungo’s: Improving hospital admission and discharge for people who are homeless. 2012. Reference Source
47. Siersbaek R: How accessible is healthcare for single adults experiencing long-term homelessness and complex needs? A realist evaluation protocol. 2020. http://www.doi.org/10.17605/OSF.IO/GCHFD
48. Farmer T, Robinson K, Elliott SJ, et al.: Developing and Implementing a Triangulation Protocol for Qualitative Health Research. Qual Health Res. 2006; 16(3): 377–94. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 02 Oct 2020

Author details Author details

Rikke Siersbaek
Roles: Conceptualization, Methodology, Project Administration, Software, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

John Ford
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Clíona Ní Cheallaigh
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Sara Burke
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Steve Thomas
Roles: Conceptualization, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board Ireland [SPHeRE/2013/1]
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 21 Jan 2021, 3:73

https://doi.org/10.12688/hrbopenres.13154.2

version 1

Published: 02 Oct 2020, 3:73

https://doi.org/10.12688/hrbopenres.13154.1

© 2021 Siersbaek R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

914

downloads

Citations

SEE MORE DETAILS

CITE

how to cite this article

Siersbaek R, Ford J, Ní Cheallaigh C et al. Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol [version 2; peer review: 2 approved] HRB Open Res 2021, 3:73 (https://doi.org/10.12688/hrbopenres.13154.2)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 21 Jan 2021

Revised

Views

Reviewer Report 08 Feb 2021

Nigel Hewett, Pathway Charity, London, UK

Approved

https://doi.org/10.21956/hrbopenres.14355.r28712

No ... Continue reading

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 25 Jan 2021

Barbara Clyne, Royal College of Surgeons in Ireland, Dublin, Ireland

Approved

https://doi.org/10.21956/hrbopenres.14355.r28713

The authors have addressed my comments. The research findings have great potential for practical ... Continue reading

CITE

Report a concern

Author Response 28 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

28 Jan 2021

Author Response

Thanks very much for your helpful feedback!
Competing Interests: No competing interests were disclosed.
Thanks very much for your helpful feedback!
Thanks very much for your helpful feedback!
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 28 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

28 Jan 2021

Author Response

Thanks very much for your helpful feedback!
Competing Interests: No competing interests were disclosed.
Thanks very much for your helpful feedback!
Thanks very much for your helpful feedback!
Competing Interests: No competing interests were disclosed. Close
Report a concern

Version 1

VERSION 1

PUBLISHED 02 Oct 2020

Views

Reviewer Report 24 Nov 2020

Barbara Clyne, Royal College of Surgeons in Ireland, Dublin, Ireland

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14273.r28276

This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.

The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:

Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4^th paragraph, the focus of this research is homeless populations, and 5^th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
Will you be using any reporting standards such as RAMESES II?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Partly
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Evidence synthsis, designing and conducting RCTs, process evaluations.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

21 Jan 2021

Author Response

(Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of ... Continue reading (Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.
The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:
Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4th paragraph, the focus of this research is homeless populations, and 5th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
Yes good point. Thank you for that suggestion. We have amended the title.

The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.
Thank you for pointing out the need to have more detail here. We have added more text to the document.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
Might you be referring to Figure 1 which is the full programme theory? We do briefly explain the full programme theory in the paragraph just before figure 1 but we have added more content to provide a fuller explanation.

The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
Thank you for this suggestion.

We have chosen not to describe each of the six CMOCs in the protocol because they are included in the full set of findings described in the full programme theory and because to do so it would repeat a lot of the realist review in which they are each fully explained and which is available as a preprint (https://www.researchsquare.com/article/rs-79236/v1) (and hopefully soon will be published!) We selected the two which, based on the existing literature, had the greatest potential for impact while also being under-researched and we felt that two provide the right balance of depth and breadth. We are not able to comprehensively build on each of the CMOCs from the review and the two were chosen as a starting point for the realist evaluation because as they are under-researched and policy relevant. It is important to note also that they are just a starting point which will guide the initial data collection and initial coding of data in the evaluation. The end product will move beyond them to produce a number of subsequent CMOCs which will be refined and combined to produce a set of CMOCs and a final programme theory for the realist evaluation.

For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Good point. Have made that change in the graphic and replaced and have updated text as well.

Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
The two CMOCs carried over will be a starting point for generating many more using the data collected for this study. New CMOCs will be developed to further investigate the topics explored in these two CMOCs in further detail. To this end, the data for the study, including outcomes data, will be generated from policy documents, interviews and focus groups. Additionally, we’ll use other data sources such as funding information from HSE Social Inclusion to understand how funding flows to service providers in the voluntary sector, as well as peer reviewed and grey literature which discuss the topics. Each CMOC will always draw on several data sources. At this stage, since the CMOCs have not been constructed we cannot include a table to demonstrate the data source for each of them as that will depend on the data collection to come.

For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
The interview technique is informed by a number of sources (eg (1,2)) as well as practical learning undertaken by the primary researcher in trainings provided by experts in the field (Geoff Wong and Justin Jagosh). As outlined in the RAMESE guidelines for realist interviews(1), theory testing in the interview setting is a process that takes place bit by bit and the researcher’s theories are typically not introduced in their full form nor formally as ‘here is my theory.’:

‘Interviews should start with general questions about interviewees’ role in, experiences of and views about the programme. Subsequent questions follow up their responses, asking them to tell their stories about specific experiences or issues with the programme, its participants, and constraints. Each of these questions investigates something about the programme theory being tested. Specific elements of programme theory can then be explicitly introduced and tested with the respondent. In order to avoid allegations of ‘leading the interview’, it is a good idea to test multiple, including contradictory, theories about the same aspect of the programme with the same respondent’. (1)

The realist interview process is often described as a ‘teacher-learner’ process whereby the interviewer presents theories under examination using normal (non-realist) language as the ‘teacher’ explaining some aspect of their findings to the interviewee. For example, in our study the researcher may ask ‘I have heard from other people I have spoken with (or I have read in the published literature) that funding cycles in health services from homeless populations are often short and unstable. Is that your experience? Could you tell me more about that?’ to begin to test CMOC1. The teacher and learner roles are not static – the interviewer moves fluidly back and forth. We may then, moving into learner mode, depending on what the interviewee answers, follow up asking about the effects of long or short or mixed funding cycles they’ve experienced in relation to staffing stability and their own experience of working for a given service. We might then ask about other aspects of the topic such as, for example explaining that in the literature we have read that that people working at the frontline are seen by their managers as naturally caring people who are rewarded by doing caring work and therefore are not always compensated fairly, and get their thoughts on that. And so on and so forth. (1,2)

In this manner, the causal relationship between mechanisms and outcomes is investigated by asking about specific aspects of the theory rather than by laying out the whole theory and asking for validation. This should diminish the risk of social desirability bias as the researcher does not present theories as ‘my work’ but rather as what they’ve been told by other interviewees or learned from the literature, and often will present contradictory findings and points of view for discussion with the interviewee.
Interviewees will be frontline medics, nurses and social workers and service managers with many years’ experience as well one individual from HSE Social Inclusion (programme management level). Others will likely be added per snowballing.

The researcher undertaking the interviews is a PhD student.

Will you be using any reporting standards such as RAMESES II?
We will not formally report on each of the aspects of the RAMESES II checklist but it will be used to guide and to continually check the quality of our work throughout the process.

References

1.            Greenhalgh T, Pawson G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J. The Realist Interview: The RAMESES II Project. 2017;1–3.
2.            Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016 Jul 1;22(3):342–60.
(Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.
The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:
Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4th paragraph, the focus of this research is homeless populations, and 5th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
Yes good point. Thank you for that suggestion. We have amended the title.

The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.
Thank you for pointing out the need to have more detail here. We have added more text to the document.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
Might you be referring to Figure 1 which is the full programme theory? We do briefly explain the full programme theory in the paragraph just before figure 1 but we have added more content to provide a fuller explanation.

The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
Thank you for this suggestion.

We have chosen not to describe each of the six CMOCs in the protocol because they are included in the full set of findings described in the full programme theory and because to do so it would repeat a lot of the realist review in which they are each fully explained and which is available as a preprint (https://www.researchsquare.com/article/rs-79236/v1) (and hopefully soon will be published!) We selected the two which, based on the existing literature, had the greatest potential for impact while also being under-researched and we felt that two provide the right balance of depth and breadth. We are not able to comprehensively build on each of the CMOCs from the review and the two were chosen as a starting point for the realist evaluation because as they are under-researched and policy relevant. It is important to note also that they are just a starting point which will guide the initial data collection and initial coding of data in the evaluation. The end product will move beyond them to produce a number of subsequent CMOCs which will be refined and combined to produce a set of CMOCs and a final programme theory for the realist evaluation.

For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Good point. Have made that change in the graphic and replaced and have updated text as well.

Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
The two CMOCs carried over will be a starting point for generating many more using the data collected for this study. New CMOCs will be developed to further investigate the topics explored in these two CMOCs in further detail. To this end, the data for the study, including outcomes data, will be generated from policy documents, interviews and focus groups. Additionally, we’ll use other data sources such as funding information from HSE Social Inclusion to understand how funding flows to service providers in the voluntary sector, as well as peer reviewed and grey literature which discuss the topics. Each CMOC will always draw on several data sources. At this stage, since the CMOCs have not been constructed we cannot include a table to demonstrate the data source for each of them as that will depend on the data collection to come.

For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
The interview technique is informed by a number of sources (eg (1,2)) as well as practical learning undertaken by the primary researcher in trainings provided by experts in the field (Geoff Wong and Justin Jagosh). As outlined in the RAMESE guidelines for realist interviews(1), theory testing in the interview setting is a process that takes place bit by bit and the researcher’s theories are typically not introduced in their full form nor formally as ‘here is my theory.’:

‘Interviews should start with general questions about interviewees’ role in, experiences of and views about the programme. Subsequent questions follow up their responses, asking them to tell their stories about specific experiences or issues with the programme, its participants, and constraints. Each of these questions investigates something about the programme theory being tested. Specific elements of programme theory can then be explicitly introduced and tested with the respondent. In order to avoid allegations of ‘leading the interview’, it is a good idea to test multiple, including contradictory, theories about the same aspect of the programme with the same respondent’. (1)

The realist interview process is often described as a ‘teacher-learner’ process whereby the interviewer presents theories under examination using normal (non-realist) language as the ‘teacher’ explaining some aspect of their findings to the interviewee. For example, in our study the researcher may ask ‘I have heard from other people I have spoken with (or I have read in the published literature) that funding cycles in health services from homeless populations are often short and unstable. Is that your experience? Could you tell me more about that?’ to begin to test CMOC1. The teacher and learner roles are not static – the interviewer moves fluidly back and forth. We may then, moving into learner mode, depending on what the interviewee answers, follow up asking about the effects of long or short or mixed funding cycles they’ve experienced in relation to staffing stability and their own experience of working for a given service. We might then ask about other aspects of the topic such as, for example explaining that in the literature we have read that that people working at the frontline are seen by their managers as naturally caring people who are rewarded by doing caring work and therefore are not always compensated fairly, and get their thoughts on that. And so on and so forth. (1,2)

In this manner, the causal relationship between mechanisms and outcomes is investigated by asking about specific aspects of the theory rather than by laying out the whole theory and asking for validation. This should diminish the risk of social desirability bias as the researcher does not present theories as ‘my work’ but rather as what they’ve been told by other interviewees or learned from the literature, and often will present contradictory findings and points of view for discussion with the interviewee.
Interviewees will be frontline medics, nurses and social workers and service managers with many years’ experience as well one individual from HSE Social Inclusion (programme management level). Others will likely be added per snowballing.

The researcher undertaking the interviews is a PhD student.

Will you be using any reporting standards such as RAMESES II?
We will not formally report on each of the aspects of the RAMESES II checklist but it will be used to guide and to continually check the quality of our work throughout the process.

References

1.            Greenhalgh T, Pawson G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J. The Realist Interview: The RAMESES II Project. 2017;1–3.
2.            Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016 Jul 1;22(3):342–60.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

21 Jan 2021

Author Response

(Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of ... Continue reading (Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.
The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:
Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4th paragraph, the focus of this research is homeless populations, and 5th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
Yes good point. Thank you for that suggestion. We have amended the title.

The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.
Thank you for pointing out the need to have more detail here. We have added more text to the document.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
Might you be referring to Figure 1 which is the full programme theory? We do briefly explain the full programme theory in the paragraph just before figure 1 but we have added more content to provide a fuller explanation.

The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
Thank you for this suggestion.

We have chosen not to describe each of the six CMOCs in the protocol because they are included in the full set of findings described in the full programme theory and because to do so it would repeat a lot of the realist review in which they are each fully explained and which is available as a preprint (https://www.researchsquare.com/article/rs-79236/v1) (and hopefully soon will be published!) We selected the two which, based on the existing literature, had the greatest potential for impact while also being under-researched and we felt that two provide the right balance of depth and breadth. We are not able to comprehensively build on each of the CMOCs from the review and the two were chosen as a starting point for the realist evaluation because as they are under-researched and policy relevant. It is important to note also that they are just a starting point which will guide the initial data collection and initial coding of data in the evaluation. The end product will move beyond them to produce a number of subsequent CMOCs which will be refined and combined to produce a set of CMOCs and a final programme theory for the realist evaluation.

For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Good point. Have made that change in the graphic and replaced and have updated text as well.

Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
The two CMOCs carried over will be a starting point for generating many more using the data collected for this study. New CMOCs will be developed to further investigate the topics explored in these two CMOCs in further detail. To this end, the data for the study, including outcomes data, will be generated from policy documents, interviews and focus groups. Additionally, we’ll use other data sources such as funding information from HSE Social Inclusion to understand how funding flows to service providers in the voluntary sector, as well as peer reviewed and grey literature which discuss the topics. Each CMOC will always draw on several data sources. At this stage, since the CMOCs have not been constructed we cannot include a table to demonstrate the data source for each of them as that will depend on the data collection to come.

For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
The interview technique is informed by a number of sources (eg (1,2)) as well as practical learning undertaken by the primary researcher in trainings provided by experts in the field (Geoff Wong and Justin Jagosh). As outlined in the RAMESE guidelines for realist interviews(1), theory testing in the interview setting is a process that takes place bit by bit and the researcher’s theories are typically not introduced in their full form nor formally as ‘here is my theory.’:

‘Interviews should start with general questions about interviewees’ role in, experiences of and views about the programme. Subsequent questions follow up their responses, asking them to tell their stories about specific experiences or issues with the programme, its participants, and constraints. Each of these questions investigates something about the programme theory being tested. Specific elements of programme theory can then be explicitly introduced and tested with the respondent. In order to avoid allegations of ‘leading the interview’, it is a good idea to test multiple, including contradictory, theories about the same aspect of the programme with the same respondent’. (1)

The realist interview process is often described as a ‘teacher-learner’ process whereby the interviewer presents theories under examination using normal (non-realist) language as the ‘teacher’ explaining some aspect of their findings to the interviewee. For example, in our study the researcher may ask ‘I have heard from other people I have spoken with (or I have read in the published literature) that funding cycles in health services from homeless populations are often short and unstable. Is that your experience? Could you tell me more about that?’ to begin to test CMOC1. The teacher and learner roles are not static – the interviewer moves fluidly back and forth. We may then, moving into learner mode, depending on what the interviewee answers, follow up asking about the effects of long or short or mixed funding cycles they’ve experienced in relation to staffing stability and their own experience of working for a given service. We might then ask about other aspects of the topic such as, for example explaining that in the literature we have read that that people working at the frontline are seen by their managers as naturally caring people who are rewarded by doing caring work and therefore are not always compensated fairly, and get their thoughts on that. And so on and so forth. (1,2)

In this manner, the causal relationship between mechanisms and outcomes is investigated by asking about specific aspects of the theory rather than by laying out the whole theory and asking for validation. This should diminish the risk of social desirability bias as the researcher does not present theories as ‘my work’ but rather as what they’ve been told by other interviewees or learned from the literature, and often will present contradictory findings and points of view for discussion with the interviewee.
Interviewees will be frontline medics, nurses and social workers and service managers with many years’ experience as well one individual from HSE Social Inclusion (programme management level). Others will likely be added per snowballing.

The researcher undertaking the interviews is a PhD student.

Will you be using any reporting standards such as RAMESES II?
We will not formally report on each of the aspects of the RAMESES II checklist but it will be used to guide and to continually check the quality of our work throughout the process.

References

1.            Greenhalgh T, Pawson G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J. The Realist Interview: The RAMESES II Project. 2017;1–3.
2.            Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016 Jul 1;22(3):342–60.
(Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.
The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:
Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4th paragraph, the focus of this research is homeless populations, and 5th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
Yes good point. Thank you for that suggestion. We have amended the title.

The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.
Thank you for pointing out the need to have more detail here. We have added more text to the document.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
Might you be referring to Figure 1 which is the full programme theory? We do briefly explain the full programme theory in the paragraph just before figure 1 but we have added more content to provide a fuller explanation.

The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
Thank you for this suggestion.

We have chosen not to describe each of the six CMOCs in the protocol because they are included in the full set of findings described in the full programme theory and because to do so it would repeat a lot of the realist review in which they are each fully explained and which is available as a preprint (https://www.researchsquare.com/article/rs-79236/v1) (and hopefully soon will be published!) We selected the two which, based on the existing literature, had the greatest potential for impact while also being under-researched and we felt that two provide the right balance of depth and breadth. We are not able to comprehensively build on each of the CMOCs from the review and the two were chosen as a starting point for the realist evaluation because as they are under-researched and policy relevant. It is important to note also that they are just a starting point which will guide the initial data collection and initial coding of data in the evaluation. The end product will move beyond them to produce a number of subsequent CMOCs which will be refined and combined to produce a set of CMOCs and a final programme theory for the realist evaluation.

For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Good point. Have made that change in the graphic and replaced and have updated text as well.

Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
The two CMOCs carried over will be a starting point for generating many more using the data collected for this study. New CMOCs will be developed to further investigate the topics explored in these two CMOCs in further detail. To this end, the data for the study, including outcomes data, will be generated from policy documents, interviews and focus groups. Additionally, we’ll use other data sources such as funding information from HSE Social Inclusion to understand how funding flows to service providers in the voluntary sector, as well as peer reviewed and grey literature which discuss the topics. Each CMOC will always draw on several data sources. At this stage, since the CMOCs have not been constructed we cannot include a table to demonstrate the data source for each of them as that will depend on the data collection to come.

For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
The interview technique is informed by a number of sources (eg (1,2)) as well as practical learning undertaken by the primary researcher in trainings provided by experts in the field (Geoff Wong and Justin Jagosh). As outlined in the RAMESE guidelines for realist interviews(1), theory testing in the interview setting is a process that takes place bit by bit and the researcher’s theories are typically not introduced in their full form nor formally as ‘here is my theory.’:

‘Interviews should start with general questions about interviewees’ role in, experiences of and views about the programme. Subsequent questions follow up their responses, asking them to tell their stories about specific experiences or issues with the programme, its participants, and constraints. Each of these questions investigates something about the programme theory being tested. Specific elements of programme theory can then be explicitly introduced and tested with the respondent. In order to avoid allegations of ‘leading the interview’, it is a good idea to test multiple, including contradictory, theories about the same aspect of the programme with the same respondent’. (1)

The realist interview process is often described as a ‘teacher-learner’ process whereby the interviewer presents theories under examination using normal (non-realist) language as the ‘teacher’ explaining some aspect of their findings to the interviewee. For example, in our study the researcher may ask ‘I have heard from other people I have spoken with (or I have read in the published literature) that funding cycles in health services from homeless populations are often short and unstable. Is that your experience? Could you tell me more about that?’ to begin to test CMOC1. The teacher and learner roles are not static – the interviewer moves fluidly back and forth. We may then, moving into learner mode, depending on what the interviewee answers, follow up asking about the effects of long or short or mixed funding cycles they’ve experienced in relation to staffing stability and their own experience of working for a given service. We might then ask about other aspects of the topic such as, for example explaining that in the literature we have read that that people working at the frontline are seen by their managers as naturally caring people who are rewarded by doing caring work and therefore are not always compensated fairly, and get their thoughts on that. And so on and so forth. (1,2)

In this manner, the causal relationship between mechanisms and outcomes is investigated by asking about specific aspects of the theory rather than by laying out the whole theory and asking for validation. This should diminish the risk of social desirability bias as the researcher does not present theories as ‘my work’ but rather as what they’ve been told by other interviewees or learned from the literature, and often will present contradictory findings and points of view for discussion with the interviewee.
Interviewees will be frontline medics, nurses and social workers and service managers with many years’ experience as well one individual from HSE Social Inclusion (programme management level). Others will likely be added per snowballing.

The researcher undertaking the interviews is a PhD student.

Will you be using any reporting standards such as RAMESES II?
We will not formally report on each of the aspects of the RAMESES II checklist but it will be used to guide and to continually check the quality of our work throughout the process.

References

1.            Greenhalgh T, Pawson G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J. The Realist Interview: The RAMESES II Project. 2017;1–3.
2.            Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016 Jul 1;22(3):342–60.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 18 Nov 2020

Nigel Hewett, Pathway Charity, London, UK

Approved

https://doi.org/10.21956/hrbopenres.14273.r28256

This paper sets out a protocol for a realist evaluation approach to understanding the the impact of funding procedures and health system performance management on accessibility of services for people experiencing homelessness.

The protocol is clearly explained and the study design is practical and takes account of potential difficulties related to the COVID pandemic.

The realist evaluation approach is of increasing interest in the field of inclusion health, and this protocol looks likely to result in research findings that are of real practical use for future service planning and delivery.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Inclusion health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

21 Jan 2021

Author Response

Thank you very much for your feedback. Much appreciated!
Competing Interests: No competing interests were disclosed.
Thank you very much for your feedback. Much appreciated!
Thank you very much for your feedback. Much appreciated!
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

21 Jan 2021

Author Response

Thank you very much for your feedback. Much appreciated!
Competing Interests: No competing interests were disclosed.
Thank you very much for your feedback. Much appreciated!
Thank you very much for your feedback. Much appreciated!
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 02 Oct 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 21 Jan 21	read	read
Version 1 02 Oct 20	read	read

Nigel Hewett, Pathway Charity, London, UK
Barbara Clyne, Royal College of Surgeons in Ireland, Dublin, Ireland

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

15 Views

08 Feb 2021 | for Version 2

Nigel Hewett, Pathway Charity, London, UK

15 Views Cite this report Responses(0)

Approved

No new comments.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Inclusion health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

18 Views

25 Jan 2021 | for Version 2

Barbara Clyne, Royal College of Surgeons in Ireland, Dublin, Ireland

18 Views Cite this report Responses(1)

Approved

The authors have addressed my comments. The research findings have great potential for practical use in future service planning and delivery. I look forward to reading future papers.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Evidence synthsis, designing and conducting RCTs, process evaluations.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

28 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

Thanks very much for your helpful feedback!

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

30 Views

24 Nov 2020 | for Version 1

Barbara Clyne, Royal College of Surgeons in Ireland, Dublin, Ireland

30 Views Cite this report Responses(1)

Approved With Reservations

Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4^th paragraph, the focus of this research is homeless populations, and 5^th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
Will you be using any reporting standards such as RAMESES II?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Partly
Are the datasets clearly presented in a useable and accessible format?

Not applicable

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Evidence synthsis, designing and conducting RCTs, process evaluations.

Respond to this report

Responses (1)

Author Response

21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

(Comments in bold – answers in normal)
18 Dec 2020
This paper describes a protocol for a realist evaluation of funding procedures and health system performance management on accessibility of services for adults experiencing chronic homelessness. Research on this vulnerable population represents an important area of health services research and the findings could have important policy impacts. The research builds on a previous realist review by the study team, demonstrating their expertise in this area.
The protocol is clear, well written and the overall study is well described. However, there are a number of clarifications that I feel would improve the manuscript overall. These are as follows:
Title: The title and the study aim don’t seem to align to me. The title “How accessible is healthcare for single adults experiencing long-term homelessness and complex needs?” implies that this study is about accessing healthcare from the perspective of adults experiencing homelessness. But the study aim is to understand how funding procedures and health system performance management impact service settings, staff, providers and their ability to make services accessible. So providers of homeless services and the system are really the focus of the study. If providers can make the service available, is different to whether or not adults experiencing homelessness then access it. I think this is especially the case for the focus group with adults who previously experienced homelessness who may or may not be part of the study. There is also a little contradiction within the introduction, 4th paragraph, the focus of this research is homeless populations, and 5th paragraph, aim to understand system perspective. Suggestion: making healthcare accessible for homeless adults....
Yes good point. Thank you for that suggestion. We have amended the title.

The introduction gives a good justification as to why realist evaluation is suited to exploring barriers to health care access from the perspective of homeless populations, but as above, the focus of this study is the system so a sentence or two justifying why this approach is suitable would be good.
Thank you for pointing out the need to have more detail here. We have added more text to the document.

The research builds on a previous realist review by the study team. Figure 2 outlining the full programme theory is a nice visual, but I think a full statement of what the programme theory is would be beneficial.
Might you be referring to Figure 1 which is the full programme theory? We do briefly explain the full programme theory in the paragraph just before figure 1 but we have added more content to provide a fuller explanation.

The protocol is focused on 2 of the 6 elements of the overall theory from the realist review. Could the authors provide brief info on the other 4 elements and a more detailed description of why these 2 elements were selected over the other 4. It would also be helpful if the authors reflected on how the focus on 2 elements only impacts the overall theory?
Thank you for this suggestion.

We have chosen not to describe each of the six CMOCs in the protocol because they are included in the full set of findings described in the full programme theory and because to do so it would repeat a lot of the realist review in which they are each fully explained and which is available as a preprint (https://www.researchsquare.com/article/rs-79236/v1) (and hopefully soon will be published!) We selected the two which, based on the existing literature, had the greatest potential for impact while also being under-researched and we felt that two provide the right balance of depth and breadth. We are not able to comprehensively build on each of the CMOCs from the review and the two were chosen as a starting point for the realist evaluation because as they are under-researched and policy relevant. It is important to note also that they are just a starting point which will guide the initial data collection and initial coding of data in the evaluation. The end product will move beyond them to produce a number of subsequent CMOCs which will be refined and combined to produce a set of CMOCs and a final programme theory for the realist evaluation.

For consistency, it would be helpful to describe CMOC 2 consistently with the rest of the manuscript in terms of calling it performance management, as opposed to goals and incentives. Or define what is meant by performance management earlier on.
Good point. Have made that change in the graphic and replaced and have updated text as well.

Realist evaluations tend to require good data on outcomes and where possible, data about outcomes should be triangulated (at least using different sources, if not different types, of information). In terms of your CMOCs, where is the outcome data coming from? In terms of data collection overall, I think it would be helpful to provide a table/figure demonstrating the data source for each of the CMOs to ensure you have all the adequate data.
The two CMOCs carried over will be a starting point for generating many more using the data collected for this study. New CMOCs will be developed to further investigate the topics explored in these two CMOCs in further detail. To this end, the data for the study, including outcomes data, will be generated from policy documents, interviews and focus groups. Additionally, we’ll use other data sources such as funding information from HSE Social Inclusion to understand how funding flows to service providers in the voluntary sector, as well as peer reviewed and grey literature which discuss the topics. Each CMOC will always draw on several data sources. At this stage, since the CMOCs have not been constructed we cannot include a table to demonstrate the data source for each of them as that will depend on the data collection to come.

For your realist interview approach, you will be interviewing key stakeholders. With this approach, they will be asked to contradict/refine your theory. Could you comment on how you might reduce social desirability bias and get the respondents to really challenge the researcher. How experienced in their area will the participants be and how senior/junior will the researcher be?
The interview technique is informed by a number of sources (eg (1,2)) as well as practical learning undertaken by the primary researcher in trainings provided by experts in the field (Geoff Wong and Justin Jagosh). As outlined in the RAMESE guidelines for realist interviews(1), theory testing in the interview setting is a process that takes place bit by bit and the researcher’s theories are typically not introduced in their full form nor formally as ‘here is my theory.’:

‘Interviews should start with general questions about interviewees’ role in, experiences of and views about the programme. Subsequent questions follow up their responses, asking them to tell their stories about specific experiences or issues with the programme, its participants, and constraints. Each of these questions investigates something about the programme theory being tested. Specific elements of programme theory can then be explicitly introduced and tested with the respondent. In order to avoid allegations of ‘leading the interview’, it is a good idea to test multiple, including contradictory, theories about the same aspect of the programme with the same respondent’. (1)

The realist interview process is often described as a ‘teacher-learner’ process whereby the interviewer presents theories under examination using normal (non-realist) language as the ‘teacher’ explaining some aspect of their findings to the interviewee. For example, in our study the researcher may ask ‘I have heard from other people I have spoken with (or I have read in the published literature) that funding cycles in health services from homeless populations are often short and unstable. Is that your experience? Could you tell me more about that?’ to begin to test CMOC1. The teacher and learner roles are not static – the interviewer moves fluidly back and forth. We may then, moving into learner mode, depending on what the interviewee answers, follow up asking about the effects of long or short or mixed funding cycles they’ve experienced in relation to staffing stability and their own experience of working for a given service. We might then ask about other aspects of the topic such as, for example explaining that in the literature we have read that that people working at the frontline are seen by their managers as naturally caring people who are rewarded by doing caring work and therefore are not always compensated fairly, and get their thoughts on that. And so on and so forth. (1,2)

In this manner, the causal relationship between mechanisms and outcomes is investigated by asking about specific aspects of the theory rather than by laying out the whole theory and asking for validation. This should diminish the risk of social desirability bias as the researcher does not present theories as ‘my work’ but rather as what they’ve been told by other interviewees or learned from the literature, and often will present contradictory findings and points of view for discussion with the interviewee.
Interviewees will be frontline medics, nurses and social workers and service managers with many years’ experience as well one individual from HSE Social Inclusion (programme management level). Others will likely be added per snowballing.

The researcher undertaking the interviews is a PhD student.

Will you be using any reporting standards such as RAMESES II?
We will not formally report on each of the aspects of the RAMESES II checklist but it will be used to guide and to continually check the quality of our work throughout the process.

References

1. Greenhalgh T, Pawson G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J. The Realist Interview: The RAMESES II Project. 2017;1–3.
2. Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016 Jul 1;22(3):342–60.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

28 Views

18 Nov 2020 | for Version 1

Nigel Hewett, Pathway Charity, London, UK

28 Views Cite this report Responses(1)

Approved

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Inclusion health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

21 Jan 2021

Rikke Siersbaek, Centre for Health Policy and Management, Trinity College Dublin, Dublin 2, Ireland

Thank you very much for your feedback. Much appreciated!

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Focus Ireland: Latest Figures on Homelessness in Ireland. 2020. Reference Source

[2] 2. Focus Ireland: Homeless numbers drop for a fifth consecutive month to 8,699 as Focus Ireland calls on government to keep vulnerable renters safe in their homes. Focus Ireland. 2020 [cited 2020 Aug 25]. Reference Source

[3] 3. FEANTSA: What is ETHOS ? European Typology of Homelessness. 2017; 13–13. Reference Source

[4] 4. Long AE, Sheridan S, Gambi L, et al.: Family Homelessness in Dublin: Causes, Housing Histories, and Finding a Home Full Report. Focus Ireland; (Insights into family homelessness). Report No.: 2019; 2(1): 72. Reference Source

[5] 5. Fazel S, Geddes JR, Kushel M: The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet. 2014; 384(9953): 1529–40. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Hopper EK, Bassuk EL, Olivet J: Shelter from the Storm: Trauma-Informed Care in Homelessness Services Settings. Open Health Serv Policy J. 2010; 3: 80–100. Reference Source

[7] 7. Luchenski S, Maguire N, Aldridge RW, et al.: What works in inclusion health: overview of effective interventions for marginalised and excluded populations. Lancet. 2018; 391(10117): 266–80. PubMed Abstract | Publisher Full Text

[8] 8. Cornes M, Whiteford M, Manthorpe J, et al.: Improving hospital discharge arrangements for people who are homeless: A realist synthesis of the intermediate care literature. Health Soc Care Community. 2018; 26(3): e345–59. PubMed Abstract | Publisher Full Text

[9] 9. de Sousa E, O’Gara R, Power C, et al.: Premature Ageing in the Homeless Population. Dublin: Depaul; 2018. Reference Source

[10] 10. O’Reilly F, Barror S, Hannigan A, et al.: Homelessness: An unhealthy State Health Status, Risk Behaviours and Service Utilisation among Homeless People in Two Irish Cities. Partnership for Health Equity. 2015; 97. Reference Source

[11] 11. Queen AB, Lowrie R, Richardson J, et al.: Multimorbidity, disadvantage, and patient engagement within a specialist homeless health service in the UK: an in-depth study of general practice data. BJGP Open. 2017; 1(3): bjgpopen17X100941. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Ivers JH, Barry J: Mortality amongst the homeless populatin in Dublin. Dublin: Department of Public Health and Primary Care. Institute of Population Health, School of Medicine, Trinity College Dublin; 2018. Reference Source

[13] 13. Campbell DJT, O’Neill BG, Gibson K, et al.: Primary healthcare needs and barriers to care among Calgary’s homeless populations. BMC Fam Pract. 2015; 16(1): 139. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. O’Carroll A, Wainwright D: Making sense of street chaos: an ethnographic exploration of homeless people’s health service utilization. Int J Equity Health. 2019; 18(1): 113. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. O’Donnell P, Tierney E, O’Carroll A, et al.: Exploring levers and barriers to accessing primary care for marginalised groups and identifying their priorities for primary care provision: a participatory learning and action research study. Int J Equity Health. 2016; 15(1): 197. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Hart JT: THE INVERSE CARE LAW. Lancet. 1971; 1(7696): 405–12. PubMed Abstract | Publisher Full Text

[17] 17. Argintaru N, Chambers C, Gogosis E, et al.: A cross-sectional observational study of unmet health needs among homeless and vulnerably housed adults in three Canadian cities. BMC Public Health. 2013; 13(1): 577. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Baggett TP, O’Connell JJ, et al.: The Unmet Health Care Needs of Homeless Adults: A National Study. Am J Public Health. 2010; 100(7): 1326–33. PubMed Abstract | Publisher Full Text | Free Full Text

[19] 19. Booth ML, Bernard D, Quine S, et al.: Access to health care among Australian adolescents young people’s perspectives and their sociodemographic distribution. J Adolesc Health. 2004; 34(1): 97–103. PubMed Abstract | Publisher Full Text

[20] 20. Kertesz SG, McNeil W, Cash JJ, et al.: Unmet Need for Medical Care and Safety Net Accessibility among Birmingham’s Homeless. J Urban Health. 2014; 91(1): 33–45. PubMed Abstract | Publisher Full Text | Free Full Text

[21] 21. Salem BE, Nyamathi A, Idemundia F, et al.: At a Crossroads: Reentry Challenges and Healthcare Needs Among Homeless Female Ex-Offenders. J Forensic Nurs. 2013; 9(1): 14–22. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. WHO: Health Systems Strengthening Glossary. WHO. World Health Organization; [cited 2020 Nov 17]. Reference Source

[23] 23. Aday LA, Andersen R: A Framework for the Study of Access to Medical Care. Health Serv Res. 1974; 9(3): 208–220. PubMed Abstract | Free Full Text

[24] 24. Davidson PL, Andersen RM, Wyn R, et al.: A Framework for Evaluating Safety-Net and other Community-Level Factors on Access for Low-Income Populations. Inquiry. 2004; 41(1): 21–38. PubMed Abstract | Publisher Full Text

[25] 25. Dixon-Woods M, Cavers D, Agarwal S, et al.: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006; 6: 35. PubMed Abstract | Publisher Full Text | Free Full Text

[26] 26. Evans DB, Hsu J, Boerma T: Universal health coverage and universal access. Bull World Health Organ. 2013; 91(8): 546–546A. PubMed Abstract | Publisher Full Text | Free Full Text

[27] 27. Levesque J-F, Harris MF, Russell G: Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013; 12(1): 18. PubMed Abstract | Publisher Full Text | Free Full Text

[28] 28. Jagosh J: Retroductive theorizing in Pawson and Tilley’s applied scientific realism. J Crit Realism. 2020; 19(2): 121–30. Publisher Full Text

[29] 29. Pawson R: Evidence-based policy: a realist perspective. London ; Thousand Oaks, Calif: SAGE; 2006; 196. Reference Source

[30] 30. Pawson R, Tilley N: Realistic Evaluation. Sage Publications Inc; 1997. Reference Source

[31] 31. Siersbaek R, Ford J, Burke S, et al.: Contexts and mechanisms that promote access to healthcare for populations experiencing homelessness: a realist review. BMJ Open. Under review. Publisher Full Text

[32] 32. Crane M, Cetrano G, Joly L, et al.: Mapping of specialist primary health care services in England for people who are homeless. 2018; 105–105. Reference Source

[33] 33. Pleace N, Filipović Hrast M, Benjaminsen L, et al.: The Costs of Homelessness in Europe: An Assessment of the Current Evidence Base. Brussels: FEANTSA; (European Observatory on Homelessness Comparative Studies on Homelessness). 2013. Reference Source

[34] 34. The King’s Fund: Delivering health and care for people who sleep rough: Going above and beyond. London; 2020. Reference Source

[35] 35. Department of Housing, Planning & Local Government: Homelessness Report May 2020. Dublin; 2020. Reference Source

[36] 36. Department of the Environment, Community and Local Government:Independent Review of Homeless Services. 2015. Reference Source

[37] 37. The Faculty for Homeless and Inclusion Health: Homeless and Inclusion Health standards for commissioners and service providers. London; 2018; 64. Reference Source

[38] 38. Gill P, MacLeod U, Lester H, et al.: Improving access to health care for Gypsies and Travellers, homeless people and sex workers. 2013; 42. Reference Source

[39] 39. Page A, Hilbery O: Turning the tide: a vision paper for multiple needs and exclusions. London; 2011. Reference Source

[40] 40. Cortis N, Blaxland M: Workforce Issues in Specialist Homelessness Services. Social Policy Research Centre, UNSW Sydney, Sydney; 2017 [cited 2020 Aug 24]. Reference Source

[41] 41. Mahon E: More Than a Caring Personality: Factors Affecting Staff Retention in Non Profit Organisations in Ireland? Ir Soc Work. 2016; 8. Reference Source

[42] 42. The Queen’s Nursing Institute: Nursing care for people experiencing homelessness: A Survey of the QNI Homeless Health Network. London: The Queen’s Nursing Institute; 2018 [cited 2020 Sep 9]; 20. Reference Source

[43] 43. Focus Ireland: Recommendations to Government: Budget 2020. 2019. Reference Source

[44] 44. Cabinet Office Social Exclusion Task Force: Inclusion health: Improving the way we meet the primary health care needs of the socially excluded. London; 2010. Reference Source

[45] 45. Making Every Adult Matter Coalition: Solutions from the Frontline. London; 2015. Reference Source

[46] 46. Homeless Link and St Mungo’s: Improving hospital admission and discharge for people who are homeless. 2012. Reference Source

[47] 47. Siersbaek R: How accessible is healthcare for single adults experiencing long-term homelessness and complex needs? A realist evaluation protocol. 2020. http://www.doi.org/10.17605/OSF.IO/GCHFD

[48] 48. Farmer T, Robinson K, Elliott SJ, et al.: Developing and Implementing a Triangulation Protocol for Qualitative Health Research. Qual Health Res. 2006; 16(3): 377–94. PubMed Abstract | Publisher Full Text

Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Figure 1. Full programme theory of health system features that contribute to healthcare accessibility for homeless populations³¹.

Study design

Figure 2. Stages of a realist evaluation.

Initial programme theory

Figure 3. CMOC 1: Funding stability.

Figure 4. CMOC 2 - Health system fragmentation and performance management.

Data collection

Data analysis

Synthesis

Ethics

Dissemination of information

Study status

Discussion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

Making healthcare accessible for single adults with complex needs experiencing long-term homelessness: A realist evaluation protocol

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Figure 1. Full programme theory of health system features that contribute to healthcare accessibility for homeless populations31.

Study design

Figure 2. Stages of a realist evaluation.

Initial programme theory

Figure 3. CMOC 1: Funding stability.

Figure 4. CMOC 2 - Health system fragmentation and performance management.

Data collection

Data analysis

Synthesis

Ethics

Dissemination of information

Study status

Discussion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!

Figure 1. Full programme theory of health system features that contribute to healthcare accessibility for homeless populations³¹.