Introduction

According to 2011 World Health Organization (WHO) data, over a billion people (or about 15% of the world’s population) are estimated to be living with disability [1]. In Korea, the number of persons with disabilities (PWDs) registered nationwide is about 2.54 million people (or about 4.9% of the total population) [2]. The PWD population is expected to increase steadily owing to population aging, increased incidence of chronic diseases, various accidents, and disasters [3].

PWDs are more susceptible to diseases, and the incidence of chronic diseases in this group is high. Specifically, about 77% of Korean PWDs suffer from such chronic diseases as hypertension and diabetes, and their mortality rate is more than four times that of the total population [3]. However, health care services and support systems for PWDs in Korea remain insufficient, and the health disparity between people with and without disabilities persists because of insufficient health care management system [4,5].

Since the 1980s, the international community has begun recognizing the health disparity issue, acknowledging that health problems are not the responsibility of individuals and families but should be solved by society [6]. In line with these international trends, Korea enacted related laws, such as the Act on Welfare of Persons with Disabilities and Act on the Prohibition of Discrimination against Persons with Disabilities, Remedy against Infringement of their Rights. However, these laws only prescribe content related to the welfare promotion of PWDs and their participation in social activities, but not health and health care services [7]. In December 2015, the Act on Guarantee of Right to Health and Access to Medical Service for Persons with Disabilities (Act on Right to Health for Persons with Disabilities) was enacted to define legally the health care system for PWDs and the guarantee of access to medical care; the Act was implemented in December 2017.

According to Article 6, the main content of the Act on Right to Health for Persons with Disabilities, the Minister of Health and Welfare has to establish Comprehensive Plans for Health Care for Persons with Disabilities (Comprehensive Plan) every five years. Specifically, a comprehensive plan shall include the objectives and direction-setting for health care programs, a plan and methods for implementing health care programs, training of specialists, education, and health management according to gender characteristics [8]. However, the requirements are specified only at the item level, and the details of each are not comprehensive and systematic. For this Act to be an act for the actual promotion of the health of PWDs, rather than declarative content, specific goals and performance indicators for each detail must be presented in a comprehensive and systematic manner.

This study aimed to develop specific goals and performance indicators for each detail of the Comprehensive Plan, and then use them as base data to establish health care management policy for PWDs.

Methods

Sources to construct the indicators

Using existing policy indicators.

To construct the performance indicator, we analyzed the representative health policies currently implemented in Korea to improve people’s health and the performance indicator developed to evaluate the performance of the policy. Specifically, we analyzed the Comprehensive Plan for National Health Promotion (Health Plan 2020), Comprehensive Policy Plan for Persons with Disabilities, and others. Among them, applicability and available indicators were selected as the performance indicators of this study.

Health Plan 2020 is a comprehensive government mid- to long-term comprehensive plan that introduces the direction of the health promotion policy for health promotion and disease prevention of Korean people every five years. The Health Plan 2020 is composed of six fields, 27 tasks, and 367 performance indicators to achieve the goal of extending the life span of the Korean people and improving health equity [9]. Article 6 of Act on Right to Health for Persons with Disabilities requires that the content of the Health Plan 2020 must be included when establishing and implementing a Comprehensive Plan [8]. In other words, the Comprehensive Plan should be established in the same direction as the goal of the Health Plan 2020 because the latter is a key plan that is the backbone of Korean health policy. Therefore, among the performance indicators pursued by the Health Plan 2020, the indicators applicable to this study were selected as the performance indicators (Fig 1). The performance indicators of this study constructed using the performance indicators of the Health Plan 2020 are as follows: (1) establishment of statistics on the health of PWDs (nationally approved statistics), (2) proportion of public health centers, (3) beneficiary service rate for PWDs in residents, (4) current smoking rate, (5) smoking cessation attempts by current smokers, (6) annual high-risk drinking rate of current drinkers, (7) a month’s drinking rate, (8) physical activity practice rate, (9) regular meal rate, (10) rate of health checkup of PWDs, (11) prevalence of obesity, and (12) level of satisfaction in life.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Comprehensive plan for National Health Promotion (Health Plan 2020).

https://doi.org/10.1371/journal.pone.0208651.g001

The Comprehensive Policy Plan for Persons with Disabilities is a plan designed to promote the interests and welfare of PWDs, mainly designed to support welfare services rather than plans for the health promotion of PWDs [10]. Among the performance indicators promoted by the 4^th Comprehensive Policy Plan for Persons with Disabilities, this study used constructed “establishment of statistics on the health of PWDs (nationally approved statistics)” and “physical activity practice rate.”

Newly developed performance indicators.

The use of existing health policy performance indicators alone cannot provide key areas, specific goals, or specific performance indicators for each of the details of the Comprehensive Plan. To construct indicators that need to be newly developed, we first analyzed the details that should be included in the Comprehensive Plan. Article 6 of the Act on Right to Health for Persons with Disabilities should include information on the education and training of specialists required for the management of PWDs, gender characteristics, and management of health care for women with disabilities, such as maternity protection [8]. Therefore, this study constituted performance indicators related to training and education specialists, women with disabilities, and pediatric developmental disabilities.

The performance indicators of this study, constructed according to Article 6 of the Act on Right to Health for Persons with Disabilities, are as follows: (1) training and improvement of awareness of specialists in PWDs (regular curriculum, special curriculum, and medical personnel), (2) implementation of tracking inspection support, (3) financial support for early detection of developmental disabilities, (4) diagnosis of autism spectrum disorder in early childhood (average age and awareness), (5) regular screening rate during pregnancy, (6) infant mortality rate, (7) maternal mortality rate, (8) rate of breast cancer screening, (9) rate of cervical cancer screening, and (10) sexual education experience.

Through review of the literature, we analyzed the problems and risk factors in the field of health care currently faced by PWDs in Korea and the indicators that need to be further developed. PWDs in Korea have a higher prevalence of chronic diseases, such as hypertension, diabetes, dental caries, and periodontal diseases. In addition, depression and stress levels are high, and the rate of suicidal ideation is also exceedingly high owing to a complication of economic difficulties and pessimism [11–13]. Currently, national research data, such as the National Survey of the Disabled Persons and Korea National Health and Nutrition Examination Survey, identify the health status of PWDs in Korea. However, the National Survey of the Disabled Persons, conducted every three years, is not an actual measurement but a self-reporting survey. In addition, PWDs living in facilities are excluded from the survey. Indeed, various limitations that prevent the understanding of the actual health level of PWDs in Korea have been pointed out [11]. In the Korea National Health and Nutrition Examination Survey, a question on disability registration was added to the survey questionnaire for the sixth period (2013–2015), thereby making it possible to analyze the data according to disability status. However, the questionnaires were originally intended for the entire population, and not designed to assess the level of health of PWDs, setting a limit to understanding of the characteristics of disabilities [14]. Therefore, this study aimed to establish systematically the health-related data of PWDs to shed light on the level of health of PWDs by creating indicators that can reveal their level of chronic disease. The performance indicators developed are as follows: (1) prevalence of accidents/poisoning, (2) rate of cancer screening of PWDs, (3) prevalence of hypertension, (4) prevalence of diabetes, (5) prevalence of dental caries, (6) prevalence of periodontal disease, (7) level of oral hygiene, (8) level of oral care, (9) depression level, (10) stress level, and (11) level of suicide attempts.

Finally, opinions on the performance indicator items were collected through three expert advisory meetings and a representative meeting of six PWD groups. In this process, it was suggested that a medical facility that is accessible to PWDs is needed, as well as a system that provides integrated information on the health of PWDs. In response to these opinions, this research team newly added indicators for the following: (1) establishment of statistics on the health of PWDs (statistics on characteristics of disabilities), (2) establishment of facilities for PWDs in health care facilities (building and personnel), and (3) strengthening provision of health information services. The initial performance indicators were then constructed. The initial performance indicators are composed of six areas, 21 sub-areas, and 40 individual indicators (S1 Table).

Procedures of the modified Delphi

A total of three Delphi surveys were conducted through email from May to October 2017 for selected panelists. The first Delphi survey was based on the initial performance indicators; the importance and possibility of each indicator was evaluated by a four-point Likert scale (1 point: not at all important / no possibility, 2 points: not important / low possibility, 3 points: important / high possibility, 4 points: very important / very high possibility). “Importance” referred to the level of significance of the indicators in monitoring the health status of PWDs and suggesting a strategic direction, whereas “possibility,” whether the indicators could accurately quantify the measured and collected data.

The second Delphi survey was conducted based on the results of the first Delphi survey to inform the panel of the indicators adopted, revised/supplemented, and deleted for their reference in reevaluation. The participants were asked to describe freely their opinions on each indicator. As with the first Delphi survey, the “importance” and “possibility” of each indicator was evaluated on a 4-point Likert scale.

The third Delphi survey also provided guidance on the results of the second Delphi survey and asked for individual opinions. Likewise, as in the first and second Delphi surveys, the “importance” and “possibility” of each indicator was evaluated on a 4-point Likert scale.

The content validity index (CVI) was calculated for each survey stage to verify the content validity of each indicator. According to the criteria suggested by Lynn [17], indicators with a CVI of 0.8 or more were adopted, those with 0.5 to less than 0.8 were discussed (mediated differences of opinions), and those with less than 0.5 were dropped. The final selected indicators were not repeatedly measured (e.g., “the rate of health checkup of PWDs” was selected in the first Delphi survey, so it was not re-measured in the second Delphi survey). However, for the unadjusted indicators, the focus was on continuing the next Delphi round and drawing consensus from the expert panels. In principle, indicators were selected only if the CVI value was 0.8 or greater in both importance and possibility of the indicator. However, a number of indicators were selected as the final performance indicators even if the CVI for the possibility was less than 0.8 (Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Establishment of the performance indicators and modified Delphi process.

https://doi.org/10.1371/journal.pone.0208651.g002

Results

Selection of final performance indicators

This study conducted three Delphi surveys based on initial performance indicators and then selected the key areas and final performance indicators based on the collected results. The final performance indicators consist of a total of 6 areas, 22 sub-areas, and 40 individual indicators; the details are shown in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Final performance indicators.

https://doi.org/10.1371/journal.pone.0208651.t001

Discussion

Korean laws for PWDs have mainly focused on the promotion of PWDs’ welfare and participation in social activities. The Act on Right to Health for Persons with Disabilities is of great significance in that the rights to health of PWDs are stated in the law for the first time and their healthcare access is legally guaranteed. For factors hindering the health care access of PWDs, previous studies have identified the following: economic hardship, physical difficulties (i.e., difficult to visit a health care facility owing to transportation problems), difficulty in accessing information, medical staff’s negative perception of PWDs, among others [18–20]. According to a survey in Korea, the most uncomfortable experience by PWDs in using hospitals/clinics and receiving medical care is “physicians’ lack of understanding of disability and insufficient consideration” (34.8%), followed by “lack of PWD-friendly facilities in hospitals/clinics” (26.8%) and “difficulties in communication and information access” (14.1%) [21]. In the present study, such indicators as “establishment of facilities for PWDs in healthcare facilities (buildings and personnel),” “strengthening provision of health information services,” and “training and improvement of awareness of specialists in PWDs (regular curriculum, special curriculum, and medical personnel)” were developed to increase the health care accessibility of PWDs. Indeed, many policy studies conducted so far in Korea have largely been insufficient in terms of reflecting health assessment factors pertinent to PWDs. In this context, the present study is of significance in that to increase the health care accessibility of PWDs by considering various realistic difficulties faced by PWDs and their characteristics, new performance indicators were developed and evaluated for representativeness and validity.

Currently, various movements are being made gradually at the government level to guarantee health care accessibility of PWDs in Korea. Hospitals/clinics are required to set up an information desk devoted solely for PWDs, and government agencies are required to operate health information sites for PWDs to provide them with quality health information [8]. Furthermore, to develop personnel specialized in PWDs and improve public perception of PWDs, the Ministry of Health and Welfare recently announced to 11 health care-related associations, including the Korean Medical Association, the availability of “training for healthcare professionals on the health rights of PWDs” beginning in 2018 [22]. To establish and execute policies and projects such as these, evidence-based performance indicators that are objective and can be used for continuous monitoring should be developed. From this perspective, the performance indicators proposed in this study will be helpful in suggesting the direction for various national policy projects currently planned to improve the health care accessibility of PWDs.

In Korea, women with disabilities face problems as women and as PWDs, as well as problems with complex characteristics arising from the combination of the two. The health issues of women with disabilities have not received appropriate concern in Korea, taken as too sensitive to handle [23]. In conducting the present study, too, indicators in the area of women with disabilities underwent a number of revisions and modifications. In the beginning, topics that could include major issues like “abortion rate,” “safe contraceptive use rate,” and “sexual violence rate” were selected as initial indicators, but they were excluded during the expert advisory meetings and group meetings with PWDs. The reasons for excluding these indicators were “sensitive to handle” and “difficult to measure in reality.” Eventually, indicators in the area of women with disabilities came to consist of “regular screening rate during pregnancy,” “infant mortality rate,” “maternal mortality rate,” among others, after referring to the WHO data on reproductive health indicators [24]. To ensure continuous monitoring based on these indicators, an integrated database that combines raw data spread across various organizations should be constructed. However, at present, the statistics on the PWD population cannot be established for still being rudimentary. The construction of the aforementioned database requires a system linking the raw data sources spread across the Ministry of Health and Welfare, Korean National Health Insurance Service, Statistics Korea, and Korean National Rehabilitation Center, which can operate and manage them in an integrated fashion. The integrated data could be used to establish the foundation for improving the health level of PWDs by identifying the status of registered PWDs and computing health statistics by disability characteristic. With the integrated data, it would be possible to compute annual health statistics for PWDs, which will in turn enable continuous monitoring of the health status of PWDs.

There are a few principles to consider in developing performance indicators. The Public Health Agency of Canada proposed such principles as “data are collected regularly and trends can be compared over time (ongoing),” “data are available and of sufficient quality to report on or data collection can be put into place at a relatively low cost (feasible),” and “provides information that is considered to be meaningful and relevant to the target user (relevant)” [25]. Similarly, NHS Institute for Innovation and Improvement proposed principles of “importance and relevance,” “measurement possibility,” and “meaning” [26]. In the present study, these developmental principles were referenced, and importance and measurement possibility were examined for each of the indicators. That is, “whether the indicator is important in examining the health status of PWDs and suggesting the policy direction (importance)” and “whether the issue is actually possible to measure (possibility)” were simultaneously tested. After three rounds of Delphi survey, some indicators were high in CVI (0.8 or higher) in both importance and possibility. However, a few indicators had high CVIs for importance but somewhat low CVIs for possibility. These indicators require special attention to improve the problem of being difficult to measure. For example, for an indicator (i.e., sexual education experience) that can be collected through a survey, one might wonder which group of samples to choose. In the case of indicators for “depression level,” “stress level,” and “level of suicide attempts,” it is necessary select an assessment tool that can objectively and effectively measure mental health conditions. For example, using structured assessment tools such as “PHQ-9” and “Beck depression inventory” will allow for a more objective assessment and comparison of participants’ mental health to that of non-disabled people. In the case of indicators for “diagnosis of autism spectrum disorder in early childhood (average age and awareness),” institutions and funding to properly manage personal information are required. Moreover, national promotion and awareness improvement strategies are needed to improve awareness of children with autism. In the future, additional in-depth expert discussion should be held in regard to measurement tools for this indicator and the relevant survey data sources.

The present study set up key areas to be addressed in the Comprehensive Plan, and utilized and modified existing policy indicators to create performance indicators to monitor and evaluate them. New indicators were developed, necessary in the future to guarantee the health rights of PWDs and health care accessibility. However, a measurement study was not conducted on the newly developed indicators, and thus, specific suggestions were not made in the study as to the computational system and approach to utilize them. For the performance indicators developed in the present study to be applied in practice, follow-up research should be performed to test the validity more systematically and to determine specific uses of the indicators. Currently, in Korea, hospitals equipped with assistant personnel, facilities, and equipment are designated as disability-friendly health care institutions so that anyone, regardless of whether they have disabilities, can receive health checkups safely and comfortably. In addition, research is underway to develop customized medical examination items for the disabled in order to develop examination items according to disease prevention and disability characteristics. The proposed indicators for health management of PWDs can be used as basic data for establishing national policies for the disabled in Korea. In addition, it is also expected that these will enable continued understanding of the health level of PWDs and contribute to the production of statistics to improve health standards.

Supporting information