A novel approach to the classification of performance on inpatient perception of hospitalization experience across the US 52 states


 Background: Medical groups identified as high-performing with different approaches have been proposed for classification of performance in the literature, but no consistently-applied approach exists for identifying high performers for the benefit of healthcare providers. Most organizations classify hospitals using domain scores and assign a letter grade (e.g., from A to F in Leapfrog Groups) to the performance level. Whether bibliometrics can be an alternative for classifying healthcare givers’ performances is worthy of study. This study was performed to visualize survey results about inpatients’ perceptions of hospitalization experience for the US states using bibliometrics. Methods: We downloaded the 2014 summaries of HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Services) data. Four indices of h, PI, Ag, and x were applied to evaluate performance based on the core domains derived from the ten domain scores for each state in the US, and then displayed online dashboards to show hospitalization satisfaction across states on Google Maps. Choropleth maps were plotted for classifying the performnces into six grades using the quantiles method.Results: The top three states for hihg quality-of-care are Louisiana, Nebraska, and South Dakota using either x or Ag-index to assess. The Ag-index earns the mean correlation coefficient (=0.86) higher than the other three. The inpatient perception of hospitalization experience or the US states were classified and displayed on choropleth maps. Conclusions: The classification of healthcare performance is sensitive to the domain scores, the approach in classification, and the choice of metrics. The absence of a consistently-applied approach to identify healthcare performers impedes efforts to reliably compare, select, and reward high-performing providers. The x-index is recommended for quantifying the performance of healthcare givers in the foreseeable future.


Background
The quality of healthcare is always an issue in hospitals, particularly in the age of patient involvement in policy and the agendas of healthcare providers [1].There are many ways to classify quality performance on healthcare givers to the public.However, those professional indicators are unfamiliar to patients; they do not find it helpful to be provided with a static table or image format when searching for the best care [2][3][4][5].
Designating a provider as "high-performing" requires an agreed-upon definition of high performance, with standard decision rules and performance measures [6].Regrettably, no consistent definition of what constitutes a high-performing healthcare provider or system was found [7].
The England Picker Institute Europe (EPIE) [8], Hospital Compare (HC) [9], the Hospital Consumer Assessment of Healthcare Providers and Services (HCAHPS) [10], and the Leapfrog Groups [11] are four nationwide organizations that periodically assess patient expectations of hospital healthcare, experience, and satisfaction, and publicly report their perceptions of their experience and level of satisfaction [12][13][14][15].However, several limitations exist when classifying the performance for each hospital or state in the US.For instance, the Leapfrog Group applies 28 evidence-based measures of patient safety to compute a numerical score through the safety score for the eligible hospital.The numerical score is then converted into one of five letter grades."A" represents the best safety grade, followed in order by "B," "C," "D," and "F" [16].In 2018, letter-grade cutoff points were set for standard deviations at ≥0.6, ≥0, ≥−1.5, ≥−3.0, and<−3.0.Any hospital missing more than seven process measures, or more than five outcome measures would not be graded [16].
Similar to the study [6] adopting an all-or-nothing approach to defining high performance, only 58 medical groups reporting all selected measures across the four domains were selected from the total 240 medical groups in the database.
In the bibliometric analysis, an author individual research achievement is solely on the core publications at h, for example, using h-index [19] to interpret.The h-index is defined as the maximum value of h such that the given author has published h papers that have each been cited at least h times.As such, the index is designed to improve upon simpler measures instead of the total number of citations or publications applied to evaluate the achievements in academics.We thus have the motivation to use the metrics for classifying the performance of hospitals or the US states we did in this study.
The study aims to visualize survey results about inpatients' perceptions of hospitalization experience for the US states using bibliometrics to display.The tasks include (1) calculating metrics and determining the core domains on inpatient perception of hospitalization experience for each state in the US; (2) understanding the relations between types of metrics used in this study; (3) classifying the grades for each state using choropleth maps; and (4) highlighting the characteristics of the US states by using the metrics and the Kano model [20].

Study Data
We downloaded data (Summary Analysis of HCAHPS survey results in 2014) on discharges of inpatient perceptions of their hospitalization experience across all 50 US states, Washington, DC, and the territory of the United States Virgin Islands, for a total of 52 areas, at the HCAHPS website [8].The freely available spreadsheet there includes 10 dimension scores (range: 0-100 [higher is better]) consisting of the following:

Task 1: Calculating metrics and determining the core domains
Four metrics [18,[20][21][22] below were applied to calculate performances and determine the core domains based on the ten dimension scores mentioned in the previous subsection.
We define these metrics like the following: [Due to technical limitations, this equation is only available as a download in the supplemental files section], where ci is the citations at the ith publications, g is determined at the squared g not greater than the total citations at the gth publications, C(Pπ) is the total citation till the [Due to technical limitations, this equation is only available as a download in the supplemental files section] publications.
All the citations for each publication have been sorted at the first step by descending order.
For instance, on the x-index applied to the inpatient perceptions (ci) across the ten domains, we sort the domain scores for a specific state first, and then determine the number of domains at I, the responding ci, and the core at i.The greater is the metric, the higher is the performance of the state.It is worth noting that the domain scores have been divided by a value (e.g., 10 or 8) to meet the number of domains in this study.
Otherwise, the h and g indexes would be at ten because h and g are always smaller than .or the Ag (i.e., 10 < the least domain score in the dataset).

Task 2: Understanding the relations between types of metrics
The four metrics, along with the mean score for all states, were assessed by their correlation coefficients.From this, we can determine which metric earns the more or less mean coefficient.Two scenarios were manipulated by transforming domain scores by dividing 10 and 8, respectively.

Task 3: Classifying the grades for each state using choropleth maps to display
We created choropleth maps [23][24][25] to display the grades for each state.The quantiles method was used for classifying the grades to place an equal number of observations in each class.The problem with quantiles is that value legends with classes have very different numerical ranges (e.g., 1-4, 5-9, 10-250; the last class is huge): a situation that is very undesirable.The advantage of using quantiles is that they are suitable for computing Gini coefficient [26] and analysis of variance (ANOVA), owing to the almost equal number of sizes among classes.

Task 4: Highlighting the US states by the metrics using the Kano model
The Kano model is used for product development and customer satisfaction.It was developed in 1984 by Professor Noriaki Kano [1], and classifies products or items into four major categories: the must-be quality, the one-dimensional quality, the attractive quality, and the indifferent quality, on a diagram based on the satisfaction perceived by customers on Axis Y and the effort achieved by providers on Axis X.The first three categories have been transformed into various terms such as "delighters/exciters," "satisfiers," "dissatisfiers," but all refer to Kano's original articles.
In this study, we plot a Kano diagram to present the states on a dashboard displayed on Google Maps.The number of domains is on the X-axis and the scores at the number of core domains on the Y-axis.The bubbles are sized by the metrics and colored by the characteristics classified by the Kano mod

Task 1: Calculating metrics and determining the core domains
We present the x-index and the number of core domains in Table 1.The top three states are South Dakota ( = 80.3), Louisiana ( = 80), and Nebraska ( = 80), using the total (or the mean) to compare.If x-index is applied, Louisiana ( = 7.9) ranks first and followed by South Dakota ( = 7.8), and Nebraska ( = 7.8).All of them have an identical number of core domains ( = 9).The domain of Overall Hospital Rating on item 9 is redundant and therefore excluded from the core domain by many states.

Task 2: Understanding the relations between types of metrics
We were curious about whether any difference exists if disparate metrics are applied.The original summation (or average) score and the Ag own the highest mean coefficient ( = 0.86), followed by x-index ( = 0.79) and PI ( = 0.77).The h-index has a lower coefficient ( = 0.67); see Table 1.It is worth noting that the x-index remains identical regardless of whether the domain scores were divided by 10 or 8.This is because the linear transformation for domain score (ci) do not affect the ranking x-index, which is determined by the maximum product of i and ci [Due to technical limitations, this equation is only available as a download in the supplemental files section] using the rectangle area to evaluate.

Task 3: Classifying the grades for each state using choropleth maps to display
Choropleth maps are shown in Figures 1 and 3 using x and Ag-index to display, respectively.The top three states are identical in these two maps, but the difference in order exists, indicating that the classification approach is sensitive to the results of the performance assessment [6].Six grades on the inpatient perception of hospitalization for the US states were classified and displayed on choropleth maps; see the colors in Figures 1 and 2.

Task 4: Highlighting the US states by the metrics using the Kano model
The number of core domains is ranged within 8 and 10 if x-index is applied, different from the Ag in a range between 6 and 7, see Figure 3 and 4. Due to g similar to h-index at ci = h, all bubbles called the one-dimensional quality in Ag-index are present between the two blue limit lines in Figure 4.In contrast, some are classified as the productive quality in red bubbles using x-index to display in Figure 3.
If the domain scores are transformed by dividing the value of 8 instead of by 10 in Figure 3, the rankings for x-index across states are retained, but the dispersion is changed, and more states in green are located at the top in Figure 5 because the scores have been augmented by dividing by 8 instead of 10, as in Figure 3.

Discussion
We observed that the top three states with higher quality-of-care are Louisiana, Nebraska, and South Dakota using either x or Ag-index to assess.The Ag-index earns the mean correlation coefficient ( = 0.86) higher than the other three metrics.Six grades on the inpatient perception of hospitalization were classified for the US states.

What this study contributes to current knowledge
We applied the metrics to rate inpatient perceptions of hospitalization experience in the US states.The principle of the bibliometric analysis disregards redundant domains and the excess scores for taking into account, which is a simple and easy way to compute metrics and categorize the classes, particularly using the Kano model for classification, which is never seen in past articles.
The quantiles methods or others [27,28], such as the equal interval method and natural breaks (e.g., using an algorithm as k-mean [29]) can be properly used for classifying classes for entities.We particularly highlight the vital few classes labeled by the number from 1 to 3 on choropleth maps (see Figures 2 and 4), which is one of the features in this study.
Another feature is the Kano model applied to classify the entities.The reason there are no such states at the top is that the transformed domain scores (i.e., divided by 10) are less the score in Figure 3 than that score (i.e., divided by 8) in Figure 5.For instance, Louisiana in Figure 1 has the highest x-index at 7.88, but the domain score (ci) is 6.9 less than the number of x-core at 9. Another example is the District of Columbia, with x-index = 6.82, domain score at 5.8 less than the core at 6 in Figure 4. Interested readers are invited to scan the QR-code in Figures to see the details about any state.The dashboard on Google Maps is another feature that allows the public to easily understand and quickly select the high-performing states.

Strengths of this study
We recommend using the x-index to evaluate the quality-of-care for hospitals or the US states.The reason is not only the feature of invariance in Table 2, but also the suitability of three main classifications displayed in the Kano diagram.Although the Ag has higher discrimination power than the h and g-index [30], the x-index also owns the discrimination power for ranking institutional-level research achievements.
On the other hand, we usually discard hospitals with missing items in classifications of performance [16] or like the study [6] only including 58 medical groups reporting all selected measures across the four domains.We mimic the bibliometric analysis considering the core publications (or domains in the current study), like the individuallevel metrics (h, g, PI, or x) that can evaluate any one with a variety of publications and citations in the past.Accordingly, many hospitals (or states) can be assessed together on a diagram, see Figures in this study.Furthermore, many examples of disparities in health outcomes across areas, such as dengue outbreaks [31,32], disease hotspots [33], and the Global Health Observatory (GHO) maps on major health topics [34], have been presented using choropleth maps [23][24][25] to display.Our representation in Figures 1 and 2 on Google Maps is unique and innovative and has no precedent in the literature.

Limitations and suggestions
Despite the findings shown above, several potential limitations require further research efforts.First, the sample for this study only comprised HCAHPS survey results from 2014.
As such, the top three states (Louisiana, Nebraska, and South Dakota) with higher qualityof-care cannot be generalized to other years or areas.
Second, there might be some biases when downloading HCAHPS survey results from the website.For example, it is worth further examining why the domain score for item 9 of Overall Hospital Rating in Tabel 1 is so much lower than other domains.Third, we recommend using the Kano diagram to classify entities.There is no software [35] that can be commonly used for ordinary researchers.Our technique of plotting a Kano diagram in MS Excel for visualizing and classifying the characteristics of entities needs to be explained in detail in another future article.
Fourth, although the x-index is recommended for hospital evaluations in the classification of performance according to Table 2, other indices or approaches, such as relative thresholds or absolute values of performance, are worthy of discussions and further study.
Finally, numerous classification methods used for displaying legends on choropleth maps have been proposed before [27,28].We applied the quantiles method to classify six grades for US states (see Figures 1 and 2), which is also worth discussing the merits and disadvantages used for displaying choropleth maps in the future.

Conclusions
The classification of healthcare performance is sensitive to the domain scores(e.g., composite or summation score), the approach in classification(e.g., Kano model), and the choice of metrics(e.g.,x-index).The absence of a consistently-applied approach to identify healthcare performers impedes efforts to reliably compare, select, and reward highperforming providers.The Kano model and the x-index is recommended for quantifying the performance of healthcare givers in the foreseeable future.

Supplementary Files
This is a list of supplementary files associated with the primary manuscript.Click to download. dataset22.xlsxquestions.pdf

Table 1 .
a The scores of inpatient satisfaction survey in 2014 for each State in the US