Satisfaction with primary health care in Ukraine in 2016 – 2020: A difference-in-differences analysis on repeated cross-sectional data

The aim of this study is to examine the general satisfaction with primary health care services in Ukraine among service users and nonusers before and after the implementation of the capitation reform in 2017 – 2020. Data from a repeated cross-sectional household survey ‘Health Index. Ukraine ’ in 2016 – 2020 were used. The survey had a sample size of over 10 000 participants per survey round. Effects were estimated using difference-in-differences methods based on matched samples. Our findings show that in general, respondents are ‘rather satisfied ’ with the services of district/family doctors and pediatricians. Satisfaction with family doctors comprised 72.1 % (users) and 69.2 % (nonusers) in 2016; and 75.3 % and 71.9 % in 2020. For pediatrician services, these shares were 73.6 % (users) and 71.1 % (nonusers) in 2016; 74.7 % and 70.2 % in 2020. Our study also revealed an increase in satisfaction with the district/family doctor over time. However, this does not seem to be due to the reform. The results for pediatrician services were mixed. Why satisfaction with primary care is fairly high and slightly increasing over time is unclear. However, we offer several possible explanations, such as low expectations of primary health care, subjective perception of quality of health care services, improved access and affordability, and general improvements in primary health care settings not directly linked to the reform.


Introduction
The aim of the 2017 -2020 reform of the health care financing system in Ukraine is to improve people's health and to reduce the financial burden for patients.This reform is expected to transform the outdated centralized health system and eliminate informal payments.A modernized, efficient and high-quality health system is the intended result of the reform [1].
During the period 2017-2020, the reforms consisted of two stages.In the first stage, the focus was on primary care (2017-2019), while in the second stage (2020), the focus was on secondary and tertiary care.Primary health care reform included a change from a fixed line-item budgeting system of primary health care (central financing) to percapita financing [2], creating managerial autonomy for health care providers and free choice of provider for health care users.The law 'About the state financial guarantees of medical services to the population' N • 2168-VIII was adopted in October 2017.Several other supporting laws were adopted in November -December 2017, and in March -April 2018.A national payer (the National Health Service of Ukraine) was created in April 2018.The first agreements with health care services providers were concluded in spring -summer 2018, enabling the providers to obtain reimbursement for the treatment of their patients at predefined rates.For this, the providers needed to register their patients by concluding a declaration with them and introducing the data into the electronic medical system.The implementation of this stage of the reform started in July 2018 and was completed in February 2019 with the adoption of the state budget by the Parliament (Verkhovna Rada) of Ukraine [2].The changes are expected to influence satisfaction as increased competition between the providers will trigger a change in managerial and clinical practices, making the health services more responsive to the needs and expectations of users.However, this expectation has not been investigated and is therefore the focus of our paper.
Before or after the reform, not many studies have been conducted in Ukraine on satisfaction with health care.Footman et al. [3] revealed in a multi-country study that in Ukraine, the level of satisfaction with health care increased from 12.3 % in 2001 to 17 % in 2010.Another cross-country study [4] reported 41.4-45.9% respondents' satisfaction with quality of and access to health care services used in Ukraine, which was low, for example, compared to Hungary where 67.3-70.3% satisfaction level was observed.Also, the study of Luck et al. [5] underlined that only 33 % of the household respondents in Ukraine were satisfied with the current health system and 79 % stated that it needed reforms with a focus on quality.So far, reports on satisfaction with health care after the reform have not been identified.The aim of this study is to examine the general satisfaction with primary health care services among service users and nonusers before and after the implementation of the first stage of the primary health care financing reform in Ukraine, i.e., before and after the capitation-based payment, creation of managerial autonomy for providers and free choice of provider for patients.Managerial autonomy gives freedom to the provider to allocate funds in compliance with the needs of patients, becoming more responsive.The free choice of the provider enables patients to choose their own doctor and is intended to create competition to attract patients between providers at the level of quality of the services.Capitation-based payment is expected to trigger the application of up-to-date managerial and clinical practices to sustain the satisfaction and loyalty of the patients.Thus, we expect that the first stage of the reform in primary care triggered positive changes in the service provision, increasing satisfaction with primary health care services.We investigate this expectation in our paper.

Materials and methods
Data from the annual household survey 'Health Index.Ukraine' were used.The data were collected with the objective of studying general satisfaction with health care, health behaviors and experiences in seeking health care services, health expenditures, as well as attitudes towards the health care reforms.The survey was supported and managed by the International Renaissance Foundation and was carried out by the Kyiv International Institute of Sociology among household representatives by means of individual face-to-face interviews carried out by trained interviewers.This method allows a maximum representation of all population strata and also has the benefit of more prolonged direct communication with a survey person.
Five rounds of the 'Health Index.Ukraine' survey were conducted in May-June 2016, in May-June 2017, in June-July 2018, in June-July 2019 and in August-October 2020.A new sample was drawn for each survey round.The sample was representative of both the entire country and each region (oblast) and had a large sample size (over 10 000 participants per survey round).A multi-stage sampling technique, random at each stage, was used.Inhabited locations were first chosen proportionally to the oblasts (administrative-territorial unit) population size.Then, the random route method was applied in choosing areas, streets, buildings and apartments.As a result, one individual was randomly chosen for an interview from a household.If a respondent could not be reached twice, the same approach was applied and another respondent was chosen.More detailed information about sampling can be found on the webpage of the 'Health Index.Ukraine' survey [6].
The 'Health Index.Ukraine' questionnaire was designed using the experience of the EuroHealth Consumer Index [7] and the health science research by the Government of Canada [8].It was also validated by expert discussions and approved by the International Scientific Board established specifically for the purposes of the 'Health Index.Ukraine' project.
Before each survey round, the questionnaire was pretested by surveying 25 respondents (24 in 2016) from Kyiv city and oblast.Every year, the questionnaire was slightly modified.However, the wording of the questions chosen as the data source in this study remained the same (for the exact wording of questions selected for analysis, see Appendix A).
As in Ukraine there is no obligation to obtain ethical approval for research of a non-clinical character, the International Scientific Board of the 'Health Index Ukraine' survey took the decision not to seek such approval.In all survey rounds, participants were assured that they were free to withdraw from the survey at any time without any negative impact.All data are kept confidential.No identifying information is shared with third parties.
To capture the effect of the first stage of the reform in primary care, we used the data on general satisfaction with services of district/family doctors and general satisfaction with services of pediatricians (primary health care) before and after the health care financing reforms.
As explained in the introduction, the first stage of the reform was implemented in primary health care from July 2018 to February 2019.Data for 2018 that we used, were collected before this reform period and data for 2019 were collected shortly after this reform period (June-July 2019).At the time of the survey in 2019, 26 million patients out of million people in Ukraine were registered with a family doctor (According to the official website of the Ministry of Finances of Ukraine, the population of Ukraine was 42 122.7 on 01.02.20219).
Given the above explanation, the data for 2016, 2017 (year of legislation) and 2018 refer to the period before the first stage of primary care reform, data for 2019 refer to the period of implementation (during), and data for 2020 refer to the period after the reform.Respondents who had used primary health care in the period of 12 months before being interviewed, were coded as 'users' and respondents who had not used primary health care in the preceding 12 months before the interview were coded as 'nonusers'.
We first performed a descriptive analysis based on summary statistics for all variables included in the study.We also applied two-sample t-test to continuous variables as well as Wilcoxon rank-sum (Mann-Whitney) test to binary variables to assess whether the differences in sociodemographic characteristics between the group of users (treatment group directly exposed to changes in health services resulting from the health care reform) and the group of nonusers (control group not directly exposed to changes) were statistically significant.
We included the following socio-demographic variables: age, gender, education, type of settlement, self-reported health status, and the number of persons in the household.Age, education, and health status are the socio-demographic factors known to have an effect on satisfaction with health care [9].Gender and social class have an unclear influence on satisfaction with health care services [9].Women are found to be more satisfied with health care services than men, whereas men are found to be more satisfied with nursing care, comfort and cleanliness than women [10].We also included indicators for oblasts (administrative-territorial units) because previous analysis showed its significance [6].
Next, we applied a difference-in-differences (DiD) approach by means of ordered logistic regression.The DiD approach is a quasiexperimental technique that was developed to measure the 'effect of the treatment on the treated' based on before and after comparison between the treatment and control group in the case of panel data [11].We performed four regressions where satisfaction with services of district/family doctors was the dependent (outcome) variable using data for 2016 and 2020, 2017 and 2020, 2018 and 2020, and 2019 and respectively (as mentioned above, 2019 was the year of reform implementation).We also performed the same four regressions where satisfaction with services of pediatricians was the dependent variable using data for the same years.All 8 outcome variables were ordinal variables ranging from 1 = completely dissatisfied to 4 = completely satisfied.In all regressions, the following three dummy variables were added as explanatory variables: an indicator of the treatment group (users = 1; nonusers = 0), and an indicator of the period (before/during the reforms = 0; after the reforms = 1), as well as the interaction between these two indicators (DiD effect).We first performed all 8 regressions without and then with the socio-demographic variables (covariates) V. Anufriyeva et al. mentioned above.
The repeated cross-sectional nature of the data implies that not only individual characteristics of the respondents are not similar between users and nonusers, but also there might be time variation within these two groups.In this case, propensity score matching can make the groups more comparable.Difference-in-differences analysis on matched data is widely used with panel data, whereas its application for repeated crosssectional data analysis is rare [12].Thus, this study examines satisfaction with primary health care services and reports on the application of difference-in-differences after matching repeated cross-sectional data.
Since we applied the DiD approach to repeated cross-sectional data, we also had to control for time-invariant imbalances [12].Therefore, we did matching across the years within the treatment group and within the control group.Matching between the treatment and control groups was also performed in order to make the DiD analysis more robust [13].For the matching, we used the abovementioned covariates except for oblast, and we performed matching using the nearest neighbors method (command psmatch2 in the software package Stata 15).The Oblast covariate was not included in the matching because including a lot of covariates made the matching difficult [14].We then re-run the above eight regressions on the matched data with and without covariates.
To check the robustness of the DiD analysis, we run the DiD analysis using different matching techniques.Therefore, we also used the command diff in the software package Stata 15 [15], which performs the three matching steps using kernel propensity score matching followed by the DiD analysis.For the illustration of DiD analysis, see the flowchart (Appendix B, Figure 1).
As outcome variables, we used the answers to questions about satisfaction with the different parts of the health system: 'From your own experience of consuming private or public health care, or from the experience of other people, please state how satisfied or dissatisfied you are with the way each part of the health system is functioning'.The response was measured using the scale 'completely satisfied', 'rather satisfied', 'rather dissatisfied', 'completely dissatisfied'.This means that we studied general satisfaction with services and not satisfaction with specific services used.The questions were asked to respondents, irrespective of whether they had used health care in the papers period.The respondents rated services of district/family doctors, pediatricians, dentists, as well as maternity care, emergency care, hospitalization and outpatient services (details are provided in Appendix A).
As can be seen from Table 1, the respondents are mostly 'rather satisfied' with the services of district/family doctors and pediatricians.
Table 2 presents the summary statistics for the socio-demographic characteristics of respondents used in the analysis.As can be seen in the table, these characteristics are similar across 2016-2020.The mean age of the participants was 50.5 years old.Women comprised 65.7 % of all respondents.Participants with good or average self-reported health from urban areas with specialized secondary education prevailed.The average number of people in the household was 3.
The results of the two-sample t-test with equal variances and the Wilcoxon rank-sum (Mann-Whitney) test on the unmatched data shown in Table 2 indicate statistically significant differences in the variables gender, education, type of settlement, self-reported health status, number of persons in the households and oblasts between the usernonuser groups.
The results of the DiD analysis using ordered logistic regression, with and without covariates, before matching are shown in Appendix C. All regression models have a good fit (Prob > chi2 between 0.0000 to 0.0232).Users slightly more often state that they are satisfied with services of district/family doctors than nonusers, and the level of satisfaction increased over time in both groups.The results for satisfaction with pediatrician services are mixed.The reform effect (DiD coefficient) is statistically significant only for the satisfaction with pediatricians in 2017-2020 and 2018-2020.The coefficient indicates that after the reform, satisfaction with services of pediatricians had increased among users more relative to nonusers.
Table 3 shows the results of the three matching steps based on the nearest neighbors matching, namely matching across years within the  treatment and control group, respectively, and between the treatment and control groups.Figures 2-9 compare the data before and after matching (Appendix D).The results show that matching has improved the comparability of the groups' data.
The results of the DiD analysis using ordered logistical regression on the matched data are shown in Appendix F. Overall, these results confirm the results of DiD analysis on the unmatched data (see Appendix C).Specifically, the coefficient of the before-after variables and the usernonusers variable is statistically significant only for the satisfaction with pediatricians (without covariates) in 2017-2020 and 2018-2020.
To check the robustness of the DiD analysis, we run the DiD analysis using different matching kernel propensity score matching for repeated cross-sectional data.The results are presented in Table 4. Again, the results confirm previous results (Appendix C and F).Specifically, the DiD coefficient is statistically significant only for the satisfaction with pediatricians in 2017-2020 and 2018-2020.

Discussion
The main findings indicated an increase in satisfaction with the district/family doctor over time among both users and nonusers of primary care.However, this increase does not seem to be due to the reform.The results for pediatrician services were mixed.The reason why satisfaction with primary care is fairly high and slightly increasing over time seems unrelated to the reform of primary care is unclear.However, there could be other possible explanations based on previous literature.
First, our findings show that in general, respondents (both users and nonusers) are 'rather satisfied' with their district/family doctors and pediatricians.Patient satisfaction is related to the difference between patient expectations and perception of the services used [9].Ukraine was found to have the lowest expectations about the health system among 12 countries included in the study of Kressens et al. [16].Thus, general satisfaction with primary health care might be partly explained by the low expectations that people have of primary health care.Because of the rigidness of the health system, patients often use self-coping strategies in seeking directly specialized care and avoiding using family doctor services [5].
Second, the general satisfaction with primary health care might also be explained by the subjective perception of quality of health care services by individuals.Despite the dissatisfaction with the conditions of service provision like accessibility [4], patients were generally satisfied with the qualification of their doctors and treatment results [17].Thus, patients may perceive quality as high because they were treated by a qualified medical doctor and the treatment was effective for them, irrespective of other quality aspects.This is confirmed by a study showing that outpatient care quality, qualification of medical personnel and effectiveness of treatment were most important for health care users in Ukraine [18].
Users of primary health care in our study rated satisfaction with family doctor and satisfaction with pediatrician higher than nonusers.They had access to reimbursed medicines once patients enrolled in the program 'Affordable medicines' and could obtain pharmaceuticals prescribed electronically by the family doctor.Affordability and access are important determinants of health care services utilization [19], which could explain higher satisfaction of the users.At the same time, users evaluated their own experience of the services used.Whereas nonusers based their evaluation on those of other users, like family members.Thus, nonusers might focus more on satisfaction with the health care system in general.Satisfaction with the services used tends to be evaluated higher than satisfaction with the health care system in general, as the respondents might mix satisfaction with governance and satisfaction with the health care system [3].Our study also revealed an increase in satisfaction with the district/family doctor.As the DiD effect is not statistically significant, this increase appears not to be influenced by the reform.Over the years, other improvements have taken place in primary health care in Ukraine.For example, renovations have been made, equipment has been procured, qualification of doctors has improved also by means of training in management, etc. Health care users also became more aware of payment policies [2].Specifically, the physical environment (including flexible payment mechanisms) is positively associated with patient satisfaction [10].
The results on satisfaction with pediatrician care were mixed.We found a statistically significant increase in satisfaction with the pediatrician in 2017-2020 and 2018-2020.These results, however, should be treated with the appropriate caution as pediatricians in Ukraine are only vaguely defined.Pediatricians are employed both in primary health care (where they perform the family doctor functions for children) and in secondary care (inpatient care in children's hospitals).Patients do not always clearly distinguish between the two.Thus, we cannot clearly assign the level of satisfaction found in our study to primary care pediatricians.This statistically significant increase in satisfaction with the pediatrician might be explained by the slight shift of functions of the family doctors.Family doctors before the reform were seeing only adults whereas pediatriciansonly children.Now, patients can choose the family doctor who would combine these functions and also see the child.The quicker access and easier use for parents might be the reasons for increased satisfaction with pediatric services [20].Furthermore, the change to a per capita financing model enhanced provider competition, and may have provoked better clinical practices along with better managerial and communication practices.
There is a general consensus among the population that the health system of Ukraine requires reforms [21], and expectations among the population about the outcomes of the reforms were high.However, the overall absence of the effect of reform on satisfaction might be explained by the fact that the frequent changes of governments in Ukraine, including in the Ministry of Health during the pre-transition period (from March 2010 till August 2016 Ukraine had 9 Ministers of Health), have neither facilitated consistency in goals and practices in the health system nor helped to maintain trust of the population in the health system.
The absence of a direct effect of the health care financing reform in Ukraine on satisfaction with primary health care services in our study, is similar to results reported in the review by Kutzin et al. [22], who compared the experience of post-soviet and some European countries in health care financing reforms (including Albania, Georgiya, Czech Republic, Estonia, Kyrgyzstan, Moldova, Slovenia, Russian Federation, Tajikistan, etc., including Ukraine).This review showed that quality improvement was limited if based only on financial reforms.The financial reforms should be combined with the medical education development (e.g.efficiency in the delivery of services is promoted) and quality improvement at the provider level (e.g.internal quality improvement processes and accreditation are the preconditions for contracting) to have an impact on patients [22].This seems to be the regional peculiarity as other studies in low-and middle-income countries found correlations between patient satisfaction and changes in health care driven by policy interventions [23,24,25].The results of our analysis could help to develop policies that further the implementation of necessary changes that make the system more responsive to the needs and expectations of health care users.
Other confounding variables included in our analysis, such as age, gender, education, type of settlement, self-reported health status, and the number of persons in the household also showed an influence on satisfaction in users.However, only gender and self-reported health status had statistical significance over the years (2016-2020).Whereas age showed no statistical significance and other variables were statistically significant three years out of five included in the analysis.Thus, the results of our analysis confirm the results of previous analyses described in the literature [9].At the same time, the reason why some of the variables showed influence in certain years is unclear.
Our study has certain limitations that should be mentioned.The reform of health care financing (capitation-based payment, managerial autonomy and free choice of the provider) in Ukraine was planned to be realized in two stages: first stage focused on primary care, which was completed in 2019 and second stage focused on the secondary and tertiary care, which started in 2020.Thus, in this paper we only analyzed the impact of the first stage focused on primary care.As mentioned in the introduction, the first stage was completed in February 2019 and on the moment of data collection in 2019, 26 million patients were registered with a family doctor [2].The registration does not mean that the primary health care services were actually used.Thus, the results for 2019 need to be interpreted with caution.The definition of primary health care services is not clear in Ukraine, especially in case of pediatrician services.There are also limitations connected with the data used in the analysis.We had access to repeated cross-section data.These may not be able to capture changes in individual characteristics over time.For this, longitudinal data are needed.Furthermore, for the identification of the control group.In our analysis, we used nonusers as a control group.Nonusers may not be the best control group because of their statistical dissimilarity from the group of users.In addition, a repeated cross-sectional design has limitations in capturing individual-level changes over time.Thus, we applied matching techniques to make relevant observable characteristics similar in order to facilitate a comparison between them.At the same time, nonusers referred not to their own experience but based their stated satisfaction on the experience of others, for example, family members who used the services in question.The interpretation of our results, specifically the explanation of high satisfaction levels in the absence of a direct influence of the reform, was based on previous literature.In addition, we applied only quantitative measures of satisfaction for our analysis.Supplementing the quantitative data with qualitative research methods, could provide deeper insights into the drivers of satisfaction and could help to assess the impact of health care financing reforms on the quality and accessibility of primary care.

Conclusions
Our study provides new insights into the general satisfaction with primary health care services among users and nonusers as well as into the impact of the first stage of health care financing reform on satisfaction with primary health care in Ukraine.Overall, we did not find evidence for a direct influence of this part of the reform on satisfaction with primary health care services, even though that was our expectation, as indicated in the introduction.At the same time, satisfaction with the primary health care services increased over time in both groups: users and nonusers.In the discussion, we offer several possible explanations, such as low expectations of primary health care, subjective perception of quality of health care services, improved access and affordability, changes in health care management and general improvements in health care, including more transparent payment policies.Therefore, we recommend further study to investigate the underlying factors for these findings.Specifically, the analysis of factors influencing the increase in patient satisfaction in Ukraine and the reason for mixed results in pediatric services are needed to provide policy makers and primary health care services providers with evidence that can be used for further quality improvement.
Our analysis might be interesting for countries with either similar health systems or with health systems in transition, undergoing the same change of the Semashko system through capitation reform.Our results suggest that payment reform may not lead to higher satisfaction with the health care system.

Table 1
General satisfaction with primary health care: descriptive statistics.

Table 2
Socio-demographic variables: descriptive statistics and results of the comparative test for the unmatched data Two-sample t-test for continuous variables and Wilcoxon rank-sum (Mann-Whitney) test for categorical variables.
(continued on next page) V.Anufriyeva et al.

Table 2
(continued ) a Descriptive statistics are estimated excluding missing values.b Oblast = administrative territorial unit.c Kyiv city is a separate administrative territorial unit.

Table 3
Differences after matching (k-nearest neighbors matching, stepwise matching across years per user/nonuser group and across groups).

Table 4
Difference-in-differences analysis for repeated cross-sectional data with Kernel propensity score matching based on linear regression.