ANALYSIS OF THE LEVEL OF USE AND ACCEPTANCE OF THE MEDICAL INFORMATION SYSTEM IN PRIMARY HEALTH CARE

This paper presents the analysis of use and acceptance of the medical information system (MIS) within the primary healthcare. Analysis is mostly based on data from the Health Care Center Niš, although the conclusion was made on data from a dozen health care centers which use the medical information system MEDIS.NET developed within the Laboratory for Medical Informatics at the Faculty of Electronic Engineering in Niš. Analysis of the use of MIS is based on calculating the percentage of successfully entered records of visits, provided medical services, recipes, referral letters and physical examinations. In the context of the analysis of the use of MIS, successfully entered medical service is actually the service that was not changed or deleted later. Results of this analysis are significant for further technical development of the medical information system, and support the identification of these functionalities that are hardly accepted by the end-users and should be further developed. The acceptance of MIS is analyzed in the light of the technology acceptance model. Registration of provided services and keeping the record of physical examinations are taken as representative functionalities. Registration of provided services has been observed as a functionality that is accepted by the users due to simplicity of use (perceived ease of use), while the registration of physical examinations is observed as functionality is presumed to be accepted by the users as useful (perceived usefulness). For the functionalities with the expected acceptance based on the simplicity of use, the rate of correct data input is over 90% in each of the category. However, the rate of correct data inputs for visits and provided services is more than 99%. This is very significant having in mind the fact that these functionalities are often used and the high rate of incorrect inputs would slow down the work of doctors. On the other hand, the percentage of use of special functionalities for input of physical examinations varies considerably. Specially designed functionality for the most common physical examinations of children is used in more than two thirds of cases (sometimes more than 97%), while for the registration of adult’s physical examinations the percentage is lower than 20%. Since the users could input data on physical examinations using the form for visit input, as well as with special form, they will probably use the other option only when the frequency of use is high enough or when the improved functionality of the specialized form provides improved system performances. Under users of MIS we consider the medical staff which uses MIS functionalities in accordance with their duties and privileges (doctors, nurses, medical technicians, etc.). Acta Medica Medianae 2018;57(4):122-136.


Introduction
Medical Information Systems (MIS) are designed in order to improve the work of healthcare institutions, enable better resources management and be a good basis for generating various types of reports (1).Even though they are developing during the last half century, their acceptance from the potential users did not always go smoothly (2).Globally, the trend of acceptance of medical information systems and their efficient use starts mid nineties (3), while the final expansion was actually during the first decade of the 21 st century (4).Massive use of the medical information systems within the primary healthcare of the Republic of Serbia starts from 2010-2011 with the great support of the Ministry of Health.Health Care Center in Niš, as the leading regional health center, started the implementation of the medical information system as the pilot project with the Faculty of Electronic Engineering in Niš (5,6).Information system, named MEDIS.NET had been in full use since January 2012, and after that was installed in twenty more healthcare centers in South and East Serbia.
After a four-year period, and active use of MEDIS.NET, it is possible, from the point of technology acceptance model, to analyse the effects of use of MIS.Our goal was to assess the level of acceptance of medical information system in general through the analysis of collected data, as well as to assess its individual parts that were given special attention during the development phase.Our basis for analysis was technology acceptance model ( 8) which considers the system acceptance through two categories of functionality -the ones presumed to be easy to use and the ones that enhance the efficiency of the healthcare institution.
There are a large number of papers dealing with the analysis of the MIS functionality acceptance.Within this research, the particular importance was given to the part (9) on detailed analysis of the use of medical records in primary health care.Beside this, the authors presented the overview of great number of positive and negative aspects of introducing the medical information systems as the collection of implementation strategies that contribute to system acceptance from end users.
In general, the usefulness of the system is commmonly marked as the key element of system acceptance (10).For users, a well designed software system that does not follow their working processes and needs is less significant.During the system development and in communication with potential users it was very important to mark the most useful things (11) and to develop the system in this direction.After the development phase is finished, and starts the system use phase, it is necessary to monitor the users' behaviour and make adaptations of critical parts.
This paper is the result of such monitoring and the results will be the basis for improvement of the most important parts of the MIS.As mentioned in (12) and (13), the acceptance of medical information systems is not a linear process and after initial analysis it is necessary to monitor users' behaviour and react to their changes, suggestions and recommendations.
Potential users could have different opinions on certain parts of the system, and the promptness of functionalities that are not considered as basic could vary a lot (14,15).Although it is nowadays considered that the medical workers are determined to accept information technologies in their everyday work, a various operative inefficiencies of realized softwares could result with complete rejection of some initially good functionalities.On the other hand, some simple functionality could be identified by users as extremely important and generally could be very quickly accepted.

Material and methods
From the technical point of view, it was of great interest to analyse the system use by medical workers, and in accordance with this to define the guidelines for further development and adaptation of the existing functionalities, as well as forming internal recommendations for more efficient realization of new MIS segments.In order to get objective results, we followed the analysis of collected data for the period from 01/01/2012 to 31/12/2015.Results were processed through the prism of technology acceptance model (TAM).Figure 1 presents block scheme of TAM.Previous experience, domain knowledge and social context could be defined as so called external variable that affects the acceptance of a system.Accordingly, users will accept different system functionalities either because they perceive them as ease of use (Perceived Ease of Use -PEOU), or because they find them useful (Perceived Usefulness -PU).From the point of view of these two categories, the reaction of users considering the functionality will affect the level of functionality acceptance.Another measure for the level of acceptance is the number of mistakes, or corrected records.Ideally, the percentage of corrected records should be the lowest as possible and with the entities that are more frequently created should be lower.(8) In this paper, the primary focus was given to three, from our point of view, the most significant set of entities processed by the medical information system, and these are provided services, referral letters and recipes.The most important entity defined at the level of MIS is provided service that presents the record on provided medical services to patients.They present the basis for all further actions, such as creation of report or invoicing of provided medical services.Also, they present the parent entity for all medical documents such as referral letters and recipes.Provided medical services are modeled in a way that every change upon them is recorded and kept in database as provided service.Once the provided service is changed, the previous version stays recorded as non-valid provided service and could be seen only through the process of revision and generating the report.Secondarily, we have analysed the use of specially defined forms for physical examinations, as specific categories that have specially defined forms for registration and data view.Basic part of the physical examinations is recorded as the entity of provided service, while the specially defined fields that are specific for physical examinations are kept separately.
For both focus points of our paper, the elements of MIS were mainly developed in such a way as to provide full support of established practice and to make it possible for the users to transfer from paper to electronic form of record keeping.Through PEOU approach, our idea was to enable users to initi-ally accept MIS functionalities with simple and easy training, and later with system finishing, to focus on PU in order to increase the work efficiency and recognize the advantages of MIS use.

Results
First important thing that we have analysed is the percentage of successfully entered records for entities that describe provided services.Since MIS users in primary healthcare mainly have large number of patients, it is very important for system to be designed in such a way that users change or delete records as less as possible.Each of these actions is unnecessary waste of time and the general goal is to have less of these actions.For entities such as provided services that percentage should be less than 5%.The level of 95% of valid records means that users had to change or delete on average one out of twenty provided services.Having in mind that data from our MIS could not be deleted, but marked as non-valid in case of deletion or update, absolutely all created records are available from the database special reports or revision.
At the end of period of our research, in the fourth quarter of 2015, the percentage of valid provided services was 99.23 % (Table 1), the best quarterly result since the beginning in 2012.Initially, the percentage of valid provided services reached the level of 95% already in the second quarter of 2012.Initially, this percentage was 92.86% which was satisfactory.It is important to mention that during the first quarter of 2012, the system was used by 136 users who worked in 16 organizational units.At the same time, the training was provided for new users and many initially noticed disadvantages were corrected.
Already in second quarter of 2012, the number of users went up to 342, while the number of active departments went up to 30.Over time, the percentage of successfully recorded services had the mild uptrend, which could be explained by the fact that the medical staff was getting more and more secure and more easily accepted the provided technology.During the third quarter of 2012, the percentage of successfully recorded services went up to 98%, and in the second quarter of 2014 went over 99%.
With the percentage of correctly entered provided services, important factor is their total number, as well as the moment of increase of these recorded data.At the beginning (January -March 2012), the number of recorded visits was around a couple of tens of thousands a month.In the first quarter of 2012, over 50 thousand provided services were recorded.But, in the second quarter of 2012, a significant increase of registered visits arose and went over 700 thousand.In the third quarter of 2012 this number went down a bit, while in the fourth quarter, there were over a million registered records.
Also, each of these great lumps in number of registered services included a great lump in the number of users, as well as in number of departments that use MIS.For example, in April 2012 not only the number of services went up from 45 to 170 thousand, but the number of departments that use the system went up from 14 to 29, and the number of active users from 225 to 376.The biggest number of users in one month was 377, and it was registered in July 2013.After this we had the trend of decrease of users.In July 2014 the number of users went down to around 300 and stayed there until the end of 2015.The percentage of users who registered nonvalid visits was around 70%. Although, a great number of users, from time to time, made mistakes during the recording of provided services, the general trend was positive.
On the other hand, the trend for referrals and recipes was a bit different.Table 2 presents general statistics related to referral letters.Having in mind the total number of created referrals, the trend was going up.The biggest number of registered referrals was recorded in the last quarter of 2015.Also, the number of valid referrals was bigger, but their percentage went down ovrt time.During the system development, the number of supported types of referrals was increased from initial 9 to current 13.Table 3 presents distribution of referral letters by type.Even though the number of created referrals was different from type to type, the highest percentage of non-valid was among the categories that were least used, while within the categories that cover the highest percentage of crated entities the percentage of non-valid was 10%.Similar trend could be recognized for recipes.Table 4 presents general statistics for recipes.As the number of recipes went up, the percentage of non-valid was also increasing, similar like referrals.
Further, forms for physical examinations were addressed in this analysis as an addition to the sys-tem developed on the request of users in order to support one specific category of entity.Physical examinations could generally consist of few provided services.Table 5 presents data related to services that are usually registered through the form of phy-sical examinations.It is important to mention that it is possible for users to register any kind of service, even the physical examination, through the standard form for registration of provided services.The level of acceptance is the best with the physical examinations for school children.The lowest percentage is with adults, where the most various services were registered with the form of physical examinations.

Analysis and discussion
In the light of the TAM, registration of visits has been treated as PEOU.Users should find easy to use the forms for entity registration which they already are familiar with -in this case visits and pro-vided services (Figure 2).As already mentioned, the percent-age of successfully created provided services is over 99%.
The difference in percentage of non-valid provided medical services (changed and deleted) is significant if we look at the departments which have medical records and the ones that do not have (Table 6 and Table 7).In Primary Health Center Niš, a total of 7 departments have registration of services through medical records, while 33 departments do not have separate medical record.Also, within the 7 departments that have medical records, certain number of medical services is registered as direct medical services.Medical recording, from the point of view of TAM, could be identified as a significant external variable.Medical recording requires larger number of administrative operations than simple registration of provided medical services to patients.Users, who have the obligation to lead electronic medical record within MIS, have more contacts with the system and due to large scope of work they accept the MIS functionalities more quickly.In December 2015, 122 users recorded services that were registered using electronic medical record of the patient, while 298 registered services directly.Users that record services through electronic medical record of patient added on average 3264.71services during December 2015, with an average of 6.61 non-valid records.This made 0.2% of changed or deleted services.Considering the services recorded out of patients' record, for the same month, average number of recorded services was 1275.84where 13.02 were non-valid and that was 1.02%.At the beginning of MIS use, in January 2012, this percentage was 15.48% within the departments that had electronic medical records and 48.5% within the departments that did not have this type of recording.After three months this percentage went down below 5% within both categories.
Considering the recipes and referrals, the percentage of unchanged records is lower and it goes around 90%. Furthermore, after the initial percentage of valid records of over 95%, the increase of records, as well as the number of users that create these entities has lead to decrease of successfulness.Here, however, the records that are marked as invalid, and that could be excluded from the analysis (so called false positives) must be taken into account.We can divide them into following categories: -Recipes and referrals that belong to deleted provided services and visits, -Recipes and referrals created by coping the existing ones, -Non-copied recipes and referrals deleted during the forming of provided service or visit.When provided service or visit is deleted, that action necessarily leads to deletion of all related documents.In this way all related referrals and recipes will be marked as deleted, even though they were not directly deleted.Having in mind the fact that the number of this kind of services and visits is very low, the percentage of recipes deleted in this way is around 3.65% (total number is 18575, and the number of non-valid recipes is over 509 thousand, Table 8).
Copying of existing referrals and recipes into the new visit or provided service is the functionality created to accelerate the work of system users.The main purpose is to enable the prescription of the chronic therapy to the patient (prescription of recipes for continuation of existing therapy) or the creation of another referral for patients who suffer from chronic disease, in cases where it is necessary (Figure 3).The basic mode for use is that the user should choose one of the existing recipes or referrals from the list and copy it within the existing visit.If in this case the user decides to change some of the parameters after the copying, one non-valid entity will be created.This category gives around 10% of the total number of non-valid recipes.
Also, since the data are not physically erased from the database during the work in MIS, prescribed recipes and referrals, either they were erased or updated during the creation of new visit or provided service, will be registered as non-valid.These entities do not get valid protocol numbers until the moment when the complete visit is not saved, so we can ignore them from the total number of non-valid entities.This is the largest group of non-valid recipes and it actually presents one third of the total number of non-valid ones.
When we eject the above mentioned recipes from the total number of recipes, the result is that the percentage of invalid recipes goes within the acceptable 5%.During a couple of quarters, the number of changed and deleted recipes goes over 5% and it is not significant (Table 8).From the point of accepting technology, these are expected results since all the above mentioned categories are entities created within the well known processes and there was no need to explain to users any additional elements.Therefore all the observed functionalities that are classified into PEOU category are accepted in satisfying way.
As the representative of PU category we have chosen the overview of the physical examinations.During the system development and considering the demands of users, they were mainly focused on specific design of input forms and comparative view of values (Figure 4).The whole functionality was developed with the aim to be used instead of standard function for data input on visits for physical examinations recording.Generally, a user can input data on physical examination both through specific and standard functionality.Specific functionality was supposed to be the first choice in most of the cases in order to justify its PU nature.Usage results may vary from department to department, as well as from the type of physical examination.The best percentage of acceptability is within physical examinations (81.87%) and preventive (97.86%) examinations of school children (Table 5).Within the physical examinations of school children there were totally 24 various services registered, at it is important to mention the examinations related to school enrolment.Their percentage is lower 11.1% and 4.03% while the percentage of use with all other registered services is insignificant.From the remaining 20 services there are 520 records that are made.
Considering the physical examination of preschool children, the best indicator of acceptance is within the preventive examinations of children -86.21%.None of the remaining services does have the usage percentage more than 50%, and only 7 of them is on the level of 1% and higher.Considering the examinations of infants, percentage of usage for PU functionalities is 64% for preventive and 55% for physical examinations.Here we can notice great number of various services for which the functionality has been used, but with the small percentage.
For preventive and physical examinations of adults, the percentage of acceptance for PU functionality is the lowest -17.27%, or 10.44%.For physical examinations of adults there are totally 42 different services registered, but similar like with infants it has low percentage, less than 1%.
It is important to point out that these services are registered through various types of physical examinations; therefore in the overview given by the categories they have lower percentage of use for PU based functionality.For example, these services are Preventive examination of children from age 1 to school age and Physical examination of children from age 1 to 6 years.These services are registered through physical examinations of infants and pre-school and school children.There are totally 17746 registered preventive examinations, of which 15229 physical examination of pres-school children, 715 examinations of school children and 1234 infant examinations.It is totally 17248 from 17746 examinations.It represents more than 97% of all registered ones.For physical examinations the total percentage of registered ones is 55% (6023 out of 11129).
For registration of physical examinations through specific PU functionalities it can be concluded that the functionality itself has been used for large number of various services through each of offered forms, but only for couple of characteristic ones the percentage of use was significant.Users are usually using the above mentioned functionality for preventive and physical examinations of children, since they are carried out according to a predefined model and there is a detailed record about it.Unfortunately, there is no large number of registered physical examinations of adults, so the percentage of use for PU functionality is on the lower level than expected.

Conclusion
During the exploitation phase of the medical information system, very important activity is constant system improvement in order to achieve higher level of the efficiency and satisfaction of users.In order to improve MIS, it is necessary to conduct the data analysis on how the system is used after the appropriate period of exploitation.This paper shows the results of this analysis and adequate conclusions.
Considering the number of users and number of generated records we could be satisfied with the scope of system acceptance.The percentage of valid records is on the extremely high level considering the provided services (more than 99%), while the referrals and recipes are on satisfying 90%, or 95% when we drop out the records marked as false positives.It is confirmed that users do accept much easier and on a larger scale these functionalities which are considered easy to use, especially if they follow in detail the existing working processes.What we did not cover with this analysis is the quality of collected medical data.The validity of records has been measured only through users' actions aimed at their creation, deletion and updating.
Considering the acceptance of additional functionalities that are expected to provide additional sy-stem quality, on the example of physical examinations we have received high level of acceptance for those examinations which are more frequent and in larger number.The level of acceptance for physical and preventive examinations is much higher with physical examinations of children than with adults.Unfortunately, even the number of provided physical examinations of adults is lower due to the fact that there are many various examinations (our users have registered even 42 types of different physical examinations) and does not create the sense of need for use of special functionalities.
As already mentioned in (14), when accepting the new technology, the level of functionality acceptance, if presented through the acceptance model as simple, is very high.This is also significant because these functionalities are the ones that are often used and the high level of bad records would significantly slow down the work of doctors.In cases where users have the choice between basic and improved functionalities that presumably could contribute with their usefulness, the users will choose the other one only when the usage frequency is high enough or the improved functionality offers the obvious improvement of system performances.

Figure 2 .
Figure 2. Example of entering the provided medical service using the form for entering the new visit from the electronic health record of the patient

Figure 3 .
Figure 3. Copying the existing diagnosis, referral letters and recipes into the new visit/provided service

Figure 4 .
Figure 4. Comparative review of data obtained after the physical examinations.

Table 2 .
Statistics related to the referral letters

Table 3 .
Distribution of referral letters according to the type

Table 4 .
Recipes : A) month, B) year, C) total number of recipes, D) number of valid recipes, E) percentage of valid recipes, F) number of patients, G) number of users, H) number of different prescribed medications, I) number of different diagnosis, J) number of recipes marked as the recipes of specific importance, K) percentage of recipes of special importance

Table 5 .
Overview of using the forms for physical examinations: A) number of recorded examinations through specific form, B) total number of recorded examinations, C) percentage of examinations recorded through specific form

Table 6 .
Statistics for departments that have medical records: A) month, B) year, C) total number of registered visits, D) number of valid visits, E) number of departments that have medical records, F) number of various registred services, G) number of patients, H) number of MIS users, I) number of users with non-valid visits, J) number of registered services by user, K) average number of non-valid services by user, L) relative percentage of non-valid services.

Table 7 .
Statistics for departments that do not have medical records or have visit records out of the medical records.A) month, B) year, C) total number of registered visits, D) number of valid visits, E) number of departments that do not have medical records, F) number of various registered services, G) number of patients, H) number of MIS users, I) number of users with non-valid visits, J) number of registered services by user, K) average number of non-valid services by user, L) relative percentage of non-valid services

Table 8 .
Potential false positives for non-valid recipes: A) quarter, B) year, C) total number of non-valid, D) non-valid from erased services and visits, E) non-valid from copied entities, F) non-valid generated during the creation of provided service or visit, G) number of non-valid after taking out potential false positives, H) percentage of non-valid after deletion of false positives records.