Designing a Framework of Technical Hospital Information Systems Requirements and Evaluating the Systems Implemented through this Framework

Background Hospital Information System (HIS) implementation is complex and costly compared to other information systems. Therefore, the present study was conducted with the aim of designing and evaluating technical HIS requirements. Methods This study was conducted in 2016 in two parts: designing technical requirements and evaluating the HIS. The first part of the study was performed through the Delphi technique. In the second part of the study, evaluation of systems was done using a checklist based on the approved requirements. Results In the first part of the study, the final list of technical HIS requirements was designed with 73 items in four domains including ‘communication service’, ‘system architecture’, ‘security service’, and ‘system response time’. In the second part, the results of the evaluation indicated that the approved requirements had been observed in 63.8%, 65.5%, 72.4%, and 76.3% of the HIS software programs in the dimensions of ‘communication service’, ‘system architecture’, ‘security service’, and ‘system response time’, respectively. Conclusions The technical HIS evaluation tool was designed to be used in the evaluation and selection of the system. The evaluation results also indicated that the studied HISs were weaker in ‘communication service’ and ‘system architecture’ domains compared to the other two domains under study.


Background
Today, the advancement of Information Technology (IT) has changed the face of the world altogether and healthcare services are significantly supported through the emerging IT [1,2].Among those, HISs are one of the most commonly used health information systems [3].
Not only do these systems offer a tool for information management [4], but they also provide opportunities for rapid and precise processes, immediate access to information, economic savings, increased overall efficiency and even reduced clinical errors as well as improved treatment quality [5].One of the problems arisen from HIS implementation is lack of proper identification of technical requirements and integrity of these systems [6].
Therefore, a framework of technical HIS requirements and assessing HISs based on this framework is of utmost importance.
Despite potential benefits, HISs are neither widely used within healthcare providers nor readily accepted by users where they are launched [7].Polack (2009) reported the failure rate of Electronic Health Records (EHRs) as 30-50% [8].Implementing and accepting information systems in hospitals is relatively more complicated and costly compared with other businesses.Thus, a number of countries spend at least 2-6 percent of their health budgets on information and communication technologies [6,9].Given the fact that HISs gain acceptability rather slowly and the significant rate of failure among these systems, efforts need to be made in order to understand how to overcome the challenges against HIS implementation [10].A great deal of time and money may be wasted in healthcare systems unless these challenges are duly addressed [11].
Technical characteristics of information systems such as 'system architecture' type, 'communication service' of the system, and 'system response time' have a major role in determining the success or failure of a system [12].A number of studies have addressed technical factors as the second major factor [1] and even the biggest barrier to effective HIS implementation [13].According to another study, 35% of information system failures showed to be rooted in technical factors [14].Technical obligations help users ensure that a system is capable of communicating with other systems and guarantees information security and confidentiality [15].Technical aspects help network designers to identify technical features and the software version that supports these features; however, in some cases, these technical features may affect the choice of hardware platforms [16].
Given the strong link between technical factors and success rate of HISs [17], it is of great importance to establish appropriate evaluation criteria for solving technical barriers to the system implementation [18] inasmuch as accurate assessment of HISs is considered as an integral part the successful implementation of these systems [19].In this regard, much care should be taken to choosing both HIS software programs and software suppliers.The selected software need to be capable of meeting organizational goals [20].Therefore, the Request For Proposal (RFP), which forms the basic document for HIS implementation, must be prepared and approved by the executive team, end users and a number of system analysts [6].A couple of studies conducted to rate HISs have evaluated some technical requirements in these systems [21,22].Some studies have referred hardware, software, networking and technical support as the technical challenges faced during development of health information systems [11,23].
Regarding the program of Iran's Ministry of Health and Medical Education for the application of IT services in the healthcare sector and development of EHRs [24] and considering HISs as good examples of EHRs [25], implementation principles and the technical requirements of these systems need to be continuously assessed and revised [26].Therefore, the present study aimed at proposing a framework for the technical HIS requirements and assessment of HISs implemented based on these requirements.The requirements set out in this study can be employed by hospital executive teams and HIS developers.

Methods
The present study was conducted in two parts: designing technical HIS requirements and evaluating HISs based on these requirements.First Stage: Focus Group Discussion At this stage, 10 experts holding master's degrees or higher in software, networking, and IT with at least 7 years of work experience in hospital IT departments or the IT departments of medical universities were selected from among faculty members and the staff.FGD was utilized at this stage.During four 4-hours sessions, technical barriers and challenges of implementing HISs were discussed.The FGD pursued two goals: identifying the most important technical requirements desired by the experts and outlining all of the problems the experts had ever faced regarding technical HIS requirements during system implementation or execution.At this stage, the experts represented their ideas freely and addressed any technical HIS requirements through brainstorming.The experts' comments were recorded and then transcribed during the meetings.Finally, a list of technical requirements was compiled and, based on their views, the requirements were grouped into four domains including 'communication service', 'system architecture', 'security service', and 'system response time' and then the primary questionnaire on the framework of technical HIS requirements was designed.The questionnaire included introduction of the study, objectives and a list of the technical HIS requirements with an open-ended question in order to add up experts' comments to the list at a further stage.

Second Stage: Modified Delphi Technique
In the second stage, Modified Delphi Technique was employed to reach consensus on the requirements.Experts of this stage were the same experts participating in the first stage of the study.Modified Delphi Technique is generally used as a Group Delphi Technique to reach consensus based on experts' opinions through structured questionnaires.This method was a multi-stage process, which continued until consensus was reached.The

Modified Delphi
Technique was an open-ended round Delphi guided by the focus group or one-to-one interviews [27].At the end of the focus group, the initial technical requirements were formatted in the form of a five-point Likert questionnaire (Strongly Disagree = 0 to Strongly Agree = 4).Each technical HIS requirement was independently assessed by the experts and the assigned a score.The requirements with a final mean score of 3 and above were approved, and those with a final score of less than 2 were eliminated.Moreover, the requirements with an average score of 2-3 were reassessed by the experts to be either approved or removed from the questionnaire.The experts were encouraged to freely propose any new ideas about technical HIS requirements.They reached an agreement on the final list of technical HIS questionnaire after four focus group rounds.Finally, four groups of technical HIS requirements were accepted with 72 items and an average of 3.4 scores.

Third Stage: Classic Delphi Technique
In this stage, the questionnaire containing 72 items approved in the second stage was distributed among 40 IT experts and hospital and medical sciences university's managers across the country holding master's degrees or higher and at least 5 years of work experience in the field of HIS.The questionnaire included demographic information, 72 closed-ended questions and an open-ended question to explore the possibility of getting additional technical requirements.Participants were asked to express their opinion through a five-point Likert scale (from strongly disagree to strongly agree).At all rounds of the Delphi technique, feedback was anonymously provided to the participants.To ensure all Delphi characteristics (anonymity and feedback), research collaborators as another focal point were introduced to the hospitals under study across the country.They were responsible for familiarizing IT experts and managers with research goals and providing feedback on the data in each round.Out of the 40 distributed questionnaires, 38 were filled out.The mean score of each requirement was computed using SPSS software program (Version 18, SPSS Inc., Chicago, IL, USA) and then used for statistical purposes.This Classic Delphi Technique was done in two rounds.Finally, one item was added to the questionnaire and four groups of technical HIS requirements were approved with 73 items and an average of 3.6 scores.

Second Part: Evaluating HISs
Iran's Ministry of Health and Medical Education was inquired about the number of companies providing HISs and it turned out that the software was being implemented by 19 software suppliers throughout the country.Then, 19 suppliers were contacted for permission to assess hospital information systems out of which 16 suppliers gave the permission.Finally, a list of the hospitals which had been implementing HISs bought from these suppliers since at least 5 years earlier was provided.For each software supplier, one hospital was randomly selected which was running the HIS software program.If not all software modules of the supplier company were operational in the selected hospital, the hospital was eliminated from the study and replaced by another hospital through random selection.Finally, 16 hospitals were selected to evaluate HIS software programs.Using the technical HIS checklist containing Yes/No questions designed in first part, IT users and experts working at the information technology units of the selected hospitals were interviewed in person and the checklists were filled out.Next, the data was analyzed using SPSS software program (Version 18, SPSS Inc., Chicago, IL, USA).If hospital HISs met the checklist criteria, they were given a score of 1. Otherwise, a score of zero was given to the HISs.To show system status, the percentage scores were calculated.The percentage scores obtained for each domain were considered as very weak, weak, average, good, and very good for percentages of 0-20%, 20-40%, 40-60%, 60-80%, and 80-100%, respectively.

Ethical Considerations
Before starting the FGD round, all participants signed an informed consent form in which study objectives, voluntary participation in the FGD, and information confidentiality had been explained.Furthermore, in the second part of the study, participants were informed of study objectives before filling out the questionnaire.The voluntary nature of participation was explained, and participants were assured of information confidentiality and anonymity.

Results
In the Classic Delphi stage, 76.3% of experts were men and 23.7% were women.The mean age of participants was 34.7 ± 6.3 years, with a minimum and maximum age of 28 and 52 years, respectively.As for the level of education, 63.2% of participants held a bachelor's degree at the time of the study.The mean work experience of participants was 9.9 ± 5.3 years and their mean HIS work experience was 6.6 ± 1.7 years.
In the first round of Classic Delphi Technique, all requirements related to the domains of 'communication service', 'system architecture', 'security service', and 'system response time' assessed by the experts were finally approved.Regarding 'system architecture' requirements, one of the requirements did not gain the minimum score in the first round; hence, it was reassessed along with the proposed new requirements in second round of the Classic Delphi Technique and was then approved.Eventually, the list of technical HIS requirements was confirmed for the four domains containing 73 items with a mean of 3.6.These requirements existed in 68.9% of HISs implemented in the hospitals under study.
The results of the Delphi technique and evaluation of technical HISs requirements are presented in Tables 1 to 5 'Communication service' is one of the domains of technical requirements confirmed by the experts.Requirements related to 'communication service' showed a compliance rate of 63.8% in the evaluated HISs, which is considered to be at a good level.However, compared to the other domains under study, this compliance rate was lower.Additionally, the standard protocol approved by the national healthcare authorities for patient information exchange was only met in half of the information systems of the research community.The National Coordination Office of Health Information Technology of America, appreciates the need for standards, making incentives to use standards and a credible environment for interoperability, sharing and use of electronic health information due to the fact that such standards can prove useful in design stage of the development of a new system [28,29].Another study on the HIS implementation difficulties indicated that poor integration of various systems is a potential source of problems [30].
In his study, Feko stated that HISs in Hungary are capable of communicating within hospitals.Nevertheless, the ability to inter-communication outside the hospital is limited [31].Surveying the acceptability of health information technology in 7 industrial countries suggested that, despite the high number of EHR, they still lag behind the health information exchange [25].Several other studies have pointed out that the level of interoperability among HISs is disappointingly low, and they have scored low in the assessments [32][33][34].The lack of interoperability restricts care provider organizations to use product portfolio of a software supplier in a way that their data is stored in the supplier's specific format and cannot be easily transferred to another system [35].
Ultimately, lack of interoperability among HISs results in an increased load of paper [36].
In a study, users believed that working with integrated systems would reduce working hours and increase work speed [6].Not only is the capability of system information exchange effective in indirect nursing care [37], but it also leads to reduced patients readmission [38].Failure to exchange information among various HISs imposes financial burdens on hospitals and causes failure to refunding the costs to Healthcare institutions [39].Furthermore, information exchange among these systems helps to reduce the cost of transmitting laboratory reports, imaging and clinical communication among healthcare providers [40].As Walker et al. (2005) stated, investing in Electronic Medical Records (EMR) interoperability and health information exchange could save $77 annually [41].A unified and focused EHR approach for each patient means to have regional Electronic Patient Records(EPRs) that provides a degree of internal exchange capability [42], although the main features of the fourth and current HIS generationintegration of HISs-is known as integrity with other systems [6].The specific framework for 'communication service' requirements of HISs including the standards of information exchange at system design phase need to be given serious attention.What follows can highlight this importance, a) considering the importance of information exchange capability in information systems, b) the need to develop interoperable systems, c) prevention of organizational dependence on the products of a particular company, d) the impact of lack of data integrity on treatment and cost addition, e) the heterogeneous software forests that does not meet the goal of creating and implementing an EHR.
'System architecture' is another important domain of technical requirements.Based on the findings, compliance with the requirements of 'system architecture' in the assessed HISs was proper (65.5%).However, it has achieved a third rank in comparison to the other domains of technical requirements.In software development, 'system architecture' determines system development model, environment development, and system development tools [43].Multiple distributed systems, developed in various programming languages, demand more mechanisms for practical communication in engineering perspective [44].In this regard, service-oriented architecture is a tool for defining IT infrastructures that allow various software programs to exchange data and cooperate in the process of business, regardless of the types of operating systems or programming languages through which software programs are designed.Moreover, the messaging standard enables diverse healthcare software programs to exchange office and clinical data and run it on the web [45].Systems that are based on varied platforms and programming languages appear to create numerous heterogeneous systems in an organization, which makes it difficult to develop HIS projects.Hence, the possibility of using open source tools, web-based implementation, and supported standards is part of the framework of 'system architecture' requirements which are helpful in future software development.
'Security service' is also addressed as another domain of technical requirements, which has been well met in evaluating information systems of the research community and can be regarded as good (72.4%).Security issues are among the hindering factors of using and implementing information systems in the healthcare sector [46].Furthermore, several studies have highlighted security concerns of EHRs [47][48][49].In other studies, the level of technical security in some health centers was reported as strong and in some hospitals as moderate [50].In yet other studies, more than half of respondents expressed their health security data at a good/moderate level [33,51].Previous research shows neither a specific policy for patient access to information nor punishment for unauthorized access to information.While each system user has a specific password, there is usually no expiration date for these passwords, and thus, system users can access them with no time limitation [50].The system must have security mechanisms for controlling user access in terms of tasks or classified access [52].An effective access control system ensures that system users cannot access system resources unless they are permitted to do so [7].On the other hand, HISs should provide easy access to healthcare information at the caring institution [53].A vast number of user-friendly techniques including setting passwords, biometric techniques, smart cards, and certifications are widely used to guarantee data confidentiality and user validation [54].Typically, these systems attempt to prevent illegal operations before they occur [55].In a study investigating the priority of nursing data security requirements in EHRs, making a backup was introduced as the highest priority of majority of IT experts [56].Providing encrypted, incremental backups of information is one of the security measures for EHR protection [57].Considering the fact that patient data security management using the HIS influences care quality, the rights of patients and healthcare professionals and their working practices [58], patients may conceal their information due to lack of trust in the security of HISs, which is healthdamaging [59].Thus, recognizing security features and privacy in HISs is of great importance and, in the event of facing such risks, some data protection measures need to be taken [60].Hospitals should follow a standardized set of instructions to enhance information security [61,62].The level of security seems to vary from one study to another.Regarding the important role of data in providing patient care, data backups and data access control are known as the major security concerns of the systems.Therefore, a security framework which guarantees system security and makes information access easy by authorized individuals is required where HISs are being implemented.
Another domain of technical HIS requirements approved by the experts is 'system response time'.Based on our findings, 'system response time' was at a good level (76.3%) in the HIS evaluation.'System response time' is one of the major technical challenges of the HIS [30,63].Numerous studies have highlighted system response dissatisfaction, and referred to it as an obstacle for EHR application and the main reason for its abandonment by users [63][64][65].In organizations which are heavily dependent on computerized systems in patient care, the downtime should be 0% or a little above 0. Yet, in these organizations, business continuity procedures should be in place to ensure safety and patient care continuity [66].Therefore, one of the general technical aspects of the HIS is timely system access at a high speed, which is identified as an important factor in the efficiency of HISs [52,53].Low 'system response time' reduces the chances of system acceptance by users and successful implementation of the system [67].Therefore, in order to increase system efficiency and consequently user satisfaction, it is recommended to highlight the importance of 'system response time' requirements while choosing a system.

Conclusion
Technical HIS requirements were designed with 73 items in four domains including 'communication service', 'system architecture', 'security service', and 'system response time'.These requirements can be used in the lifecycle of information systems while customers attempt to choose systems so as to meet minimum user requirements for using HISs.These requirements can help designers and developers of HISs to adapt the software program to the user needs.Furthermore, HIS evaluation of the research community suggests that, in spite of a good score in the evaluation, in technical HIS requirements, the domains of 'communication service' and then 'system architecture' were weaker than the other ones.Hospitals are constantly developing their HISs, which leads to the emergence of many heterogeneous systems.Integration of these systems is also difficult to achieve since each one is designed and implemented in accordance with its own principles.In addition, patient data is stored in these systems and their transfer to new systems is not an easy task.Therefore, this can be a major challenge and obstacle in establishing interoperability between different systems for creating EHRs and developing systems in accordance with changes and needs of organizations and users.

List Of Abbreviations
IT: Information Technology MF made substantial contributions to the conception, design and acquisition of funding.
MSJ and RDB participated in data collection and performed the statistical analysis.ZM and EN contributed to manuscript drafting, revision, and approval, and MF was responsible for the general supervision of the research group.All authors read and approved the final manuscript.

First
Part: Designing technical HIS requirements The research was conducted using Delphi technique in three stages: Focus GroupDiscussion (FGD), Modified Delphi Technique, and Classic Delphi Technique.The details of each stage are as follows:

.
Table1.Mean score of technical requirements and HIS evaluationAs presented in Table3, 'system architecture' with 27 requirements and the mean score of 3.53±0.37wereconfirmed, which met 65.5% of implemented HISs.Table3.Mean score of 'system architecture' requirements and HIS evaluation 'Communication service' requirements including 10 items are shown in Table 2. Delphi score regarding the importance of 'communication service' was 3.49±0.38,and the assessed HIS met 63.8% of the 'communication service' requirements.Table2.Mean score of 'communication service' requirements and HIS evaluation

Table 4 ,
'service security' with 31 requirements and the mean score of 3.5±0.39were approved, which was observed in 72.4% of the implemented HISs.
rate of the requirements in the hospitals under study was considered as 'good' (68.9%).