Approach to the Methodological Assessment of the Performance of Information Systems at Libyan Universities (Based on Multiple Case Studies)

The CD-ERP model describes an approach to the collaborative development (CD) of information systems based on the framework of an Enterprise Resource Planning (ERP) system. This approach is proposed to solve the problem of IS development in Libyan Higher Education (LHE). This paper presents an assessment of IS performance at three Libyan Universities. The findings from these cases are analysed and discussed on the basis of selected models. This analysis indicates that the level of both the ISs implemented in LHE and the capability for in-house system development is low.


Introduction
As in other universities worldwide, Libyan Universities are purchasing or developing information systems (ISs). Both approaches have shown negative results. Alternatively, the CD-ERP approach has been proposed for Libyan universities. This model is based on two main principles, namely: ERP packages and communitysource applications. The latter involves Libyan higher education (LHE) institutes jointly developing ISs. Community-source development, which is referred to as the collaborative development approach in this article, can be defined as an open-source project which is governed by a consortium of institutions [1]. Similar projects are found in many countries, including some EU countries and the USA. Besides its potential benefits, ERP is used as a framework in order to avoid rebuilding systems from scratch. To consider the applicability of the CD-ERP model in LHE, similar international projects and the Libyan context

Related Studies
The need to assess IS performance has emerged from the increasing importance of IT in determining the effectiveness and efficiency of business processes in an organization. Over recent decades, authors have addressed the evaluation of IS performance, e.g. DeLone and McLean, Bailey and Pearson, Avison and Fitzgerald, Burch and Grudnitski (for an overview of these approaches see [3]). There is a lack of research on the effectiveness of IT in higher education, particularly in LHE. Bakeer and Wynn investigated Misurata University (MU) as a case study [4], unlike our study in which several universities were studied. Also, they treated ISs and e-solutions alike, while in this article they are treated separately, which provides a much wider view as will be shown in the analysis section. IS deployment in Libyan oil companies has been assessed using similar methods in [5]. The studies mentioned investigated IS deployment from a managerial perspective, unlike this study, whose aim is to assess universities' capability for building inhouse systems. Also, neither of these studies were carried out in two phases (initial and formal studies), as in this study.

Theoretical Framework
The theoretical framework grounds a study firmly in theoretical constructs [6]. Using multiple cases helps to ensure that conclusions are based on comparable outcomes and common patterns rather than a generalization made from what could be one-off chance occurrence [7]. Based on the findings from the initial study on UOT, process mapping and system profiling were used in order to provide a more detailed representation of the current status of the ISs implemented in the three universities studied. The methodology used in this study can be summarized as follows: the authors carried out multiple case studies, where the subjects were the University of Tripoli (UOT), Misurata University (MU) and Sirte University (SU). An inductive approach based on the collection and analysis of qualitative data to develop a

International Journal of e-Education, e-Business, e-Management and e-Learning
theory is appropriate for this study. Interviews with experts in each university were used to gain information on the level of ICT with an emphasis on the deployment of ISs. In order for data collection in these case studies to be consistent, the model of business activities in HE institutes presented by Zornada & Velkavrh [8] was adopted, as shown in Fig. 1. Accordingly, business activities were classified into a) educational activities and administrative support for the education process, b) research activities and administrative support for the research process and c) other activities, such as human resources, finance, etc. All three types of IS are found in each university. UOT was investigated in an initial study. Later, the formal study took place. All the data gathered were analysed using the system profiling and process mapping of ISs, Nolan's model, Zuboff's model, and the CPIT model.

Subjects of the Case Study
Three Libyan public universities were studied, as follows: Case Study 1 -UOT (Initial Study), Case Study 2 -MU, and Case Study 3 -SU. These universities were chosen because 1) they are all Libyan public universities, 2) they are all located in the same region (Tripolitania Region) under the same regional government, 3) they all require ICT development, and 4) they have different histories. UOT is a longestablished university, SU is a relatively newly-established one, while MU's history is of intermediate length.
The choice of these universities gives a more comparative view, as will be discussed later. A description of the respondents is given in Table 1. In all of the case studies, the respondents were first introduced to the study via e-mail. They were then contacted by telephone during the first session of the interviews. After this, a list of the additional information needed was e-mailed to them. Eventually, face-to-face oral interviews took place. The length of these interviews was between 35 minutes and 2 hours. The participants from UOT were interviewed at the end of 2018/beginning of 2019. The other interviews were held six months later (June-August, 2019). Afterwards, the participants from UOT were contacted by e-mail to check whether any changes or updates had occurred in this period.

Sample Size, Data Collection, and Recruitment Strategy
Sample size assessment is used to determine the minimum number of participants necessary for the study. Based on the information gained from the respondents in the initial study [2] and the structure of the IT sections in the universities, the author determined the minimum number of respondents in each university to be 2 (an expert in ICT and an expert in ISs). Convenience sampling was used due to the specific nature of the population, individuals who are working on/familiar with ISs, i.e. a "niche" population. Data collection was based on the concepts of snowball sampling and respondent-driven sampling (RDS) [9], [10]. Each respondent was asked to give an estimate of the size of their network. This helped to decide whether to use interviews or questionnaires.

The Findings
In terms of the status of IS deployment, despite their differences, the three universities studied share common characteristics. These can be summarized as follows:  Generally, the level of IT in Libya is low, which should be considered before adopting advanced systems. Although all the respondents agreed on the low level of ICT in general and the ISs deployed in particular, some points were mentioned as strengths on the basis of the current status.  In each university, there is an office which handles information technology and communication issues. Also, the structures of these offices are similar.  In all of the universities, there are three main categories of ISs deployed in each type of business process, as itemized below: a) In-house applications developed either locally or by a university's IT teams using a variety of technologies and platforms. This makes it impossible to integrate them into a single connected architecture. All of these applications are out-of-date, not well-documented and have many flaws. b) Applications purchased from local vendors and mostly maintained by the vendor itself. In some cases, there are open-source applications that can be modified by the local IT team. However, those applications are not easy to maintain or develop. Again, due to the variety of technologies used, these applications are impossible to integrate. This problem is magnified by the fact that some of these applications are not editable. c) Standalone office automation packages (MS Excel/Access) are used to support many activities in all three universities. In some cases, these applications are used alongside ISs. The integration of these systems is limited to manual communication. There is no bridging software. Data are transferred in a format such as Excel, CSV, database files or, in some cases, re-entered manually. Such data transfer can be risky and requires a professional level of developing skills.
 Although top management appears not to be satisfied with the existing ISs, they are not ready to invest more in IT. Top management is convinced that most business and sub-business activities need applications based on the latest technology, but is worried about the consequent costs.  There has been a noticeable improvement in the reliability of information gathered in business activities due to the use of ISs. This is reflected in better decision-making. However, this improvement is limited, since the reports generated from the ISs are not available as quickly as required by top management. Most of them are transferred in paper format or by converting them into other files, such as MS Excel or Access.  Highly-qualified staff are required, which is problematic within the university environment. In fact, the current development teams are unable to trouble-shoot all of the systems currently running, which is reflected in the low expectations regarding the results from using these systems.  Paperwork based on forms is still used prior to data entry into the ISs deployed.

Analysis and Discussion
This section presents an analysis of the findings and further discussion on the basis of selected models. These techniques and models were chosen taking into account the technological and organizational differences between Libya and the countries in which the models were originally developed and tested. Analysis based on the following models provided the authors with an easy way to compare ICT levels, and in particular the ISs deployed: 1) System profiling and process mapping of ISs

System Profiling and Process Mapping of ISs
In UOT, among the three categories of business activities, the most advanced ISs systems are deployed for educational activities. However, only the Student Information System (SIS) has a fully functional system. Office 360 is the only system implemented in UOT that is rated as class I, which indicates that the system is functional and effective. Fewer systems are deployed in both MU and SU and these were most often developed by a third party (commercial applications). The issue of duplicated systems is only seen in the SIS at MU. Moreover, MU also lacks any kind of research IS, as is the situation in SU. On the other hand, MU deploys the most advanced ISs to support other kinds of business activities (non-educational and nonresearch). In SU, the only IS ranked as class II (functional, but needs to be enhanced) is the Libraries system, which was developed in-house. Compared to UOT and MU, other kinds of business activities are supported by out-dated systems. Indeed, there is no such system at SU ranked above Class III (replacement should be considered). SU had neither developed in-house nor purchased any IS for any business activity in this category. Fig. 2-Fig. 4 summarizes the process mapping for ISs in each of the universities studied. The

International Journal of e-Education, e-Business, e-Management and e-Learning
ratings given in these figures correspond to the following states: I) functional and effective; II) functional, but needs to be enhanced; III) functional, but replacement should be considered; IV) functional, but needs to be replaced; V) no longer functional; and VI) not computerized at all. Fig. 4. System profiling and process mapping of the ISs deployed in SU.

Analysis of the Overall Level of ISs in all the Universities -on the Basis of Nolan's Model
Based on [11], UOT showed the highest level of ISs deployment among the three cases in both online and offline versions. The effort made by UOT is relatively equal in the three kinds of business activities (Educational activities, Research Activities, and Other activities) although ISs deployed for educational activities is the only one considered to be transformed into a fully-integrated system. This is confirmed by the combined Fig. 5 below that shows the overall analysis of the growth of IS deployment at three cases based on Nolan's model. In fact, UOT is the only case that passes the control stage. Indeed, serious actions have been taken to integrating all systems deployed in UOT which is not seen in the other two cases.

Business Activities
Automate measures of technical aspects of IT (rate of information flow, accuracy, timeliness)

IT reports and project implementation
Transformate perceptible improvements in service, intangible benefits (trust, loyalty, brand, etc.)

Misurata University (MU)
The implementation of educational activities and administrative support for the education process Both in-house applications and purchased suites (supported by MS Office) are found at MU to serve student management such as course registration. Other activities are accomplished manually in many parts of MU.
Reports are only available for those educational activities supported by ISs. These reports are accurate. However, due to being prepared manually via data transfer to MS Office, they are not available on demand.

The implementation of research activities and administrative support for the research process
There is a lack of an integrated IS for research at MU. Isolated systems are available, such as an archive system for books, periodicals, publications, and research in each library at MU.
Partially reliable information is obtainable from reports generated by the websites of journals published by the university.

The implementation of other business activities, such as ISs for HR management, sales, finance or documentation
Some systems are available, such as a HRIS for the administration of staff, while the finance affairs system is considered to be the best of all the systems.
Reports are obtainable from the financial IS. Top management uses this information to prepare the annual financial report.

Sirte University (SU)
The implementation of educational activities and administrative support for the education process SU possesses purchased applications (supported by MS Office) that serve student management such as course registration. Other activities are accomplished manually in many parts of SU.
Reports are only available for those educational activities supported by ISs. These reports are accurate. However, due to being prepared manually via data transfer to MS Office, they are not available on demand.

The implementation of research activities and administrative support for the research process
There are no ISs for research at SU. Work is done either manually or using MS Office.
Partially, and in most cases unreliable, information is obtainable from reports generated by MS Office.

The implementation of other business activities, such as ISs for HR management, sales, finance or documentation
There are no ISs for other business activities at SU. Work is done either manually or using MS Office.
Unreliable information is obtainable from reports generated by MS Office.

Analysis of the ISs Implemented -on the Basis of Zuboff's Model
The concept of "automate, informate, and transformate" was introduced by Zuboff to describe the impact of information technologies on organizations [12], [13]. Fig. 6 presents the analysis based on Zuboff's model. As discussed in [2], educational activities and administrative support for the education process are the only processes among the university's core functions to be facilitated by advanced ISs. Despite this, enhancement of a number of current ISs and replacement of other ISs are needed to fully "transformate" the servicing of educational activities. In the cases of support for research activities or for other business activities, it is clear that the requirements of "transformating" all the related activities to being electronic are not satisfied. In MU, most business activities are supported by an IS producing reports that are available to top management. In particular, financial management is the only one of the university's core functions to be facilitated by an advanced IS. However, enhancement of a number of current ISs and replacements of

International Journal of e-Education, e-Business, e-Management and e-Learning
other ISs are needed to fully "transformate" the servicing of financial affairs. Support of both research activities and other business activities by ISs is still at a very early stage. In SU, most business activities are supported by an IS that makes reports available to senior management. Also, only educational activities are facilitated by an advanced IS. Support for both research activities and other business activities are at an early stage that does not meet the requirements of "transformating" all the related activities to being electronic.

Analysis of the Level of e-Solutions Using the CPIT Model
The CPIT model was developed by the UK Department of Trade and Industry in order to evaluate the impact of e-business technologies across an organization's main business processes [14]. As discussed in [2], the level of adoption of e-solutions for educational services seems to be beyond the level observed in research and other business activities. Interactive content is available to students and staff via the online SIS system. However, improvement is needed, as stated by each of the experts participating in the study. None of the types of business activities are close to being served by a fully transformed system, which is represented by the "transform" aspect of the CPIT model. Interaction is not possible based on the online systems available for other business activities, as opposed to the systems implemented for educational and research activities. In MU, although the financial affairs system is seen as being the most advanced system, there is no e-solution for financial affairs. However, the student portal has attained "interact" status. This confirms the authors' choice to differentiate between standalone systems (offline) and e-solutions (online web-based). The offline financial affairs system is more advanced than the online system. On the other hand, the online student services system is the only interactive system in MU and more advanced than the online system. Again, no type of business activity is close to being served by a fully transformed system, which is represented by the "transform" aspect in the CPIT model. At SU, the SIS is seen to be the most advanced system. There is no e-solution for financial affairs. Moreover, no type of business activity is supported by an interactive system, which is represented by the "interact" aspect in the CPIT model. Fig. 7 presents the overall analysis of the adoption of e-solutions in all three universities on the basis of the CPIT model.

Sirte University (SU)
Misurata University (MU) Fig. 7. Analysis of the Level of e-solutions using the CPIT Model (in the case of UOT, see [2]).

Conclusion
Based on three case studies involving Libyan universities, the authors have analysed the level of ISs implemented in Libyan higher education using selected techniques and models, namely: system profiling

International Journal of e-Education, e-Business, e-Management and e-Learning
and process mapping of ISs, Nolan's model, Zuboff's model, and the CPIT model. These techniques and models have given a comparative view of the current level of ISs deployed in the three universities studied. This analysis has determined the level of progress and where opportunities exist at the level of individual processes. In addition, treating ordinary ISs and e-solutions separately has provided more detailed results. For example, MU shows the highest level of offline ISs implemented in other business activities (noneducational and non-research), while UOT has higher-level online systems in the same category. There are three main categories of IS implemented in each university, namely: in-house applications, purchased suites, and standalone office automation packages (MS applications, AutoCAD). UOT showed the highest level of IS deployment overall for both online and offline versions. The level of development of ISs at UOT is relatively level over all types of business activities (educational, research and other activities). MU has developed more ISs for other activities, such as financial affairs, real estate, sales, and warehousing. SU has the lowest number of ISs deployed among the three universities. In each of the universities, the level of the ISs deployed for the support of research activities is very low, with UOT possessing the highest level of ISs of this type. Issues of duplicated systems are seen in UOT and MU, especially with regard to the student information system, while SU does not experience such issues. This may be due to the fact that SU is a relatively newly-established university compared to the other two. The modern technologies that appeared after the establishment of SU may have played a vital role in developing more reliable ISs. In fact, S-2 stated that duplication is a major problem in all Libyan public universities, except for SU and the University of Benghazi. Cooperation in IT has already been established between Libyan public universities, since UOT had been considering adoption of the SIS developed by the University of Benghazi before developing their current SIS by themselves. Also, Al-Mergib University and Bani Waleed University recently adopted the SIS developed by UOT. The contract includes training IT staff in these two universities to use the source code, e.g. to troubleshoot the system.

Limitations and Future Work
This study has shown the low level of ISs implemented in Libyan universities, where most systems are in need of replacement or enhancement as shown in Fig. 2-Fig. 4. The authors tried to include as many universities as possible in the study based on the research schedule. Due to the instability in Libya, the authors were unable to include more cases. It is thus recommended to carry out comprehensive interdisciplinary examinations and more case studies, both in preparation for and during the implementation of a new collaborative project to modernize ISs in LHE.