E-Service Quality Evaluation on E-Government Website: Case Study BPJS Kesehatan Indonesia

This research intends to develop a model to evaluate the quality of e-services on e-government. The proposed model consists of seven dimensions: web design, reliability, responsiveness, privacy and security, personalization, information, and ease of use. The model is used to measure the quality of the e-registration of BPJS Kesehatan, an Indonesian government health insurance program. The validation and reliability testing show that of the seven dimensions proposed, only four that suitable for the case study. The result shows that the BPJS Kesehatan e-registration service is good in reliability and responsiveness dimensions, while from web design and ease of use dimensions the e-service still needs to be optimized.


Introduction
Indonesia is one of the developing countries that already used e-government to improve the quality of its existing services. Implementation of e-government is divided into five stages: the existence of web, interaction, transaction, transformation, and e-democracy [1]. Nowadays, Indonesia e-government implementation has reached the third stage [2]. The third stage indicates that the Indonesian government has been using online transactions to improve its services [1]. Such online transaction, also known as e-service, has been widely adopted by various government sectors in Indonesia.
The implementation of e-services is expected to change the traditional transaction processes that take time and resources, to become more modern and practical. The government should not only maximize the benefits offered through services owned, but also should reduce even avoid the loss generated if the government cannot adapt to the rapid technological change [3]. With the increasing number of e-services that are starting to be used by various government sectors, it is important for developers to pay attention to the quality of e-services they developed.
Assessing the quality of e-services in e-government has attracted the attention of many researchers in the past few years [4,5,6]. In [4], e-GovQual, a multiple-item scale for assessing e-government service quality was proposed, while [5] modified an already existed model (SERVQUAL) to assess the quality of e-services in e-government. Another work by [6] conducted a study to determine what factors affect the quality of e-services in e-government sites in Oman. In Indonesia, there are a few studies that assess e-service on e-government, such as the research conducted to assess the effectiveness of e-service on local government of Serang [7]. However, the literature related to the quality of e-services especially in Indonesia's e-government is still rare.
One of the Indonesian government sectors that have already implemented e-government is BPJS Kesehatan (Badan Penyelenggara Jaminan Sosial Kesehatan), a government health insurance program. The law states that every citizen of Indonesia is obliged to become the member of BPJS Kesehatan. It is reported that in 2010, Indonesian population has reached 237,641,326 1 and this huge number of people are expected to join the program. The program provides two ways for citizen to register, by coming directly to one of the branch offices of BPJS Kesehatan or by using the e-registration service 2 . Due to the huge number of citizen that needs to register, the role of the e-registration service is very important in helping people that have no time to come directly to the respected BPJS offices. This eregistration service should have good quality so that people from different range of social and academic background can use it easily. The objective of our research is to propose a model to evaluate e-service quality and then implement the resulting model to measure the quality of e-registration of BPJS Kesehatan. This paper is organized as follows. Section 2 describes the previous researches in assessing the quality of e-service. Section 3 discussed the research methodology. Our proposed model is presented in Section 4. Section 5 describes the results and analysis. Finally, we discuss about the conclusion in Section 6.

Related Works
This research was conducted based on three previous studies on e-government service. The first study [5] modified an already existed model (SERVQUAL) that evaluates the quality of e-service in general. The research found that there were five dimensions on the model that needs to be transformed. The five dimensions are website design, reliability, responsiveness, security and privacy, personalization, information and easy to use. There were also new dimensions that need to be added to make the instrument able to evaluate e-government service quality: information and easy to use.
The second work [4] developed and validated an instrument to evaluate which factors that affect citizen's perception upon e-government service. The result of this research was a multi-item instrument named e-GovQual. The result of this research shows that efficiency, trust, reliability, and citizen support are important when evaluating the quality of e-government service.
The third work [6] determined which factors (or dimensions) are critical when evaluating egovernment service. The number of this research respondent was 324 participants who are familiar with e-government site in Oman. The study shows that efficiency is the most significant factor, followed by security, reliability, and responsiveness, when assessing the quality of e-government service.

Methodology
This research used five steps: Research Design, Instrument Evaluation, Hypothesis Definition, Data Collection, and Data Processing.

Research Design
In this step, all the previous models are analysed in order to define a new model. The new model was defined by combining all dimensions and attributes from previous researches. The attributes that have similar meanings but have different name in previous studies are represented by only one of those attributes. On the other hand, the attributes that have different meanings but still in the same context, was added to the proposed model. Meanwhile, the attributes with a different meaning and different context were removed.
Based on the resulting proposed model, a questionnaire was created and a scenario in using eregistration service was made. Our respondents are asked to do the scenario before filling the questionnaire.

Instrument Evaluation
This phase tested the proposed model that was made at research design phase. First, we conducted a readability test to evaluate whether the questionnaire is easy to read and understand. Second, we performed validity and reliability test. The evaluation produced a final questionnaire that was valid and reliable for later use in data collection. The validity test used Corrected Item-Total Correlation to see the validity of each attribute, while the reliability test was used to test the reliability of each dimension using Cronbach's Alpha (CA).

Hypothesis Definition
We conducted two-tailed hypothesis testing for one-sample case for each attribute on every dimension. For each hypothesis testing, we defined null hypothesis and alternative hypothesis. Below is an example of hypotheses for a variable.

Data Collection
We used purposive sampling technique to choose 60 respondents. They were students of Universitas Indonesia who have never used e-service on BPJS Kesehatan website. The respondents were asked to follow premade scenarios and preceded to fill out the questionnaire. The survey was conducted face to face, thereby reducing fraud and bias in the collection of data.

Data Processing
On this phase, the collected data was processed based on the attribute normality. If an attribute was found normal, then its hypothesis would be tested with t-test for one sample case. If an attribute was found not normal, then its hypothesis was tested using the Wilcoxon Sign Rank Test for one sample case. The tool that used for data processing was IBM SPSS Statistics 24.0 software. We have defined the interpretation of mean value into the respective quality label. The details of the interpretation is shown in Table 1.

Proposed Model
This model was the result of the research design phase. Our study used a research model proposed by [5] as the main model for the instrument. They proposed that in assessing the quality of e-services in e-government, there are seven dimensions that need to be reviewed, namely website design (site design), reliability (reliability), responsiveness (responsiveness), security (privacy), personalization (empathy), information (addition), and ease of use (addition). Every dimension and its attributes on the main model will be combined with two other studies: [4] and [6]. Our proposed model consists of seven dimensions and 35 attributes. The details of proposed model can be seen in Table 2.

Result and Analysis
In this section, we will see the result of each analysis that was done on this research.

Validity and Reliability Testing
An instrument was considered valid when the value of Corrected Item-Total Correlation is greater than r-table testing [8]. For our data, the threshold is 0.361, based on the number of respondents involved. The level of significance used was 0.05 (5%). As for the reliability test, our research used ranged value to qualify each dimension. The conditions were: CA under 0.6 was rejected, above the 0.7 figure was accepted and classified as reliable, and dimension with CA above 0.8 was also accepted and classified as very good.
From the seven dimensions, three dimensions (web design, reliability, ease of use) were found reliable and acceptable because the CA value was inside the range of 0.7 to 0.8. The responsiveness dimension was found reliable and questionable because the CA value was inside the range of 0.6 to 0.7. However, one studies found that a CA value depends on the number of items that have a dimension [9], so that the dimension of responsiveness remains into the final questionnaire. Other three dimensions were not reliable: privacy and security, personalization, and information. The reason was because the amount of valid attributes in each dimension is not sufficient for its dimensions to be calculated of its reliability. The explanation why these three dimensions were not suitable for eregistration of BPJS Kesehatan case study was because the research is limited to a specific e-service in e-government that is not so related with those three dimensions that made the respondents confused in answering the related questions.
In the end, from the seven dimensions and 35 attributes of the proposed model, only four dimensions and 15 attributes are valid and reliable to be used in assessing the e-registration quality of BPJS Kesehatan. The detailed result is shown in Table 2.

Hypothesis Testing Result and Analysis
Details of the hypothesis testing results can be found in Table 3. We can see that there are only three variables that its respective null hypothesis is accepted. In other words, we accepted the hypothesis that the mean for those variable are 3 that we labeled as "average" quality. For the rest 12 variables, null hypothesis is rejected, so we conclude that the mean is not equal 3. Based on its mean value, then we decide the quality of each variable. There are two variables that categorized as "poor" and ten variables are considered "good". If we observe into each dimension, web design dimension of BPJS Kesehatan has only one of four attributes that considered as "good". Since the majority of attributes are not good, we conclude that the evaluation result for this dimension is not good. For reliability and responsiveness dimensions, since all attribute qualities value are good, we conclude that the evaluation of these two dimensions are good. For the ease of use dimension, since only one of three attributes is good, we conclude that the evaluation of this dimension is not good.

Conclusion
This research has proposed a model to evaluate the quality of an e-service on e-government. The model was defined by analyzing three related researches on e-government service. The model consists of seven dimensions: web design, reliability, responsiveness, privacy and security, personalization, information, and ease of use. Under those seven dimensions, there are 35 different attributes. The proposed model had been used to evaluate the quality of e-service of BPJS Kesehatan, especially its eregistration. The validation and reliability testing show that of the seven dimensions and 35 attributes proposed, only four dimensions and 15 attributes that suitable for the case study. The result shows that the BPJS Kesehatan e-registration service is good in reliability and responsiveness dimensions, while from web design and ease of use dimensions the e-service still needs to be optimized.
The government can use heuristic evaluation to improve those aspects. The e-government service still needs much to be explored, especially in Indonesia. This is important because the government service is important in making the citizen satisfied. We hope by doing this research we can inspire other research to look more on e-service and its quality.