“Assessing citizen adoption of e-government initiatives in Gambia: A validation of the technology acceptance model in information systems success”. A critical article review, with questions to its publishers

This is a review of the article ‘Assessing citizen adoption of eGovernment initiatives in Gambia: A validation of the technology acceptance model in information systems success’ by Fengyi Lin, Seedy S. Fofanah and Deron Liang (henceforth called the authors) from April 2011, https://doi.org/10.1016/j.giq.2010.09.004. The article has three focus areas: (1) e-government, (2) Gambia and developing countries' situations, and (3) the TAM (Technology Acceptance Model), paired with IS (Information Systems). Drawing on these areas, the authors want to make the following case: when designing e-government initiatives, taking into consideration the core constructs of TAM [perceived ease of use (PEOU), perceived usefulness (PU), information quality (IQ) etc.], will help improve the acceptance of technology by the population. The authors test hypotheses by feeding data from a questionnaire survey among Gambians into a model and by calculating relationships and probabilities.


Introduction
This is a review of the article 'Assessing citizen adoption of e-Government initiatives in Gambia: A validation of the technology acceptance model in information systems success' by Fengyi Lin, Seedy S. Fofanah and Deron Liang (henceforth called the authors) from April 2011, https://doi.org/10.1016/j.giq.2010.09.004.
The article has three focus areas: (1) e-government, (2) Gambia and developing countries' situations, and (3) the TAM (Technology Acceptance Model), paired with IS (Information Systems). Drawing on these areas, the authors want to make the following case: when designing e-government initiatives, taking into consideration the core constructs of TAM [perceived ease of use (PEOU), perceived usefulness (PU), information quality (IQ) etc.], will help improve the acceptance of technology by the population. The authors test hypotheses by feeding data from a questionnaire survey among Gambians into a model and by calculating relationships and probabilities.

Problems
In brief, the article fails completely in presenting a convincing, or even a viable, case. But that is not the worst problem. Let us have a look at the text first.
1. The use of sources is unfortunate or plainly wrong. Apart from misspelt and misunderstood names (e.g., Aggelidid, Yayehiyrad), many of the references are not even talking about the claimed topic. In the first twelve author-date-references, only two (Jaeger 2003, andCarter andBélanger 2005) may have some slight relevance, but only to that extent that their very general introductions (not even their findings) are paraphrased. The other names appear to be filled in randomly (or clearly misattributed). This applies to all three areas. Kitaw 2006 never made prophecies about e-government being the fastest growing African industry. Heeks 2006 never claimed that positive attitude towards ICT in general fosters satisfaction with a given IT system. The Gambian national feasibility base line study from 2003 (falsely called UNDP-study in the text and ECA 2004 in the reference list) never addressed the TAM or citizens' behaviour in Internet use. There are many more examples of this kind.
The most severe example of incorrect citation is a block-quote from Bwalya 2009 regarding cooperation between the Government and the Immigration office in Zambia. This passage is actually already a citation (Bwalya quotes Simenda), something the authors ignore. But more importantly, they simply change the word Zambia to Gambia three times without indicating it, leaving the rest of the quote intact. This makes it appear as if Bwalya had written about Gambia, and as if there was a successful Gambian collaboration and working e-government initiative. This alone should be reason enough to discard the credibility of the article. But there is much more.
Kitaw, Heeks, and Bwalya could and should have been used to advance other points, e.g. the lack of general literacy as the greatest challenge for e-government in developing countries, a government's genuine wish to embrace transparency and accountability as a pivotal factor, the lack of linguistic and cultural localising of e-government as a reason for failure. All of these factors have become common ground in the field. The authors ignore this, as they disregard other central and necessary sources on e-government in Gambia, e.g. Chango 2007 on the organizational ecology and gaps between design and reality, or Sander et al. 2005 on the role of government officials in the equation. Here, however, Gambia is presented as a vigorous and successful promoter of e-government (no evidence is given to back up that claim), while the only problem is with user adoption, something the TAM could help fix. This is a peculiar representation of reality, and a bizarre misunderstanding of what the model could ever be used for.
2. The numbers do not add up. The authors claim a response rate of 16.7%, and a rate of 14.6% for valid responses (n = 1000), but the breakdown of demographic data consistently shows that the authors work with 276 valid responses. 276 of 1000 is 27.6%, not 16.7%. This wrong number is then compared to similar studies to justify the low response rate for the study. The rows are not filled in analogously, and lack a point of comparison (response rate) in half of the cases. Studies are erroneously reviewed (the authors mix up Venkatesh 2000, and Venkatesh and Davis 2000). The feasibility baseline study mentioned T above reports 165 responses out of 200, thus a non-response rate of 17.5%. In this study, this becomes 165 out of 515 (which was the sample frame, not the sample size), claimed to be a response rate of 17.5%. Everything is wrong here, but it is still used to justify the authors' own response rate of 16.7%.
But most importantly, the authors seem to think that response rates in telephone interviews, email surveys, tweeted requests for answers, longitudinal studies and researcher-administered surveys are comparable, and can be used to justify their own response rate. And why go to the health sector for studies, when there are plenty of studies on egovernment? Fact-checking the percentages shows that almost none of the figures are correct, and some errors are considerable (0.60% response rate in Venkatesh and Davis 2000).
The numbers calculated in this study (the authors' own data) do not add up either. Response means are only given for the constructs (cumulated means), not for each item in the questionnaire, making the calculation impossible to verify. The study reports means of 5.082, 5.142 and 5.617 for the constructs attitude, usefulness and ease of use, while using a five-point Likert scale numbered from 1 to 5. (These numbers cannot be the sum of the components; behaviour intention with four components could not have been 2.933 in this case where the lowest answer is 1.) And are they the result of division by 276, by 167, or by 146? These grave errors in the data should be enough to completely dismiss the validity of the findings.
3. The questionnaire is flawed at best. The notions intention, opinion, preference and habit are carelessly mixed. Some questions are unintelligible. Some of them confuse Internet, computer and e-government ("You need computer/internet, because: 1. e-government is not affordable", with the mentioned five-options Likert scale). Some of them mix the constructs PEOU and PU ("Using the e-Government would make it easier to do what I want to do and would be very useful in my life" in one question). Some of them have awkward scales with awkward qualifiers. What does "extremely not important -not important -neutral -importantextremely important" mean? This makes it difficult to compare the answers in this block to the scales in other blocks; on top of this, the scale is skewed in itself.
Although the authors define e-government generally, it is completely unclear throughout the article and in the questionnaire what they concretely mean by e-government, e-government initiatives, or portals, and whether they want to assess a device already in use or a hypothetical device. Some questions thus ask about (perceived) facts: "The e-government provides convenient access", while other questions in the same block ask for estimations of future feelings: "I would find egovernment services very secure enough to conduct my transactions online". The meaning of transactions remains completely unclear, as it is nowhere defined, neither in the questionnaire or in the article. Answers to these questions cannot be bundled and treated as one, as they do not compute into the same construct.
The last free text question "Are online references on the Gambian government portable websites very convenient and easy to navigate by the citizens to get the necessary information?", perfectly sums up the problem. It is formulated as a yes/no-question, but no answer could possibly reflect the convoluted content. References are followed, sites are navigated, but it remains unclear what the authors mean by "navigating references". This is important in Information Architecture, on which egovernment websites are built. It is not clear what "portable" websites are (this is the only instance of the word); it is unlikely that the authors mean mobile versions of official sites. "Very convenient" is a leading (biased) phrasing. "By the citizens" implies a shift of perspective (this is recurrent in the other questions, where the personal pronouns I, We, You are mixed). Should the respondent express what she thinks about other people's feelings? The whole question suggests that there are working e-government web sites, and that the user has at least some experience with them, while many (not all) of the former questions treat e-government as a hypothetical platform. "Learning to operate the egovernment system would be easy for me", for instance, asks about a self-assessment of probable future skills (thus it is about self-confidence), not the current perceived-ease-of-use. Still the answers to those very different categories are treated as if they belonged to only one. There is thus a mismatch between data and interpretation, and the questionnaire and its categories are fundamentally wrong. The data gathered from such an inherently flawed questionnaire could not form the basis for valid hypotheses testing, even if the figures had been calculated correctly, which apparently is not the case.
4. The data collection process is biased. Based on the description in the article, the government itself sent out the questionnaire to 1000 mostly government workers, helped collect the answers, and obviously tracked non-respondents to send out reminders. Anonymity (between respondents and government) is thus not guaranteed. We know that Gambia at the time of the data collection (2008) was a dictatorship, where critical voices were intimidated and jailed. Especially academics and government workers would fear for their jobs and well-being if they dared to give less than favourable reviews. It is obviously easier and much safer to explain the lack of behaviour intention by inadequate electrical supply than by lack of trust in the government, when this same government can track your response. The statistics suggest that the users are overly satisfied with everything (exceeding the maximum level of 5), they are just not going to use it. The article explains this gap merely by electricity problems. The real response bias lies here, not in the response rate. The authors only defend this latter bias, vigorously but not convincingly. There is no mentioning of the much more important "truthfulness" response bias in the discussion on methodology. The article shares this lack of acknowledgment of political facts and their impact on data collection and validity with most other comparable articles about TAM in autocratic governments.

Definitions and the wording of hypotheses are poor.
Defining Information Quality (the construct, not the value!) with the phrase "The information quality of the e-government service will enable the people to research for the information and look the news through online TVs and radios online" is astonishing. Referring to Carter and Bélanger 2005, and Igbaria et al. 1997 as supporting literature for this definition, or the construct, is wrong since none of these authors say anything about information quality, far less do they provide a definition. Here and in other (or even most) places, in-text references seemingly guarantee valid embedding of the argument in previous work, something which a closer inspection quickly refutes. The authors are simply engaging in random name-dropping.
Hypothesis 7 states that "The information quality of e-government filing systems positively affects the perceived usefulness of using the internet". All the other hypotheses are about e-government in general, this is the only hypothesis about its filing system. Such a filing system is, however, not explained in the text nor in the questionnaire, nor does it appear there as a question (as is the case with the transactions mentioned above). Moreover, this hypothesis is very complex, in that the IQ of a specific system is supposed to affect the perceived usefulness of the Internet as a whole. This complexity cannot be tested with this questionnaire.
6. The article is not about citizens and cultural differences. The authors repeatedly (in the abstract, the conclusion, the "highlights", and the text) make a point of focussing on citizens, and of proving that the model can explain and predict usage despite cultural differences in the country and in the differences in behaviour between Gambians and other e-government users. They even refer to "the empowerment of all Gambians" as one of the Government goals, outlined in the intention manifest Vision 2020. These are goals that this article clearly attempts to support, but no such thing is proven or even discussed. The survey was sent to 1000 potential respondents, who, the authors admit (and even defend), have better-than-average access to the Internet and to technological facilities. The respondents pool thus does not reflect the target population of the study. The demographic section of the questionnaire does not ask about the respondent's culture or language (almost no Gambian is a native speaker of English), and the analysis does not refer to any demographic values (age, income, experience). There is no direct comparison to other cultural or social groups or ethnicities, either.
An example: the authors find Hypothesis 2 ("The perceived usefulness of the e-government services has a positive effect on user behavior intentions.") not supported. In other words: If a user judges e-government convenient, she is likely to have the intention of using it. They explain the weak link between the constructs by poor infrastructure: With as low a rate of connectivity and accessibility as in Gambia, she may perceive the convenience as extremely high, but may nonetheless not intend to use it because it simply will not work and such convenience will remain mere theory. Although interesting, this view of reality (or frustration) is not surprising. It is likely to be found in all developing countries and not proper to Gambia or Gambians. I cannot see how this study is citizen-or user-centered. Rather, it is system-driven.
(This, together with the poor and wrong use of sources in the field discussed earlier, makes it clear that this article is more about feeding numbers into a model than about enhancing understanding of the culture and politics of a developing country.) 7. These explanations are not a part of the model or the study, but an external means to rescue the model and its applicability to egovernment in Gambia. They are not systematic or factorized in any way. When discrepancies between supposed and verified relationships can be explained ad hoc by external factors without modifying the model, then anything goes and the model can never be falsified. The validation of the model thus feeds on itself, i.e. its validity is based on circular reasoning; nothing is proven. (This is not exclusive to this study as many TAM articles make this fundamental error.) Proposing the use of TAM to a Government as a solution is thus questionable, as it does not explain or predict anything we did not know already. On the contrary, the external factors and explanations are far more important for the issue at hand. One can find them in the omitted and misquoted sources.

Summary
The article is on Elsevier's list of most cited articles from the Government Information Quarterly journal, and has become a key reference in the field of study, with nearly 250 citations (in Google scholar). However, it completely fails when it comes to overall linguistic expression, literature review, grounding in the field, citation practice, questionnaire design, data collection, rendering and interpreting others' and own data, calculation, claims of user-centeredness and accounting for cultural differences, and the final assertion that all this leads to Gambia benefiting from TAM. (The authors even claim that the government is already using the findings of this study, whatever that may mean.) Its premise and findings are blatantly wrong: they are not valid, reliable, verifiable or reproducible in any way. No single part could be changed to achieve integrity, and for proper results, everything would have to be redone, starting with the questionnaire design, and ending with the conclusion. The study could not really be used to improve Gambia's e-government, or government in general, for that matter.
This article is as close to a scientific hoax as one can possibly come, but I believe that it is just an unfortunate case of poor science, not a deliberate fraud as such, even if the Zambia/Gambia quote is hard to excuse as unintentional. In any case, this is an article that should obviously never have been published, and raises serious questions about the editorial rigor, the quality of the peer reviewing, and the revision process in the journal.

Consequences
My original review stopped here. The second peer reviewer for this text warned that it had no clear goal, that the purpose of driving this analysis should be made explicit, and that I must address the inevitable question if the article must be retracted. 1 There are two possible answers to this, or rather, an answer and a series of counter-questions. I will give them here, even if it means extending the nature of this regular review so that it becomes a kind of open letter. I will be very clear.
Firstly, this field of research has obviously a much bigger and more urgent problem than what could be addressed by the retraction of a single article, which is ultimately just a sad case of very poorly executed research. To the community: How could this go undetected for five years? Quite the contrary: how could it become a key reference in the field? How could even renowned professors and prominent scholars in the field cite claims from this article, re-cite both themselves and others, and relay references to an article that they cannot have read? Or even worse: not just read, but accepted as solid research? I have read them all, and there is not even the faintest expression of doubt as to the validity, reliability and integrity of this study in any of them, even in the "critical literature review" meta-articles. How could the erroneous author attributions (including misspellings) be taken at face value and be relayed again and again? (I have many examples.) To the publishers: how could a leading and prestigious journal like GIQ not prevent this article from passing peer review and editorial quality check? How is it possible that none of the other journals and editors detected the problems with this article in their own peer reviewing process -in the case of 250 publications? These questions point to a serious illness in the field, and it is one that a retraction alone would not cure.
Secondly, a call for retraction should rather come from the community than from someone like me, who observes the field from a distance. I sent this review to the journal in Spring 2016 as a report, expressing concerns and requesting comments. (This is why this review bears all the signs of a report.) I was finally invited to submit my report as an article review to the journal, "to raise awareness in the field", as the Editor in Chief put it. (I take this as a fair and honest goal and measure; he has always been open and correct in our contact.) This raising of awareness is thus the intention behind completing the present analysis and offering it for publication. But apart from this invitation, what has happened during these two and a half years? The article has obviously not been retracted or suspended. Why? And why is Elsevier still making money from the article by keeping it behind a pay wall? Why are there still no notes of concern raised as to this article, which is still sold as solid research, figuratively and literally? Why is this article still not on a "references red list" for authors, peer reviewers and the publishing process? GIQ has been publishing papers with uncritical references to it for long after my warning. Has GIQ arranged seminars, conferences, workshops or discussions to address the purpose and impact of scholarly referencing in this field? Has it initiated a broader investigation of the field's citational practices? How did and how does the Editorial Board react? Does it see grounds for making an official statement about the matter? Are the peer reviewers still asked to carry out their work? Has anyone taken responsibility? The journal should explicitly answer these questions, and should do it publicly.
The second peer reviewer also suggested that merely presenting a review of the study is not enough: there should be an investigation on the reasons why the article has been so successful. This could probably point to the reasons behind the success of poor research in general. I completely agree that this is the salient question. This short review cannot answer it, but I am addressing this question, and will attempt to offer some answers, in a much larger and thorough publication (nearing completion): Telling the wrong story and believing it. It contains the background, history and impact of the reviewed article, a close-reading of the articles citing it, and other articles in this field of study. It shows that the lack of scientific rigor in writing, referencing and publishing demonstrated here is not a one-off lapse, but symptomatic of a larger problem.