The impact of digitalization in the public sector: a systematic literature review 1

The digitalization of public administration is in-creasingly moving forward. This systematic litera- ture review analyzes empirical studies that explore the impacts of digitalization projects (n=93) in the public sector. Bibliometrically, only a few authors have published several times on this topic so far. Most studies focusing on impact come from the US or China, and are related to Computer Science. In terms of content, the majority of examined articles studies services to citizens, and therefore consider them when measuring impact. A classification of the investigated effects by dimensions of public value shows that the analysis of utilitarian-in- strumental values, such as efficiency or performance, is prevalent. More interdisciplinary cooperation is needed to research the impact of digitaliza- tion in the public sector. The different dimensions of impact should be linked more closely. In addition, research should focus more on the effects of digitalization within administration.

1 Introduction E-records, online services, assisted decision making systems: Considerable technological advances have characterized the administrative modernization of recent years, both in research and in practice. The goals are manifold: Digitalization is associated with increased efficiency and performance, and better service delivery, but also with greater transparency, participation, and cooperation (Alcaide Muñoz, Rodríguez Bolívar & López Hernández, 2017;Kim & Lee, 2012). These goals are discussed as public values related to e-government in the literature (Twizeyimana & Andersson, 2019).
However, apart from plans, purposes, and intentions, we still lack knowledge about the actual impact of digitalization in public administration. Empirical e-government research mainly deals with questions of the status quo (Mergel, 2019), IT tools (Ølnes, 2016), citizens' or employees' expectations and acceptance (Heuberger & Schwab, 2021;Fischer et al., 2019), adoption (Jacob, Fudzee, Salamat & Herawan, 2019), and success criteria (Veeramootoo, Nunkoo & Dwivedi, 2018), but overlooks outcomes and impact. However, as Ibrahim Otieno and Elijah Omwenga (2014, p. 1) note, "assessment of impact is important to justify public fund expenditure and inform future projects." By systematically analyzing research on the impact of digitalization in the public sector (N=93) in this article, we found that studies mainly analyze service provision and citizens as a target group of impact. They focus on impacts related to utilitarianinstrumental public values, such as efficiency and performance. However, we recognize disciplinary differences with regard to the analyzed values. We conclude that impacts with regard to different value dimensions need to be entangled and discussed together rather than separating them. That could, for example, be achieved by more interdisciplinary work. We also recommend taking the internal impact of digitalization into consideration to a greater extent, instead of mainly focusing on external effects.
The articles included in this symposium aim to close this research gap, at least to some extent. While the majority stems from Public Administration and Political Science, they focus on external as well as internal impacts of digitalization and acknowledge impacts on different dimensions of public value.
The remainder of this article is structured as follows. We first define and discuss terms related to e-government, digitalization, and the digitalization of public administration, that are often used interchangeably in the literature. Second, we introduce why it is essential to study the impact of digital transformation and what is meant by this term. After that, we describe the applied methodology for the systematic literature review. Next, we report and discuss the results. Finally, we present the articles encompassed in this symposium and highlight their joint contribution.

Terms: From E to Digi
The literature uses many terms to describe the use of information and communication technology (ICT) to modernize administrative work: E-government, E-governance, Eadministration, E-service provision, E-democracy, digitization, digitalization, digital transformation, and digital government. Electronic government (e-government) and related terms developed in the early 1990s, when governments began to enter the world wide web as a fundamental element of governing electronically (Coursey & Norris, 2008, p. 523). Broadly, e-government refers to efforts to make service delivery more efficient and accessible to citizens by using the web and ICTs (Meijer & Bekkers, 2015).
Models on e-government describe this construct as a linear, stepwise, and progressive process from an initial web presence to information provision, interactivity, and transactional service provision. With these higher levels of maturity, e-government evolves from a front-office issue (information on the website) to a combination of front and back-office solutions (Homburg, 2018, p. 353). However, "nearly all of the models become quite normative when describing a fully developed e-government, and they assert what e-government should become. The models implicitly presume that fully transactional systems are better and that more citizen interaction equals improved service" (Coursey & Norris, 2008, p. 524).
In some of these models of e-government, information, interaction, and transaction are followed by the stage of transformation, as a suggested endpoint of digitalization. Christopher Baum and Andrea Di Maio (2000) define transformation as a fundamental positive change in the interaction between citizens and governments accompanied by citizen-centricity, responsiveness, and increasing trust in government. However, beyond this normative idea, Kenneth Kraemer and John Leslie King (2006), for example, concluded that the ICT-driven government has changed little and, if anything, only reinforced existing power structures. According to Darrell West (2004, p. 15), " […] it is difficult to determine how much innovation and over how long a period of time is required before something can be considered a 'complete change in character and condition,' the classic definition of transformation." Therefore, Frank Bannister and Regina Connolly (2014, p. 128) note, that transformation is not a binary categorization of missing or major change, but rather a continuum without a clear endpoint showing when exactly something becomes a radical change. Hence, it is often unclear in practice when the implementation of desirable values count as transformation or not.
The literature nowadays has mainly refrained from using the term 'e-government' and mostly uses terms related to digitalization (Lindgren, Madsen, Hofmann & Melin, 2019;Jehan & Alahakoon, 2020). In our point of view, three causes, apart from a general turn in language, can be identified for this terminological development: (1) Terms are connected with a certain image and, from the point of view of a majority of practitioners and researchers, e-government was never achieved and has, therefore, failed (Lips, 2012). (2) Technological change leads to new possibilities that have not been implied in models of e-government, such as mobile or ubiquitous computing (Kiki & Lawrence, 2006). (3) New demands with regard to transparency, participation, and collaboration exceeding traditional models of e-government have developed (Bertot, Gorham, Jaeger, Sarin & Choi, 2014).
Digitization is understood as the pure conversion of analog data for digital storage, hence, the electronic replication of existing analog structures and processes without further changes in the administrative, organizational, and process structures (Mergel, Edelmann & Haug, 2019, p. 12). Indeed, these efforts have already led to significant improvements in public organizations, for example, by saving time when transfering information. However, in this stage, it is often overlooked that an inefficient digital process is still an inefficient process, and advances provided by available technology aretoo much focused on.
In a further step, digitalization relates to the transformation of analog processes into digital processes by revising these processes and introducing new organizational models (Heuermann, 2018, S. 1). When processes are not only digitalized, but a more comprehensive institutional change takes place, we speak of digital transformation. This term considers not only organizational and processual change but also a major cultural change in public authorities, in personnel and qualification structures, in the interaction with citizens, and long-term changes in the performance of public service delivery. Hence, digital transformation is accompanied with considerable developments in the production of services and the associated interactions. Therefore, digital transformation focuses on the socio-technical nature of these changes instead of merely considering technical questions. However, as discussed above, a clear cut between digitalization and digital transformation is hard to define due to the relativity of what counts as a 'comprehensive' change. Similarly, empirical work to date faces the challenge of differentiating between these stages. Therefore, we use the term 'digitalization' in this article to describe the changes in public administration due to the use of ICT, and consider digital transformation to represent the impact stage of digitalization.
All in all, this overview of terms related to digital government shows that the field has still not agreed on a common definition and concepts remain fuzzy. However, the literature agrees that -whether for e-government or digitalization -one cannot identify an endpoint of digitalization that can be achieved by organizations, but rather describe continuing processes and dynamic models that can be clustered in certain waves and milestones. Yet, how can we claim to speak about the impact of such an ongoing process? How can we understand impact before we have reached the level of digital transformation? We argue that one can analyze continuous change and impacts that should be considered as interim and preliminary. However, there is also debate as to what is actually seen as the impact, in comparison to immediate outputs or outcomes, of a certain policy, an instrument or system in use. Therefore, we discuss this term in the following chapter and agree on a definition of impact as a continuum related to the idea of continuous digitalization of government.

Studying impact
There is no generally accepted model for defining the effects of interventions, in this case, measures in the context of digitalization. Short and medium-term effects are often referred to as outcomes, while long-term effects are called impacts (OECD, 2013). Further, outputs and outcomes mainly refer to effects within an organization or an interaction with stakeholders, whereas impact is related to more general effects on society. In this section, we show why assessing the impact of digitalization is a challenge and which approaches can be suitable to analyze the effects.
The timespan that is needed to obtain results and the longevity of effects are discussed in the literature (Belcher & Palenberg, 2018). Evaluation studies are often reduced to measuring output effects, mainly because of the limited time frame. However, effects might change over time and intended effects might need time to develop. An initial negative output of digitalization might, for example, be followed by a positive long-term impact and vice versa. For example, when a government implements a digital process, the costs of working with this new process might be higher in the beginning, as employees have to get used to the new course of action. However, over time these initial costs might pay off and employees might save time with a more efficient process. Therefore, it has to be taken into consideration that the measurement of impact is always a snapshot of a certain point in time and effects of digitalization projects need to be evaluated continuously to get a more realistic view of the short-and long-term effects.
In this regard, the intentionality of the results as well as their directness are addressed in the literature, too (Belcher & Palenberg, 2018). While negative effects should usually not have been intended, but might occur anyway, that could also be true for positive effects that evolved unintentionally. Especially regarding public administration, where digitalization is complex and its impact often unpredictable, the process from inputs to impacts might be a game of trial and error, and emerge rather unplanned. This makes it all the more important to study impacts and their drivers to learn from these successes or failures.
To evaluate the benefits of digitalization projects, methods for economic evaluation are well-established in public sector organizations (Sterrenberg, 2017). However, they address less of a long-term and societal perspective, although qualitative criteria can be considered in addition to monetary ones. Therefore, they are suitable for evaluating output and outcome. Further, in Computer Science and related disciplines, the so-called Information Systems Success Model is established (Delone & McLean, 2003). This model distinguishes system quality, information quality and use, user satisfaction, individual impact, and organizational impact. Information systems success is a complex construct with multiple interdependencies between these dimensions and it is therefore necessary to examine the relations between them (Delone & McLean, 2003) instead of focusing on a single impact dimension only. However, we argue that the impact of information systems goes beyond individuals and organizations. Therefore, overarching impact categories that also address the societal level need to be considered -especially in the public sector context.
To differentiate between the effects at the impact stage more precisely, Public Value Theory (Moore, 2000) can make an important contribution. Public organizations should create public value(s), as this fulfills the needs and desires of citizens and businesses in their different stakeholder roles (e.g., as policy makers, as taxpayers, as inhabitants) (Twizeyimana & Andersson, 2019). We speak of multiple public values when specific dimensions are meant or particular target groups are addressed, and use the term 'public value' when the public in general is addressed (Bryson, Crosby & Bloomberg, 2014, p. 448). Through digitalization and related opportunities to participate and co-create, "the creation of public value, in turn, becomes more networked, more open, and more participative [for different stakeholders]" (Boin et al., 2020, p. 360). The use of Public Value Theory within digital government research is well established (Cordella & Bonina, 2012;Bannister & Connolly, 2014;Ranerup & Henriksen, 2019). While Public Value Theory focuses on how to serve the public interest, on how the government creates and sustains value on behalf of the public (Nabatchi, 2018, p. 60), an emerging stream of literature on public values (Bozeman, 2007) is aimed at "identifying and enacting those values qualifying as public values" (Fukumoto & Bozeman, 2019, p. 635). According to Tina Nabatchi (2018), public values describe a normative consensus of a society and are based on emotional and rational assessment by individual persons. Overall, there is already considerable research on the relationship between public sector values and digitalization initiatives (Twizeyimana & Andersson, 2019).
Several authors sought to list and sort public values (for an overview: Fukumoto & Bozeman, 2019). For example, Timo Meynhardt (2009) distinguishes between public values according to four dimensions of value creation: the moral-ethical, hedonisticaesthetical, utilitarian-instrumental, and political-social dimensions. These dimensions are derived from needs theory. We selected this approach since the author develops building blocks for a non-normative public value theory and also explicitly considers contradictory values. Moral-ethical values focus on a positive self-concept and a consistent relationship between the self and the environment. With regard to e-government, this concerns issues of trust and corruption, for example. Hedonistic-aesthetic values address positive emotions and the experience of self-efficacy, which in the context of e-government includes user satisfaction and quality of life. Utilitarianinstrumental values mean the ability to achieve desired outcomes and thus focus, for example, on service delivery. Political-social values are based on the need for positive relationships in the community, e. g., participation in the context of e-government. By analysing the impact of digital government by means of public value, for example by using Meynhardt's (2009) dimensions of public values, impact analyses exceed intended goals and focus on more general societal impact thus helping to increase our understanding in the context of the impacts of e-government.
In our literature review, we consider output, outcome, and impact, and analyze them in terms of the public value dimensions presented because this allows for a detailed analysis at the results level. However, we define the impact of digitalization as a continuum without a clear start and endpoint. We also take into account both intended and unintended effects.

Methodological approach
For the systematic literature review on the impact of digitalization in public administration, we used Web of Science as a database. Web of Science is both a reliable and easily verifiable source, as relevant indices are included, and the metadata necessary for further analysis is available. We selected relevant indices for the interdisciplinary field of digital government research from a Social Science, and Management and Information Systems perspective: SSCI, SCI-Expanded, CPCI-S, CPCI-SSH. These indices include both peer-reviewed journals and indexed conference proceedings.
To guide the search, we identified three groups of keywords in an inductive manner (see Table 1). They represent (a) the impact and effect dimension of the search, as well as (b) the digital government dimension, and (c) include empirical articles only. The latter was chosen to focus on empirical findings, instead of theoretical impact. Within the three groups, "OR"-operators were used, while the groups themselves were connected with "AND"-operators.
Records screened on eligibility criteria by full reading of abstracts and/or articles (n=645) Records identified through Web of Science, for keywords see Table 1 Limited to 10-year timespan 2011-2020 (n=800) Records excluded (n=155) Inappropriate records excluded (broad topic) (n=552) Records included (n=93) Source: Own illustration.
The results of the search, conducted on December 1 st , 2020, were filtered by category, representing the research fields of interest -excluding literature focusing on primarily technical issues. We filtered for Public Administration, Political Science, Computer Science, Information Systems, Management, Information Science, and Library Science, which led to 800 results. Figure 1 illustrates the following process. We limited the timeframe to the last 10 years (2011-2020) because we suppose that, due to the technological state of the art before this timeframe, no impact of digitalization can be observed. That resulted in 645 entries. The complete list was cleaned by the authors, selecting only the articles examining impacts or effects of digital government measures. Here, every article was coded by at least two of the authors in at least two rounds of coding. The final list contains 93 studies. As this literature review wants not only to present a bibliometric analysis of the selected articles, but also regarding content to generate rich knowledge on the analyzed topic, we additionally coded the articles according to the following content-related categories: First of all, digitalization in government was specified We inductively created a typology of digitalization objects. Herein, we sorted the entries according to their focus on either (a) digitalization of service provision, (b) open government and e-participation or (c) digitalization of internal administrative processes. Additionally, the target group of digitalization effects was coded according to their relationship with the government: G2C (including C2G), G2B (including B2G), G2G or internal actors. 2 Second, we specified the analyzed impact. Impact was categorized as either intended, unintended or missing. Furthermore, we separated the studied impacts according to the public values they create or foster. Based on the value scheme provided by Meynhardt (2009), public value creation was clustered into four different dimensions: moral-ethical values (such as trust), hedonistic-aesthetic values (such as user satisfaction), utilitarian-instrumental values (such as service delivery), and political-social values (such as participation).
Third, the methodological approach (qualitative, quantitative or mixed methods) was categorized, as well as the differentiation between single and multiple case studies.
The sample of the articles used in this review, as well as the detailed codings of the articles are accessible and reusable (Fischer, Heuberger & Heine, 2021) to ensure the transparency and reproducibility of our findings.

Results
We identified 93 pieces of published peer-reviewed literature (60 journal articles and 33 conference proceedings) on the reviewed topic. A first analysis of the quantity of articles on the impact of digitalization in governent by disciplines ( Figure 2) shows that, especially, Computer and Information Science study this topic, whereas the focus on impact in digitalization seems to be a minor topic in Business Administration and Economics.   . This finding is quite atypical for these disciplines, especially for Public Administration and Social Sciences, and may be explained by the specificity of the topic at hand. Sorting the papers according to the number of their citations (see Table 2) and including only the ones with higher numbers of citations (>10 citations), the areas of Business & Economics (7 frequently cited/15 papers in total) and Information Science (13 frequently cited/33 in total) are overrepresented, compared to Computer Science (8 frequently cited/36 in total) and Public Administration (7 frequently cited/31 in total), which could be expected in accordance with the relation between journal articles and conference proceedings we found in these disciplines. Furthermore, these citation rates might potentially occur due to the different 'citation cultures' in these disciplines.
The methodological approach of a study was found to be another indicator for high numbers of citations. While the distribution of quantitative and qualitative articles is relativily similar (47 quantitative, 39 qualitative, 4 mixed metods) in the sample 3 , 40% of the quantitative papers (n=19) and only 18% of the qualitative papers (n=7) and none of the mixed method papers are frequently cited.
Finally, the research focus of the paper was found to impact the citation rates: Sorting by the kind of digitalization, we observe a majority of citations for articles on service provision (51), followed by open government (36) and internal processes (24). Focusing on types of digitalization, we find that only two papers paying attention to internal digitalization are frequently cited (8%), while the numbers of frequently cited papers are much higher for service provision (16; 31%) and open government (15; 42%). Taking the analysis of these four research areas further and comparing them to the kinds of digitalization studied in the included articles, we see a general focus on service provision and a dominance of Computer Science in the field of internal impact research ( Table 3). Source: Own illustration.
In identifying relevant authors on the impact of digitalization within government, five authors can be found as ( These conferences are mainly related to Computer Science research and also wellknown apart from this focus on the impact of digitalization in government. And finally, the dominant journal in our sample is -by far -Government Information Quarterly, contributing 21 articles with 473 citations in total, while other journals with a priority in this field cannot be identified (see Table 4).
Shifting the focus to the geographical origins of the research, the country of the scientific institution with which the first author is affiliated, according to the authors' information, was selected. The US, China, and the UK can be identified as three major countries of origin (Figure 3). Looking only at articles with higher numbers of citations (>10), we can identify a clear dominance of the US (8), followed by China and South Korea (4).  Source: Own illustration. The target groups, i. e. the ones which are mainly affected by the impact of the digital change in the selected studies, are in 63 cases (68%) citizens (G2C/C2G-interaction), followed by governmental actors such as other agencies or public institutions (G2Ginteraction) with 14 cases (15%), internal actors in the same organization (n=12; 13%), and businesses (G2B; B2G: n=6; 6%). The impact analyzed in the selected studies was mostly intended (n=72; 77%; Song & Lee, 2016) and only in twelve cases (93%) unintended (Klischewski, 2014). In nine cases (10%) the analyzed impact could not be observed (Morgeson, VanAmburg & Mithas, 2011). However, we suppose that a publication bias with regard to null findings exists also within this field of research and we therefore do not expect the distribution we found between effects and non-effects, intended and unintended effects, to be representative.
Clustering the impacts according to the created categories of public value creation, a focus on utilitarian-instrumental values (higher efficiency, performance) can be found (n=47; 51%), followed by politico-social values (n=28; 30%; better participation, transparency), moral-ethical values (n=28; 30%; higher trust, anticorruption), and hedonisticaesthetic values (n=21; 23%; user satisfaction, quality of life). Sorting these impacted values according to the kind of digitalization (service provision, 89%; open government, 55%; internal, 29%), we can observe a certain pattern: Utilitarian-instrumental values are dominant for the impacts and effects of both service provision and internal digitalization projects (see Table 5), while open government projects are mainly shaped by politicosocial values. Hedonistic-aesthetic values are primarily connected with service provision, while moral-ethical values can be found in both service provision and open government projects but significantly less in internal digitalization projects. Note: Both multiple value dimensions and multiple kinds of digitalization are possible, leading to higher sums than cases; 20 studies were found to study more than one dimension, but no study considered all four dimensions. Source: Own illustration.
Combining both the geographical origin and the dimension of the analyzed public values, we find that Chinese research mainly focuses on utilitarian-instrumental values (7 studies out of 11), while studies from the US focus more on moral-ethical values (7 studies out of 13).
Comparing the different types of values with the research area, only Public Administration shows a rather balanced picture (Table 6), while the other three areas have a tendency to produce studies on utilitarian-instrumental value creation. Hedonisticaesthetic values, on the other hand, are underrepresented in all four research areas. Hence, research onuser satisfaction , for example, is not very prominent in our sample. Source: Own illustration.

Discussion and Implications
Our analysis of the literature on the impact of digitalization in government within the last ten years showed that the playing field is rather small. Few studies focus on the effect of digitalization in public administration. Similarly, the meta-analysis by Bernd Wirtz and Peter Daiser (2018) shows that comparatively little empirical e-government research is available. We narrow down the research field even further by focusing on digitalization impacts. Nevertheless, some insights can be derived that address the interdisciplinary perspective on e-government, consider the dimensions of public value, and make some observations about leading authors, journals, and conferences. Digital transformation in the public sector is a highly interdisciplinary field. Even in the early e-government research in the 1990s, the topic was mainly dealt with by Computer Science, Information Science, and Information Systems research (Murphy & Hwang, 2017). Administrative and Social Sciences lagged behind. A similar picture emerges from our systematic literature review (Figure 2). Not much seems to have changed sincethe study of Philip Murphy and Sungsoo Hwang (2017), although they analyzed e-government research in general. More intensive cooperation, especially between Computer Science (and related disciplines) and Public Administration, would certainly be promising.
Looking at the results in detail discloses further differences between the disciplines. The dominance of public values per field of action (Table 5) and per discipline ( Table 6) is hardly surprising. Open government addresses transparency, participation, and trusting cooperation, which predominantly corresponds to moral-ethical and political-social values. Service provision and internal digitalization are closely linked to the execution of concrete tasks within the framework of public service provision and the associated interaction between the different players. Accordingly, hedonistic-aesthetic and utilitarian-instrumental values are the main focus. Computer Science, Information Systems research, Information Science, and Library Science are design-oriented disciplines, i. e. they predominantly design functional and ideally attractive solutions that are oriented towards the service portfolio of public administrations (utilitarianinstrumental values). Public Administration and Social Science, by their nature, address political-social and moral-ethical values as well. Overall, the individual fields of action and public values do not have the same broad consideration in the different disciplines. It would be desirable to achieve more cooperation between the disciplines to use the different perspectives and methods fruitfully (Murphy & Hwang, 2017).
Taking the journals and conferences into consideration, where research on the impact of digitalization is presented and published, most empirical papers on impact are published by Government Information Quarterly. The journal calls for submissions from various disciplines and is one of the few ranked and peer-reviewed journals, explicitly, focusing on digital government instead of digitalization more broadly or public administration only. Furthermore, in our analysis, we did not include national journals and conferences that publish in languages other than English or are not indexed, which might also be target journals, especially for empirical work analyzing national developments in e-government. Accordingly, we only consider a section of the research landscape in this review.
Concerning the country of the first authors' institutional affiliation, the US, China, and the UK dominate. This focus is not specific for impact research but representative for the respective journals. Looking at all articles published in the journals included in Table 4 2011-2020 (n=52,476), the majority of articles is from China (28,080 articles, 53.5%) followed by the US (8,193,15.6%) and South Korea (3,601, 6.9%).
However, these countries also play a role regarding the investigated cases. The empirical data of the included studies are mostly based on single or multiple case studies in the respective country of institutional affiliation. Accordingly, there is a need for cross-national comparative studies. Regarding the public values addressed, differences can also be found concerning the countries of origin. Research contributions from China tend to focus on utilitarian values, while contributions from the US more often address moral-ethical values.
Overall, the articles tend to address only single public values. A holistic perspective of public values is missing. Although there are some approaches to defining public value through e-government (Twizeyimana & Andersson, 2019), these are not systematically addressed in any of the empirical studies we examined. How public values can be generated by digital government still needs to be further clarified (Twizeyimana & Andersson, 2019). This also applies to the measurement of public values. One way to achieve the value of e-government could be to act in a more participatory and open way (Boin et al., 2020).
Much of e-government research focuses on citizens -especially in the context of the public value debate, which is consistently oriented towards citizens. Our analysis shows that studies with a focus on hedonistic-aesthetic values (i. e., usability) mainly focus on e-governments' impact on citizens. This corresponds to the dominance of government portals (Millard, 2010) that neglects an internal view of public organizations and focuses on external impacts instead. However, employees of public organizations are also users of information systems and need to beconsidered when designing them. The performance of public administration and, thus, public value, in our opinion, could be significantly increased if the information systems used internally would also aim to be user-friendly, for instance, regarding public employees' use of algorithmbased assistance systems. This research gap needs to be closed in future research.
Digital transformation is not an end in itself and does not solve all problems, neither within public organizations nor on a societal level. However, to obtain a realistic impression of which problems can be tackled by digitalization and to legitimize these change efforts, more emphasis has to be placed on studying the impact of digitalization. The impact of digitalization in the public sector is closely linked to both specific objectives and effects that are difficult to predict. On the one hand, the impact of e-government on public value(s) should be demonstrated holistically. On the other hand, unintended effects and tensions between values must also be addressed. On this basis, an approximation of the optimal level of digitalization of single public organizations can be made. These optimal digitalization levels can, for example, be incorporated into e-government maturity models in future research, signalling that not every organization and process needs to be digitalized, but priorities must be defined.
Overall, our analysis reveals significant research gaps regarding the impact of digitalization in public administration and points to possible paths for future research. This symposium attempts to make a small contribution with the articles presented below.

Articles in this symposium
The journey of this symposium started with a paper development workshop in March 2020. From the initial thirty submissions, fifteen submissions were invited to be presented at this workshop. Ten manuscripts went into peer-review, were reviewed in a double-blind process, and were revised in several stages. Finally, this symposium includes six contributions on the impact of digitalization in the public sector. These articles refer to different target groups and public values affected by different approaches to the digitalization of government. Two articles in this issue focus on citizens as a target group affected by digitalization efforts within government. Birgit Jaeger provides a systematic literature review on the construct of digital citizenship. She identifies different streams of literature on digital citizenship, focusing on digital rights and privacy, political engagement, digital public service, and training and learning.
Julia Schwanholz, Lavinia Zinser, and Johannes Hindemith focus on a liquid democracy tool as a kind of digital participation. They analyze citizens' initiatives on the German municipal level and show that citizens' decisions in e-participation processes do not always lead to binding policy decisions by local or district councils. They find that the implementation of decisions made by citizens is more likely when it comes to decisions that are less urgent.
Two of the articles included in this issue take a closer look at the impact of digitalization on employees. Florian Lemke, Konstantin Ehrhardt, and Olha Popelyshyn focus on how public employees perceive and position themselves with respect to digital government initiatives. The authors provide insights on German and Ukrainian employees by analyzing qualitative survey data on the self-perceived contribution and participation of employees in digital government, as well as their motivation or frustration regarding this involvement.
Lucille Tetley-Brown and Ewan Klein choose another perspective. They focus on public employees and managers in local governments as users and producers of data. They analyze Scottish cases and develop a conceptual framework for the value of data stemmmig from its use. The authors, therefore, claim that developing an organizational culture that creates value with data requires a combination of suitable working practices and technical infrastructure.
Two other articles focus on questions regarding coordination in collaborative work settings and inter-organizatonal cooperation. Maike Rackwitz, Thurid Hustedt, and Gerhard Hammerschmid analyze the interaction between collaborative working structures and leadership. They test whether cross-cutting digital endeavors lead to more lateral, network-based approaches to governance. They shed light on three distinct challenges (complexity, risk, and power imbalance) encountered during the implementation of the German Online Access Act. Stijn Wouters, Veiko Lember, and Joep Crompvoets focus on the coordination of inter-organizational digital public services. Using a case study on the digitalization of invoicing services in the Belgian public administration, they review coordination instruments and study how these evolved over time. They find that successful digital transformation depends on the choice of instruments. This mix of instruments is likely to change dynamically as digital transformation objectives and governance challenges evolve over time.
In this literature review, we discussed different theoretical approaches to digitalization including Public Value Theory, economic evaluation, or the Information Systems Success Model. The challenges associated with the analysis of medium-and long-term, intended and unintended, as well as individual, organizational, and societal impacts can, thus, be met in different ways. The articles comprised in this symposium were selected to allow a broad insight into the variety of approaches to study impact. They analyze impact regarding different aspects, such as cooordination, employees, citizens, organizational culture, and specific processes and services, such as invoicing, participation, and data use. They go beyond the mere analysis of adoption of tools or acceptance by different stakeholders, which is still the focus of research on effects of digitalization in government to date. In this systematic literature review, we identified a lacking holistic perspective to study impact, especially, regarding affected public values. The articles comprising this symposium engage in filling this gap by analyzing multiple dimensions of impact, and combine, for example, effects on users and citizens with in-ternal impact. We still see the urgent need that future research further develops this approach and sheds more light on entangling these different impact dimensions to get a more comprehensive picture of digitalization effects. Last but not least, by analyzing the literature, we identified a missing focus on internal effects of digitalization, i. e. on impacts regarding processes and structures, employees, their workplace, and tasks. Some of the articles in this symposium take this perspective and we hope that this work can inspire further research on internal impact of digitalization in government.

Notes
1 We want to thank the Joint eGov and Open Data Innovation Lab at the University of Luebeck and the University of Potsdam for their financial support for the author workshop and open access fees. 2 Here, G2C means the interaction of governments with citizens, G2B stands for government to businesses, and G2G describes the interaction between governments. 3 Missing numbers indicate "not assignable" in the specific categories, due to missing information about the method in the abstract and the article.