A study on the impact of non-operational mechanisms on the effectiveness of public information technology governance

This study proposes an analysis of non-operational mechanisms that can impact the effectiveness of information technology governance in the Brazilian public administration. A questionnaire was sent to information technology managers and experienced professionals in the area of information technology governance, who work in public organizations. Based on the collected data, we used the structural equations technique to verify the impact of the following mechanisms on the effectiveness of information technology governance: top management support; performance of the information technology steering committee; use of information technology strategic planning; performance of the information technology investment portfolio management. The study indicates that the ﬁrst three mechanisms do not inﬂuence the perceived effectiveness of information technology governance in the Brazilian public sector. However, the performance of the information technology investment portfolio management has a direct and positive consequence on effectiveness; also, as a mediating variable, it signiﬁcantly affects the performance of the information technology steering committee, increasing the effectiveness of Brazilian public information technology governance. Through the achieved results and the analysis of these effects, this study aims to contribute theoretically to the literature on information technology governance, as there are few studies that examine which mechanisms assist to establish this effectiveness in public organizations


Introduction
Brazilian public administration is inspired by the theoretical models of the New Public Management and New Public Governance, which require state efficiency together with the provision of public services of higher quality, and based on social control and warranty of individual rights (Bresser-Pereira, 2007;Imasato, Martins, & Pieranti, 2011).
In this scenario, public organizations are subject to the influence of stakeholders in establishing their institutional activities, since the interested parts affect and are also affected by the achievement of the organizations' objectives (Freeman & Reed, 1983).
Hence, in the technological field, IT governance is established as part of corporate governance, which is understood as a set of mechanisms of structures, procedures and relationships defined by top management, in order to direct IT actions and to exercise control over their use and management (Information Systems Audit and Control Association [ISACA], 2011;Peterson, 2003;Weill & Ross, 2006).
In view of this context, this research aimed to answer the following question: which non-operational mechanisms affect the level of perceived effectiveness of IT governance in public administration?
Hence, this study sought a theoretical and practical contribution on how IT governance and its mechanisms evolve in public organizations, which differ from the private sector.Government institutions are strongly influenced by the political environment and by the accomplishment of legal norms and provisions, besides having to carry out ceremonies, rites and cultural myths, rather than the search for investment remuneration and financial results (Campbell, McDonald, & Sethibe, 2009;Rodrigues & Souza, 2014;Tonelli, Bermejo, Santos, Zuppo, & Zambalde, 2015).
The next sections present the theoretical framework, the methodology used for data collection, the presentation and analysis of the data, and in the end, the final considerations, with the research limitations and suggestion for future studies.

Theoretical models of public administration: new public management and new public governance
According to Denhardt (2008), the New Public Management (NPM) had its starting point in an academic seminar held in 1968 at the Minnowbrook Convention Center, Syracuse University, New York.Its bases are related to the following factors: focus on the citizen; orientation toward results, through strategic planning and performance indicators; emphasis on transparency and social control; management flexibilization, and appreciation and development of civil servants (Hood, 1995).
For NPM, the relationship between commitment to results for the citizen and responsibility of politicians and civil servants raised the idea of effectiveness in the context of the public sector.This effectiveness moves toward more flexible organizations (reduction of bureaucracy), innovative and efficient, whose main focus for the delivery of public services is the citizen (called "customer"), who receives a more personalized treatment.
In Brazil, NPM began to be implemented in 1990, during President Fernando Collor de Mello's government (1990Mello's government ( -1992)), and became central during Fernando Henrique Cardoso's first term (1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002), through the creation of the Master Plan for Reform of the State Apparatus, which defined the guidelines and established the goals for the public administration reform, justified by the crisis about the State role (Brasil, 1995;Ferrarezzi & Amorim, 2007).
However, there are gaps when NPM makes use of private sector business principles to improve government performance.This inconsistency occurs due to the lack of denominators such as "profit" or "return on investment" in the public sector, which are common in the private environment.Therefore, it is difficult to make a common comparison across the full range of public programs, as these are based on the citizens' trust in the legitimacy of their political institutions, rather than in the market-centered efficiency (Larsen, 2008).
These gaps in NPM have generated a countermovement called New Public Governance (NPG) that places political values at the center of the debate, as opposed to the strictly instrumental focus of NPM.NPM puts much more emphasis on citizen's participation than on the efficiency of public policies' implementation (Alford, 2002;Stoker, 2006).

Information technology governance and its effectiveness
The implementation of the New Public Management reinforced the inclusion of measures that combined the achievement of the requested organizational performance and the demand for accountability (Barrett, 2003).This need becomes stronger with the New Public Governance, since it considers citizens as co-producers, who actively operate in the provision of services and decision making, by demanding excellent service rendering (Bovaird, 2007;Fotaki, 2011;Powell, Greener, Szmigin, Doheny, & Mills, 2010;Simmons, Birchall, Doheny, & Powell, 2007).
These ideas are consistent with the concept of public corporate governance, since this has, in its essence, the goal of balancing the competitiveness and productivity of institutions with a responsible and transparent management, which is reflected in the public technology area especially through its new role in the institutional setting.Currently, information technology is considered a critical and strategic asset for organizations, whether public or private, and corporations are increasingly dependent on IT (Campbell et al., 2009;Gelinas, Sutton, & Fedorowicz, 2004;Schlosser et al., 2015).
Therefore, this new concept made IT in public administration involve factors that go far beyond technology -people, structures, processes and, above all, knowledge -and all of them must be articulated so that computing resources, in fact, respond appropriately to the aspirations of public administration and society (Dunleavy, Margetts, Bastow, & Tinkle, 2006, Holden, 2007;Juiz, Guerrero, & Lera, 2014).
Thus, IT plays the role of a fundamental and transforming element of the public organization, being no longer just an object of management, but of governance.Hence, public IT enters the field of IT governance, which emerges -as a field of study and practical discipline -as a subset of corporate governance.It specifies decision rights and the framework of responsibilities in order to stimulate desirable behaviors in the use of IT, being the responsibility of the organization's board of directors and executive management (Weill & Ross, 2006).
In the specific case of the public environment, governance must follow the environment's objective, which is based on responsibilities to the House of Representatives and to taxpayers, acquisition and distribution of power in society, and citizens and stakeholders' demands for advertising the acts and transparency of public service; in other words, there are activities that do not exist in the private sector or that exceed its level of requirement (Barrett, 2003;Mello, 2006).It must be emphasized that such stakeholders are important because they have resources on which organizations depend, which give them power over these institutions (Pfeffer & Salancik, 1978).
The discussion on the effectiveness of technological governance arises from the need to evaluate the quality of results provided by IT governance in the organizations.
Regarding IT governance, effectiveness is characterized by the achievement of goals related to cost, growth, asset utilization and business flexibility, as well as meeting legal and regulatory requirements (Bowen et al., 2007).According to Weill and Ross (2006), the accomplishment of these objectives is expressed by effective IT governance that meets the following critical factors: transparency, active participation, frequent redesign, establishment of governance at multiple organizational levels, simplicity, existence of a process for treating exceptions and alignment of incentive and reward systems.
Thus, the implementation of effective IT governance involves a set of processes whose purpose is to meet the expectations regarding the organization's critical factors of success (ISACA, 2011).The effectiveness of IT governance, both in public and private environments, will be achieved depending on how the quality of the results derived from decision-making structures, processes and relational mechanisms for the direction and control of IT operations affects the accomplishment of IT goals (Sambamurthy & Zmud, 1999;Torres, 2004), and consequently will be reflected in the organizational objectives.This effectiveness will be reached through the performance of a set of arrangements and practices associated with structure, processes and relationships (Grembergen, De Haes, & Guldentops, 2004), which act in order to meet the objectives of the organization regarding information technology, and have the following meanings: In discussing which and how many mechanisms are relevant for IT governance effectiveness, this research confirms previous studies in this area, which state that among several mechanisms of IT governance listed and documented, only a limited number are truly significant and can impact the effectiveness of public technology governance.
De Haes and Van Grembergen (2008) and Lunardi et al. (2014) examined some of these surveys and concluded that there are five practices which are more effective compared to others, for example, IT steering committees and portfolio management, considering that the research was done in private institutions.On the other hand, through single case studies conducted in large organizations, Bowen, Cheung and Rohde (2007) and Ali and Green (2012) explored the factors that influence the mechanisms of IT governance, showing that its performance is associated with mechanisms such as shared understanding of the objectives between business and IT, the active involvement of IT committees in management and decisions, strategies and policies shared and communicated between business and IT.
In a recent study, Heindrickson and Santos (2014) found that mediation of the process of IT investment portfolio is necessary, so that the perceived effects of performance improvement on the IT committee and on the IT solution manager upon governance effectiveness may be statistically significant.
Therefore, based on these studies, the mechanisms described below, which have a non-operational nature, were examined in this research: 1) Relational mechanism -Top management support -the commitment of top management regarding initiatives related to IT governance enhances IT success and helps to integrate it with strategies, business processes, stakeholders' expectations, and institutional mechanisms that ensure the permanence of IT investments over time (Armstrong & Sambamurthy, 1999;Trites, 2004;Weill & Ross, 2004); 2) Structural mechanism -Performance of the IT committee -literature argues that the performance of the structural mechanism "IT committee" in planning and aligning responsibilities to the business is fundamental for the support of governance structures, planning and management of processes and information systems; hence, it is an important governance mechanism used by companies that show better performance (Doll & Torkzadeh, 1987;Earl, 1989;Gupta & Raghunathan, 1989;ISACA, 2011;Ragunathan & Ragunathan, 1989;Weill & Ross, 2006).The committee is composed of a team of executives from different areas of the organization with the main function of connecting business and IT strategies, as well as making decisions regarding projects' selection and prioritization (Castro & Carvalho, 2010;McKeen & Guimarães, 1985;Moraes & Laurindo, 2003;Nolan, 1982).Hence, by determining methods, forms of management and investment priorities in IT businesses, this mechanism has the mission of managing resources coming from the stakeholders that take part in the control and provision of institutional resources (De Haes & Van Grembergen, 2009;Hardy, 2003); 3) Processual mechanism -Performance of the IT investments portfolio management -the objective of the processual mechanism of IT investments portfolio management is to improve the use of institutional resources in order to select a set of projects and programs that can bring the highest possible return to the organization (De Haes & Van Grembergen, 2008).This mechanism includes prioritization processes, investments and projects in which business and IT are involved, and its development and existence are linked to the mechanism of IT committee (De Haes & Van Grembergen, 2009;Heindrickson & Santos, 2014).
Portfolio management and the IT steering committee are interconnected, because the committee operates especially in the phases of prioritization, authorization and review of investments to be made (Archer & Ghasemzadeh, 1999;De Haes & Van Grembergen, 2008;Heindrickson & Santos, 2014).Therefore, in the joint performance of committee and portfolio management, stakeholders express their direction and actions inside the organizations are legitimated.That happens because these mechanisms provide the form of allocation/control of IT resources and deal with the tradeoff between profitability/cost efficiency and the sustainable, reliable and stable growth of IT (Frooman, 1999;Korac-Kakabadse & Kakabadse, 2001); 4) Processual mechanism -Use of IT strategic planning -it is a formal process to define and update IT strategy, and concentrates all the methods, processes and implementations necessary for IT to successfully meet the organizational strategies (Lederer & Sethi, 1988).
Based on these mechanisms, the following hypotheses were raised: H1. Top management support has a direct and positive influence on the effectiveness of IT governance.
H2.The performance of the IT committee has a direct and positive influence on the effectiveness of IT governance.
H3.The total effect (direct and indirect effects) of the performance of the IT committee has a direct and positive influence on the effectiveness of IT governance.
H4.The performance of the IT committee has a direct and positive influence on the performance of the IT investments portfolio management.
H5.The performance of the IT investments portfolio management has a direct and positive influence on the effectiveness of IT governance.
H6.The use of IT strategic planning has a direct and positive influence on the effectiveness of IT governance.

Conceptual model
Based on the studies previously mentioned we present in Fig. 1 the model that summarizes the addressed topic, with its variables, relations and hypotheses that were tested.It points out that the mechanisms Performance of the IT Committee and Performance of the IT Investments Portfolio Management were extracted from the model adopted by Heindrickson and Santos (2014), and the mechanisms top management support and use of IT strategic planning were taken from an exploratory study conducted by De Haes and Van Grembergen (2009).

Method
This research used quantitative data collection and quantitative analysis techniques; the methodological process was divided in two phases.
First, a quantitative field research (survey) was carried out through the application of a questionnaire (Appendix A) to the universe of Brazilian federal public administration.This made the study even more complex due to the difficulty of accessing IT governance executives, which we defined as the subjects of the survey.
In the second phase, the statistical technique of structural equations was applied to the collected data, to analyze the participants' responses.
The choice of the research universe came primarily from the idea that, using the New Public Management and New Public Governance as a basis, the government should undergo a reform that has the announced mission of enhancing the efficiency of public services, improving the management of available resources and increasing its transparency, which influenced the Master Plan for Reform of the State Apparatus (Diniz, Barbosa, Junqueira, & Prado, 2009).
This influence based on efficiency led the Plan to determine, in four distinct groups (Strategic Core, Exclusive Activities, Non-Exclusive Services and Production of Goods and Services for the Market), the activities that should be carried out by the State and, for each group, the most adequate form of management to be implemented.
In our study, we focused on the groups that involve activities carried out exclusively by the State, involving organizations that belong to the Strategic Core and Exclusive Activities groups.
Strategic Core is the sector that defines laws and public policies, and demands their enforcement, being responsible for strategic decision making.It corresponds to the Legislative and Judicial, the Public Ministry and, in the Executive Branch, to the President of the Republic and the ministries responsible for the planning and formulation of public policies (Brasil, 1995).
Exclusive Activities comprise the sectors where services are provided only by the State.These are services in which the extrovert power of the State is exercised, as the power to regulate, to inspect, and to promote, and are exemplified by the collection and control of taxes, the police, and basic social security (Brasil, 1995).
Within this universe the sample was delimited to managers of the information technology area and to experienced professionals in the area of IT governance, in different types of organizations that compose the Brazilian federal public scene.
These individuals answered a questionnaire that was sent with an electronic letter of presentation.The questionnaire (Appendix A) was developed using the Google Docs tool, and the questions were elaborated based on the objectives, activities and results of the governance mechanisms studied, as well as on the attributes of IT governance effectiveness, based on academic studies and best practices of IT governance (Ali & Green, 2007;Archer & Ghasemzadeh, 1999;Castro & Carvalho, 2010;Heindrickson & Santos, 2014;ISACA, 2011;IT Governance Institute, 2007;Karimi, Bhattacherjee, Gupta, & Somers, 2000;Moraes & Laurindo, 2003;Vaswani, 2003;Weill & Ross, 2006;Weill, Woerner, & McDonald, 2009).
The queries asked for responses on a Likert scale (do not know/do not respond, totally disagree, disagree, neither agree nor disagree, agree, totally agree), and were analyzed through the structural equations technique.
Structural equation modeling is a set of techniques and procedures that are an extension of other multivariate techniques, and evaluate the dependency relations between one or more variables, representing those that cannot be measured directly by using other variables called latent constructs.
The associations of these variables can be estimated through covariance and are discriminated in the linear model, which can be separated into two: measurement model, which makes the connection between the measurement instrument (observed variables) and the theoretical construct (latent variables); and structural model, which defines the casual or association relationships between variables and unobserved constructs (latent variables), specifying whether a latent variable causes direct or indirect changes in other latent variables.Fig. 2 shows the structural/measurement model according to the graphical representation of structural equation modelling.
Following this model, we did the programming by using the R Statistics free software, with the support of Lavaan package (latent variable analysis), to make the model diagnosis and to complete the hypothesis test proposed in this research.

Results
For data collection, we sent invitations by e-mail.A total of 137 civil servants from 58 federal public organizations, who are managers of the IT area, participated in the study.They are experienced professionals in their working areas or understand the subject.Table 1 shows the distribution of respondents according to important features of the sample.The sample size rule was respected, given that it should have the minimum value of ten cases per item (manifest variable) of the questionnaire (Nunnally, 1967;Everitt, 1975); therefore, for this research the minimum number of participants should be 120.
The research ensured the confidentiality of the respondents, focusing on their perception about the connection between the mechanisms of IT governance and the effectiveness of the governance in their organizations.
With these data in hand, in addition to the conceptual and structural models, we conducted the analysis through the programming built in the R statistics free software.The initial analysis was divided in two stages: study of the model's identifiability and of the indices that reveal its global fit.
In the case of models with combined latent variables and measurement, such as the model proposed in this study, identifiability was confirmed by applying the t rule and the two-step rule (Bollen, 1989;Marôco, 2010).Regarding the forms of diagnosing the model's global fit, the following statistical tests were applied, and results are shown in Table 2.
By analyzing the various tests, we observed that only the chisquare test of adjustment is against the theoretical formulation, Another item that strengthened the suitable fit of the model and showed its acceptable explanation rate was the determination coefficient (R 2 ).The model's R 2 had the value of 0.795, that is, 79.5% of all variability of IT governance effectiveness is explained by the relationships presented in the theoretical formulation.
As to the hypothesis test, as shown in Table 3, all proposed relationships are significant (significance less than or equal to 0.05), and the marked variables were excluded from the analysis.The exceptions regarding significance were found in the following associations: in the relationship between Top Management Support (TMS) and Effectiveness of IT Governance (EITG), whose p-value is equal to 0.166; in the influence of Use of IT Strategic Planning (UITSP) on Effectiveness of IT Governance (EITG), whose p-value was 0.150; and in the relationship between Performance of IT Committee (PITC) and Effectiveness of IT Governance (EITG), whose p-value is equal to 0.980.The Performance of IT Committee (PITC) on the Performance of IT Investments Portfolio Management (PITIPM) is significant, with a p-value of 0.001; and also significant is the influence of Performance of IT Investments Portfolio Management (PITIPM) on Effectiveness of IT Governance (EITG), since its p-value is 0.016.
Thus, we can conclude that the observed relationship between the independent variable Performance of IT Committee (PITC) and the dependent variable Effectiveness of IT Governance (EITG) is more effective when this linkage occurs through the mediator variable Performance of IT Investments Portfolio Management (PITIPM) than when there is a direct influence of the committee (PITC) on the Effectiveness of IT Governance (EITG).
Our understanding is strengthened when we observe the negative regression coefficient of the relationship between Performance of IT Committee (PITC) and Effectiveness of IT Governance (EITG) (estimated value = −0.0301), in opposition to the positive coefficient derived from the multiplication of the regression coefficients of the influence of the committee (PITC) on the investments portfolio management (PITIPM), and the influence of PITIPM on EITG (estimated value of total effects = 0.256 × 0.393 = 0.101).
Continuing the analysis, again it is possible to notice the nonsignificant role of IT Committee (PITC) on Effectiveness of IT Governance (EITG) when examining the variances of the system components, since this mechanism has a non-significant variance (p-value = 0.084).
Therefore, based on the observed statistical evidence, Fig. 3 presents the structural model with the estimates and significance of each hypothesis -where the rejected hypotheses are written in red.

Discussion and final considerations
This study sought to identify and understand the achievement of goals defined by the strategies of the organization for IT governance, through the relationships between the mechanisms of IT governance and its effectiveness.
The hypothesis that top management support has a direct and positive influence on the effectiveness of IT governance was rejected.This happens because the contemporary Brazilian public administration is not fully built according to the New Public Management and New Public Governance models, and shows a combination of features derived from patrimonialism and the Weberian bureaucratic model.New Public Management and New Public Governance idealize top management based on technical and meritocratic competences, inducing modernization through the use of information technologies, and supporting government programs that use technology to promote performance, efficiency, effectiveness, transparency and quality in the relationships with citizens and other stakeholders (Agune & Carlos, 2005;Bresser-Pereira, 2007;Grant & Chau, 2005;Tan and Pan, 2003).And this does not fully happen in the Brazilian federal public sector.
About the Performance of the IT Committee, we emphasize that it is a structural mechanism; therefore it is involved with the organizational composition for the functioning of IT governance, which implies the need for arrangements.And these arrangements are built differently in public and private organizations.
In the private sector, there is a clear definition of the roles and responsibilities of the committees, as they are created with the main objective of monitoring and directing the company's information technology.Having defined their goal, the executives consequently execute their functions within boundaries established inside the group, in order to give return (profit) to the institution.This way of operating explains the positive influence of the performance of the IT committee observed in previous studies in the private environment (Ali & Green, 2007, 2012;Karimi et al., 2000;Lunardi et al., 2014;Vaswani, 2003).
In the public sector this is not clear, because there are many levels of responsibility (and interests) inside and outside the organizations, at local, state and federal levels, which increase tensions and conflicts in doing business (Bermejo et al., 2014;Campbell et al., 2009;Edwards & Clough, 2005).Therefore, the hypothesis that the performance of the IT committee has a direct and positive influence on the effectiveness of IT governance was rejected.
Given this situation, the research found that only through mediation of the IT Investments Portfolio Management the perceived effects of improved performance of the IT committee on effectiveness become statistically significant (H3, H4 and H5 hypotheses).
This interaction of mediation and the significant relationship between IT committee and portfolio management is important and affects the model as a whole, probably because when dealing with the performance mechanism in IT Investments Portfolio Management we use a criterion common to the budget area.
Budget issues may have a significant impact on the process of formulating IT governance policies in the public sector.After all, the budget process is the arena where the allocation of scarce resources of the society is determined and, as such, it is a key element of the policy-making process.Even if the policies have already been approved in other levels, they must go through the budget area to ensure that the necessary resources are available for implementing them effectively (Banco Interamericano de Desenvolvimento [BID], 2007).
Hence, we observe the importance of the performance of IT investments portfolio management, since it aims to prioritize projects and investments, deciding how to split scarce resources among them, taking into account resource limitations and business needs, by aligning them with organizational strategic objectives and considering the high degree of uncertainty associated with the use of technologies (Hall & Liedtka, 2005;Xue, Liang, & Boulton, 2008).
Therefore, the IT committee alone does not impact effectiveness, and only achieves this goal when it receives the responsibility of aligning IT actions and investments, justifying technology investments in face of a rigid budget structure (Kumar, Ajjan, & Niu, 2008).
In addition, the Federal Court of Auditors, institution of the Brazilian government responsible for external control, recommends that public organizations establish an IT committee to provide the allocation of public resources according to their needs and priorities (Brasil, 2008).
Regarding the use of IT strategic planning, there is a difference between discourse and practice.In the speech, strategic planning is said to be the main method for defining actions; however, in practice it happens through institutional processes of isomorphism, best practices, and benchmarking.These processes proliferate in the environment of public institutions, which are systematized and interconnected in a discretionary way that encourages the adoption of models and practices for reasons that go far beyond efficiency (Bermejo, Tonelli, Zambalde, Brito, & Todesco, 2012;Zattoni & Cuomo, 2008).

Limitations and suggestions for future research
This study has restrictions regarding the size of the sample, which is small due to the number of Brazilian federal public organizations that belong to the 'strategic core' and 'exclusive activities' groups.However, we should highlight as a positive aspect the use of the respondents' opinions to evaluate the importance of the questionnaire items and their measure.
Concerning suggestions for future research, we propose studies that use Likert type scales and ranking categories directed to experts on the subject; in other words, scales that do not accept absence of answers, since respondents would have the ability and technical expertise to express relevant opinions.
There is also a need for studies on the effectiveness of IT governance and its mechanisms in government institutions that offer non-exclusive services and produce goods and services for the market, as well as expanding the research to state and local levels.
Finally, we suggest as future research the replication of this study with the purpose of validating its results and overcoming the limitations previously mentioned.

1)
Structural mechanisms -involve the organization and placement of the IT area in the hierarchical structure of the institution, as well as the clear definition of roles and responsibilities of the positions that compose this structure (De Haes & Van Grembergen, 2008); 2) Processual mechanisms -deal with strategic decisionmaking, IT strategic planning and frameworks for monitoring, control and processes (De Haes & Van Grembergen, 2008); 3) Relational mechanisms -complement IT governance, incorporating IT relations with other areas of the organization and their users.These mechanisms guarantee the functioning of the structure created for IT and of the established processes (De Haes & Van Grembergen, 2008).

Fig. 3 .
Fig. 3. Structural model with estimated values and significance of hypotheses.

Table 1
Characteristics of the sample.

Table 2
Results of the diagnostic tests for the model's fit.
a Obs: p-value ≤ 0.05.since most of the other tests have confirmed the model's adequate fit.

Table 3
Results of the relationship between the model variables.
a The p-value has significance level of 5%.