Abstract
The universe of evaluation in Brazil is very plural and multifaceted. Many sectors within the society use the concept with different meanings, with different practices and with quite different levels of development. Addressing this issue in its entirety is certainly a challenge. There is, therefore, a need to attempt to bring up this multiplicity: government, civil society, consultants, international agencies, private foundations, academy, researchers, for one same job. Convergence can be a way of promoting dialogue between these actors and possibly build a more integrated future for evaluation in the country. Our starting point will be the normative aspects and the advances that the country has made in this direction, then we will go through more practical aspects within the governmental and civil society spheres regarding the use of the evaluations performed. Finally, we will deal with professional initiatives in the country.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
According to the website of the Superior Electoral Court at http://www.tse.jus.br/partidos/partidos-politicos/registrados-no-tse. Accessed in May 2020.
- 2.
For more information: https://www.imf.org/en/Publications/WEO/Issues/2018/09/24/world-economic-outlook-october-2018. Accessed in May 2020.
- 3.
For more information, see: Income Inequality in the Brazilian Territory at http://repositorio.ipea.gov.br/handle/11058/5291. Accessed in May 2020.
- 4.
Social movements for land and housing, for example, are often criminalised. See: Grzybowski 1991.
- 5.
Five families control half of the 50 media outlets with the largest audience in Brazil. The conclusion is from the Media Ownership Monitor survey, funded by the German government and carried out jointly by the Brazilian NGO, Intervozes, and Reporters Without Borders, based in France.
- 6.
For more information see Sodré 1999.
- 7.
See World Bank’s Country Partnership Strategy for Brazil at https://www.worldbank.org/en/news/press-release/2017/07/13/brazil-world-bank-group-presents-country-partnership-strategy. Accessed in May 2020.
- 8.
In March 2021 it was approved an important change in the normative status in Brazil, giving the evaluation of public policies constitutional status. The Constitutional Amendment Nº 109 amended Articles 37 and 165 of the Constitution. A new paragraph (16) of Article 37 was included providing that: “The organs and entities of the public administration, individually or jointly, shall conduct evaluation of public policies, including disclosure of the object to be evaluated and the results achieved, according to the law.” The main purpose of the Constitutional Amendment was to allow the payment of emergency aid in Covid-19 context throughout 2021 and create instruments for future tax adjustments, institutionalising a process which was already in course since 2016 (Amendment Nº 95), and compensating part of the extra expenses with the aid. The institutional development of monitoring and evaluation is, therefore, in dialogue with this matter. After the approval of the Constitutional Amendment, it is possible to observe updates of evaluation reports on CMAP's website as well as the publication of an Annual Report of Evaluation of Public Policies (https://www.gov.br/fazenda/pt-br/orgaos/secretaria-de-avaliacao-planejamento-energia-e-loteria/documentos/cmap/relatorio-anual-de-avaliacao-de-politicas-publicas/view, accessed on 12 October 2021) by the new Secretariat of Evaluation, Planning, Energy and Lottery, covering CMAP’s work since its creation with the results of 16 evaluations of public policies in different sectors. The report brings information about the ongoing process of institutionalisation of M&E in the country, with CMAP in the centre and IPEA, ENAP and IBGE as main support partners. CGU is the governmental organ responsible for monitoring the recommendations of the evaluations. Also, the report indicates the challenge of systematising the evaluations results with the country’s financial and budgetary process (article 165, paragraph 16 of the Amendment Nº 109). Future actions are planned in the direction of further regulating the Constitutional Amendment Nº 109. A deeper analysis of the impact of this Amendment in the M&E context is necessary, which has not been conducted in the context of this chapter.
- 9.
For more information, see the report: Federal Public Policy Monitoring and Evaluation Committee Studies and Proposals http://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/181127_comite_de_monitoramento_inicio.pdf. Accessed in May 2020.
- 10.
Some important publications for these periods:
-
Guia de Análise Ex Post (BRAZIL, Casa Civil et al. 2018a), prepared by the Civil House of the Presidency of the Republic, by the Ministry of Finance (at the time Ministry of Finance and Ministry of Planning, Development and Management), by the CGU (at the time Ministry of Transparency and General Controllership by the Union) and by the IPEA, with the participation of other agencies;
-
- 11.
Term well detailed by Oliveira 2019, p. 23.
- 12.
CMAS Resolution, Nº 01, 30 April 2019 at https://www.gov.br/economia/pt-br/acesso-a-informacao/participacao-social/conselhos-e-orgaos-colegiados/cmap/publicacoes/atas-e-resolucoes-1/resolucoes/99-resolucao-cmas-no-01. Accessed in June 2020.
- 13.
Available on SAGI’s website https://aplicacoes.mds.gov.br/sagi/portal/index.php?grupo=164. Accessed in June 2020.
- 14.
The ordinance, among other actions, limits SAGI’s capacity for analysis as the first paragraph of article 9º stands “§1 SAGI will not comment on the criteria of opportunity or convenience of the MDS Secretariats” (Art 9º Ordinance Nº 2,227, 6 June 2018).
- 15.
Analysis of data available at: https://aplicacoes.mds.gov.br/sagi/portal/index.php?grupo=182. Accessed in May 2020.
- 16.
Available at: http://portal.inep.gov.br/web/guest/dados/monitoramento-do-pne/relatorios-de-monitoramento. Accessed in June 2020.
- 17.
- 18.
Institute linked to the Secretariat of Economy and State Planning with the purpose of producing knowledge and subsidising public policies through the elaboration and implementation of studies, researches, plans, projects and organisation of statistical and georeferenced databases, in the state, regional and municipal spheres, focused on the socioeconomic development of Espírito Santo.
- 19.
Ceará Decree Nº 29,910, 29/09/2009.
- 20.
https://ibase.br/pt/noticias/ibase-lanca-cartilha-sobre-portos-no-rio-tapajos/. Accessed in May 2020.
- 21.
- 22.
In 2001 the Bolsa-Escola programme was inaugurated and expanded by the PDSCF. The Bolsa Família, is therefore part of PDSCF, the central strategy of the Lula-Dilma governments.
- 23.
And here, there is a whole debate as to whether the methodology proposed by the study was the most appropriate to evaluate the programme.
- 24.
- 25.
For more information on the programme, some authors such as Rogério Silva, Marco Akerman, and Oswaldo Tanaka are essential.
- 26.
They are: usefulness, opportunity, feasibility, reliability, objectivity, directionality (Tanaka & Tamaki 2012).
- 27.
As participatory planning, through the cooperation of representative associations in municipal planning, as a precept to be observed by municipalities (Brazilian Supreme Federal Court (1988[1994]), Constitution of the Federative Republic of Brazil, Art. 29, XII); democratic management of public education in the area of education (Brazilian Supreme Federal Court (1988[1994]), Constitution of the Federative Republic of Brazil, Art. 206, VI); administrative management of Social Security, with the quadripartite participation of governments, workers, entrepreneurs and retirees (Brazilian Supreme Federal Court (1988[1994]), Constitution of the Federative Republic of Brazil, Art. 114, VI).
- 28.
This approach is also very well addressed by Goldman, I., & Pabari, M. 2020, p. 2.
- 29.
The early childhood is a topic that has significant political force in Brazil, there is a diverse network of the organised civil society that massively pressures the governments for municipal policies and plans for the early childhood. The network was one of the parties responsible for Law Nº 13,257, 2016 Early Childhood Legal Framework and for the Frente Parlamentar (Parliamentary Group on Early Childhood), for example.
- 30.
For a better understanding on how the large media covered the campaigns, we used Google News as search engine. This is an engine that finds news published in newspapers, magazines, portals and other medias specialised in publishing news. It does not show commercial websites that use such tool for marketing, for example. We analysed the 100 main newspaper reports that addressed the program from 2017 to 2020.
- 31.
Examples of newspaper headlines about the program:
-
“Criança Feliz undergoes globally unprecedented scientific evaluation” https://diariodopoder.com.br/politica/programa-crianca-feliz-passa-por-avaliacao-cientifica-inedita-no-mundo. Accessed in May 2020.
-
“Criança Feliz wins the Wise Awards” https://www.correiodopovo.com.br/not%C3%ADcias/pol%C3%ADtica/programa-crian%C3%A7a-feliz-recebe-o-pr%C3%AAmio-wise-awards-1.381616. Accessed in May 2020.
-
“Government wants to attend to one million children by the end of the year with the Criança Feliz program” https://www.camara.leg.br/noticias/581077-governo-quer-atender-um-milhao-de-criancas-ate-o-fim-do-ano-com-programa-para-primeira-infancia/. Accessed in May 2020.
-
“Lack of funds trigger public policy evaluation” https://www1.folha.uol.com.br/mercado/2017/11/1932367-cofres-vazios-impulsionam-a-avaliacao-de-politicas-publicas.shtml. Accessed in May 2020.
-
“The use of Michelle's image bothered Bolsonaro” https://www.terra.com.br/noticias/brasil/politica/uso-da-imagem-de-michelle-por-osmar-terra-incomodava-bolsonaro,7026b7574428d9eb06f955a17e0112d8vvv4df0q.html. Accessed in May 2020.
-
Abbreviations
- CGU:
-
Comptroller-General of the Federal Union
- CLEAR LAB:
-
Centre for Learning on Evaluation and Results for Lusophone Africa and Brazil
- CMAP:
-
Committees for Monitoring and Evaluation of Federal Public Policies
- CMAS:
-
Monitoring and Evaluation of Union Subsidies
- CMAG:
-
Monitoring and Evaluation of Direct Expenditures
- DAPP FGV:
-
Getulio Vargas Foundation Public Policy Analysis Board
- ENAP:
-
National School of Public Administration
- ENSP:
-
Escola Nacional de Saúde Pública Sergio Arouca
- ESAF:
-
Government Budgeting Management School
- FIOCRUZ:
-
Oswaldo Cruz Foundation
- FECOP:
-
State Fund for Combating Poverty
- GDP:
-
Gross Domestic Product
- GEF:
-
Executive Management of FECOP
- GIFE:
-
Group of Institutes Foundations and Companies
- GTAG:
-
Interministerial Working Group for Monitoring Public Spending
- IBGE:
-
Brazilian Geographic and Statistical Institute
- IDB:
-
Inter-American Development Bank
- IFAD:
-
International Fund for Agricultural Development
- INEP:
-
National Institute for Educational Studies and Research Anísio Teixeira
- IPEA:
-
Institute of Applied Economic Research
- IBASE:
-
Brazilian Institute of Social and Economic Analyses
- LDO:
-
Budget Guidelines Law
- MDS:
-
Ministry of Social Development
- MEC:
-
Ministry of Education
- NGO:
-
Non-governmental Organisation
- OECD:
-
Organisation for Economic Co-operation and Development
- PEC:
-
Constitutional Amendment Proposal
- PDSCF:
-
Social Development and Fight Against Hunger Programme
- PL:
-
Law Proposal
- PMAQ:
-
The National Programme for Improvement of Basic Care Access and Quality
- PNE:
-
National Education Plan
- PPA:
-
Multiannual Plan
- RBMA:
-
Brazilian Monitoring and Evaluation Network
- RePP:
-
Report on Government Policy and Programme Monitoring
- SAGI:
-
Secretary for Evaluation and Information Management
- SIAPRE:
-
Public Policy Monitoring and Evaluation System and Public Expenditure Review
- SIMAPP:
-
Public Policy Monitoring and Evaluation System
- SINAES:
-
National System of Higher Education Evaluation
- SUS:
-
Unified Health System
- TCE:
-
State Court of Accounts
- TCU:
-
The Federal Court of Accounts
- UNDP:
-
United Nations Development Programme
- USP:
-
University of São Paulo
References
Abranches, S. (1988). Presidencialismo de coalizão: O dilema institucional brasileiro, Dados: Revista de Ciências Sociais, 31(1), 3–55. IUPERJ.
BRAZIL. [Constituição (1988)]. Constitution of the Federative Republic of Brazil. Brasília, DF: Presidência da República, [2016].
BRAZIL, Casa Civil et al. (2018a). Avaliação de Políticas Públicas: Guia. Prático de Análise Ex Post. Instituto de Pesquisa Econômica Aplicada.
BRAZIL, Casa Civil et al. (2018b). Avaliação de Políticas Públicas: Guia. Prático de Análise Ex Ante. Brasília: Instituto de Pesquisa Econômica Aplicada.
BRAZIL, Ceará State Government. Decree Nº 29,910, of September 29, 2009.
BRAZIL, Decree No. 9834, of June 12, 2019. Instituito conselho de monitoramento e avaliação de políticas públicas.
BRAZIL, Espirito Santo State Government. State Law Nº 10,744/2017.
BRAZIL, IPEA. (2002). Cooperação Técnica BID-Ipea: Fortalecimento da Função Avaliação nos Países da América do Sul. Relatório técnico / Pedro Luiz Barros Silva e Nilson do Rosário Costa. Instituto de Pesquisa Econômica Aplicada. https://www.ipea.gov.br/portal/index.php?option=com_content&view=article&id=5437. Accessed on 30 May 2020.
BRAZIL, IPEA. (2020). Boletim de Análise Político-Institucional. Publicações-Boletim Institucional (ipea.gov.br). Accessed on May 2020.
BRAZIL, Ministry of Citizenship. (2019). Notebook 34, Criança Feliz implementation evaluation. https://aplicacoes.mds.gov.br/sagirmps/ferramentas/docs/caderno-estudos-34_210x280_20200124-digital.pdf. Accessed on 30 May 2020.
BRAZIL, Ministry of Economy. (1990). Law Nº 8,029, 1990.
BRAZIL, Ministry of Education. (2004). Law Nº 10,861, of April 14, 2004.
BRAZIL, Ministry of Education. (2007). Ordinance Nº 660/201, 2007.
BRAZIL, Ministry of Health. (2015). Ordinance Nº 1,645 of October 2, 2015
BRAZIL, Senado Federal. (2017). Project of Law Nº 488/2017.
BRAZIL, Social Development Ministry. Ordinance Nº 2,227, of June 6, 2018.
BRAZIL, The Federal Court of Accounts. (2019). Relatório de Fiscalizações em Políticas e Programas de Governo (RePP). Tribunal de Contas da União. https://portal.tcu.gov.br/data/files/17/B0/92/77/E933071076A7C107E18818A8/Relatorio_politicas_programas_governo_2019.pdf. Accessed on 5 November 2020.
Câmara dos Deputados. (1993). Art. 2º Resolution of Deputy Chamber Nº 48. Legislative Consultancy.
Chianca, T. K., & Ceccon, C. (2017). Pedagogy in process applied to evaluation: Learning from Paulo Freire’s work in Guinea-Bissau. New Directions for Evaluation, 155(1), 79–97.
Chianca, T. K., Patton, M. Q., & Ceccon, C. (2018). Evaluative thinking in practice: Implications for evaluation from Paulo Freire’s work. Journal of Multidisciplinary Evaluation, 4(30), 1–15.
Grzybowski, C. (1991). Caminhos e descaminhos dos movimentos sociais no campo. Vozes.
Goldman, I., & Pabari, M. (Eds.). (2020). Using evidence for policy and practice-Lessons from Africa. Taylor & Francis Group.
Jannuzzi, P. (2020). Informação estatística e políticas públicas no Brasil: contribuições de pesquisas do IBGE para as políticas de desenvolvimento social e combate a fome (2004–2014), Universidade do Estado do Rio de Janeiro.
Jannuzzi, P., Ruediger, M., & Meirelles, B. (2018). Análise da efetividade do água para todos: Avaliação de mérito do programa quanto à eficácia, à eficiência e à sustentabilidade. DAPP FGV.
Joppert , M., Heriques, A., Pinho, J., Azevedo, J., Newman, J., Wenceslau, J., & Carvalho, S. (2011). Brazilian monitoring and evaluation network: Creation, Development and Perspectives. IOCE. https://www.ioce.net/download/national/Brazil_BMEN_CaseStudy.pdf. Accessed on 30 September 2020.
Limongi, F. A. (2006). Democracia no Brasil. Presidencialismo, coalizão partidária e processo decisório. Novos Estudos. CEBRAP, 76, 17–41.
Lopes E., Scherer, M., & Costa, A. (2015). O programa nacional de melhoria do acesso e da qualidade da atenção básica e a organização dos processos de trabalho. Tempus.
Madueno, J. (2019). Current state of the institutionalization of evaluation in Brazil (Term paper), Saarland University.
Matida, A. H., & Camacho, L. (2004). Pesquisa avaliativa e epidemiologia: Movimento e síntese no processo de avaliação de programas de saúde. Cadernos De Saúde Pública, 20(1), 37–47.
Matuscelli, D. E. (2010). Ideologia do presidencialismo de coalisão. Lutas Sociais, São Paulo, 24, 60–69.
Melo, L., Martiniano, C. S., Guimarães, J., Souza, M. B. D., & Rocha, P. D. M. (2016). Análises das diretrizes para o apoio institucional das gestões da Atenção Básica das capitais brasileiras. Saúde debate.
Oliveira, P. (2019). O desafio da coordenação executiva pelo centro de governo: a experiência do Comitê de Monitoramento e Avaliação de Políticas Públicas (CMAP) de 2016 a 2017, Escola Nacional de Administração Pública.
Silva, R., Furtado, J. P., Akerman, M., & Gasparini, M. (2015). Subsídios a meta: avaliações do PMAQ. In Práticas de avaliação em saúde no Brasil: Diálogos.
Sodré, N. W. (1999). História da imprensa no Brasil (4th ed.). Mauad.
Tanaka, O. Y., & Tamaki, E. (2012). O papel da avaliação para a tomada de decisão na gestão de serviços de saúde foi publicado na Revista Ciência & Saúde Coletiva, 17(4) Rio de Janeiro, editada pela Associação Brasileira de Pós-Graduação em Saúde Coletiva (ABRASCO).
Torres, J. C. (1996). O encaminhamento político das reformas estruturais. Lua Nova, 37, CEDEC, pp. 57–76.
Vaitsman, J., Rodrigues, R. W. S., & Paes-Sousa, R. O. (2006). Sistema de avaliação e monitoramento das políticas e programas sociais: a experiência do Ministério do Desenvolvimento Social e Combate à Fome. MDS. Unesco. Management of Social Transformations. Policy Papers, n. 17.
W. K. Kellogg Foundation (2019). Learnings and legacies from the field. https://wkkf.app.box.com/s/1kncpgnp0g8w6gplifh9dufj8x1dcrnm. Accessed on 23 October 2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Dolabella, V.J. (2022). Evaluation in Brazil. In: Stockmann, R., Meyer, W., Szentmarjay, L. (eds) The Institutionalisation of Evaluation in the Americas. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-81139-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-81139-6_4
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-81138-9
Online ISBN: 978-3-030-81139-6
eBook Packages: Political Science and International StudiesPolitical Science and International Studies (R0)