The methodology of the Logical Framework with a Risk Management Approach to Improve the Sustainability in the International Development Projects

Abstract

When working with international development projects (IDPs), the use of the logical framework approach (LFA) prevails as the most important tool to plan and manage these projects. This paper presents how the methodology has been enriched, including risk management (LFRMA logical framework with risk management approach), proposing an original contribution, tested with professionals that will improve the effectiveness of IDPs by increasing their success rate and their sustainability. The steps followed to design the methodology (problem statement (literature review, interview with experts, questionnaire for professionals. and statistical analysis), case study analysis (eight case studies in Colombia, interviews with IDPs managers, focus groups, questionnaire for participants, qualitative analysis, and fuzzy analysis) and design of LFRMA (focus group with experts)) and the methodology itself (how to introduce risk management during all the life cycle through the methodology steps) are presented. Conclusions answer the research questions: can the effectiveness and sustainability of IDPs be improved? Can risk management help to improve IDPs effectiveness? Would it be useful to introduce risk management into the LFA? The LFRMA methodology consists of two fields of application, the first at the organization level and the second at the project level.

1. Introduction

Reviewing the standards in Project Management inside and outside international development, it is easy to see that, in the world of international development projects (IDP), the use of the logical framework approach (LFA) prevails, while in organizations as well-known as the Association for Project Management (APM) or the Project Management Institute (PMI), it is not even mentioned. This has led to the appearance of specific guides for these IDPs, such as PM4DEV [1] or PM4NGOs [2], which are more focused on the social aspects of projects. For example, as shown by the studies of Munro and Ika [3], there is hardly any reference to the word gender in traditional guides such as the PMBOK [4], and, of course, none make, for example, reference to the gender approach in projects [5] so highly valued today in development cooperation projects.

On the contrary, some topics so typical of project management, such as risk management, are hardly dealt with in the guides dedicated to the LFA. This research aims to fill this gap and to build bridges between the theory of project management and the management of international development projects. This paper summarizes the three years of work on this research around a doctoral thesis [6] and starts from the following three research questions (RQ):

RQ1: Can the effectiveness and sustainability of IDPs be improved?

RQ2: Can risk management help to improve IDPs effectiveness?

RQ3: Would it be useful to introduce risk management into the LFA?

These three questions, in turn, give rise to three research hypotheses (RH):

RH1: The LFA needs to be improved.

RH2: The success of IDPs depends on effective risk management.

RH3: The Logical Framework with a Risk Management Approach (LFRMA) can improve the effectiveness of IDPs.

1.1. Frameworks in International Development Projects

Although the study of Project Management in IDP has not yet reached the level of other sectors [7], the studies that exist place project cycle management (PCM) as the most widely used methodology for the management of these projects. This methodology is inseparable from the LFA, which is now over 50 years old, and although it has received much criticism, mainly due to its rigidity [8,9,10], it still represents a common language in development cooperation [11,12,13,14,15,16].

The LFA was born in 1969 when the American Agency for International Development (USAID) commissioned the consulting firm Practical Concepts Incorporated to analyse its project evaluation system. The main finding of this study was that the planning was too indefinite, which prevented not only the evaluation of what was planned compared to what was performed but also the relation of the activities to the achievement of the objectives, as well as the discovery of the contribution of the project in the generated impact [17]. The LFA was the solution offered by the consultant in response to these problems, presenting it as a methodology for the design and evaluation of projects.

The concepts used by the LFA are largely based on the previous experience that Rosenberg, who can be considered the father of this approach, had gained in the internal management of military projects, as well as in the planning of the first satellite launches and the space programs of the American Space Agency [17,18]. These origins marked the strong hierarchical character and the focus on the precise definition of the objectives [12]. Initially, the LFA was only a matrix very closely guided by these principles of planning by objectives. Therefore, a general objective was proposed, followed by a specific objective, then expected results and activities. For each of these objectively verifiable indicators, sources of verification and assumptions were included.

In the 1980s, the German Cooperation Agency (GIZ, previously GTZ) proposed a more participatory and extended version of the LFA, adding four previous steps to the logical framework matrix: (i) analysis of participants; (ii) analysis of problems; (iii) analysis of objectives or solutions; and (iv) analysis of alternatives. The fifth step would be to develop the previously used logical framework matrix [19]

Currently, there are many guidelines (see for instance [16,20,21]). However, the methodology is sometimes used only to meet donor requirements rather than as a method to manage IDPs.

The LFA is a planning methodology by objectives, defined in this way precisely to solve the first problems observed in the evaluation of the USAID projects, where the fulfilment of activities derived in the achievement of objectives that were far from the initial goal.

The main difference between planning by objectives and planning by activities was explained by Baccarini [22]. Planning from the top down provides the answer to the question of how not to lose sight from the initial moment to which the project wants to contribute. Planning from the bottom up, or by activities, would instead answer the question of why. All planning by objectives must start from the analysis of a problem, which gives rise to a solution or an objective, which needs some activities to be fulfilled. That is, before planning the activities, it is necessary to carry out the analysis of problems and objectives [21].

1.2. Risk Management in International Development Projects

Mainly, research on risk management has been conducted in developed country contexts, as reported by Hartono et al. [23]. However, according to Wang, Dulaimi, and Aguria’s research [24], country-level risks are the ones that affect a project the most.

Frequently, risks are divided into internal risks (to the project itself) and external risks. While internal risks are relevant to all projects, regardless of whether the project is developed locally or internationally, international projects are subject to greater exposure to external risks [24]. Hillson [25] found the mere fact of being implemented in a developing country to be a direct cause of risk, and IDPs are mainly implemented in developing countries. Some authors emphasize the greater existence of risks in the context of IDPs, as they are surrounded by uncertain environments that affect the development of the project [26,27,28,29,30].

Although the PMD Pro guide [2] devotes a chapter to risk management, the scarce literature dedicated to PM in IDP hardly devotes any attention to it, except for Couillard et al. [8] and Ika and Lytvynov [10], who propose adding risks in the Logical Framework Matrix (LFM). This absence has been claimed by the World Bank, which considers risk management imperative in the management of the IDP, whose ultimate goal is economic development and poverty eradication [31].

In addition to the environment itself, uncertainty and risks are two concepts inherent to the LFA [14]. This is because the IDP itself is full of internal uncertainties, given that the behaviour of the different participants in the project is unpredictable [32]. Other authors had already established that project participants are the greatest source of uncertainty at all stages of a project [33,34].

Attending to and understanding the project participants involves knowing their culture. Many authors find culture-related factors as determinants when executing an international project [35,36,37,38]. Hartono et al. [23] and Rodríguez-Rivero et al. [39] highlight that these factors are especially relevant for IDPs.

Given the uncertainties of the environment and of the project participants, it seems clear that the maxim attributed to Eisenhower, plans are nothing, planning is everything, prevails for IDPs. Therefore, the need to adapt the initial planning to deal with unforeseen events seems necessary to achieve success in these projects. Precisely, the rigidity of the LFA has been one of the most criticized aspects of this methodology [9,12,15,40]. The methodology proposed in this research introduces a continuous review based on risk management that can be very useful to achieve project success. It has already been demonstrated by some research that directly links success in project impact to appropriate risk management [39].

2. Materials and Methods

The research has been divided into three phases, presented in Figure 1.

2.1. Phase 1: Problem Statement

First, an extensive collection of documents was reviewed, including more than 150 articles from indexed journals, related, in most cases, to Project Management; reports from the main international organizations (World Bank, United Nations, etc.), as well as from International Cooperation Agencies; and books, both in the field of project management and in the area of sustainable development.

Secondly, a study was carried out of a total of 152 IDPs financed by one public university in Spain between 2004 and 2014. In this study, their managers were asked to assess the risks the project faced.

The knowledge acquired after these first two stages made it possible to design a questionnaire to contrast with professionals in the international development sector in Spain, the strengths and weaknesses detected of the Logical Framework. This initial approach was contrasted in a 2 h face-to-face interview with the head of International Cooperation at Oxfam Intermon.

The final questionnaire consisted of a first section to collect information of the organizations for which the professionals worked (three questions); a second section to know the experience of the person surveyed in IDPs (four questions); and a third section designed to know the use of the steps of the methodology and his/her opinion on the Logical Framework (15 questions). The questionnaire was measured on a scale of 1–4 (completely disagree, disagree, agree, and completely agree), also with a possibility for Don’t know/No answer. This scale was chosen based on the well-known Likert scale, but without including the central value, to eliminate the tendency to be at this value, following statistical recommendations [41] and as already contemplated by some authors [42,43]. In addition, a final open question was included so that the participants could include their opinion about the LFA and its fields of improvement.

The data of this questionnaire were studied through descriptive statistics in the first stay to continue analysing the correlations between the 15 statements and finalize by performing an analysis of variance (ANOVA). To carry out the ANOVA, groups were formed according to the different characteristics of the people surveyed (for example, type, size, and experience of the organization), and the differences between the means for each of the groups were verified, with an interval of 95% confidence.

2.2. Phase 2: Case Studies

As a case study, the Department of Cauca (Colombia) was chosen, after having had the opportunity to participate in an IDP executed in the area and having been able to appreciate that Colombia is going through a very complex moment and that the Cauca population presented a high level of vulnerability, together with great ethnic and cultural wealth. This context was very suitable for the study to be carried out.

The information provided by the coordinators of eight IDPs in Cauca was compiled and analysed by researchers. For each of them, focus groups were held with the people who participated in the projects. Interviews were also conducted with the coordinators of these projects to contrast the results obtained with the work of the focus groups and to know first-hand how the projects were managed and especially how risks were handled, what difficulties they encountered, and what would change for the future projects.

The focus groups with the participants were held in the field, in sessions with an average duration of three hours. During these sessions, risk management was evaluated, as well as two different perspectives of the success of the project: impact on the community, and project management itself in achieving objectives.

Participants individually defined project success, being able to use up to three definitions per person, each one written on a post-it. These definitions were subsequently grouped by all participants in a consensual manner on a maximum of three factors.

Next, and following the same dynamic, each participant defined the project objectives individually, and then, as a group, prioritized them. Afterward, together, the participants were asked what they would have done differently if the project started again and what tools they would have used. The aim here was to know the lessons learned with each project.

The session continued with risk identification and assessment. The results obtained were contrasted with the project coordinators, who had also taken part in the focus group.

The results of these workshops were analysed first qualitatively and secondly through the fuzzy methodology.

2.3. Phase 3: Design of the Logical Framework with a Risk Management Approach

Based on the study carried out and the results obtained, a participatory methodology is designed based on the incorporation of risk management into each of the five steps of the LFA.

This methodology is contrasted with eight professionals in the sector. Among the 29 people willing to participate, these eight were selected by researchers to have multidisciplinary and heterogeneous points of view, trying to cover all the possible actors involved in cooperation interventions.

3. Results and Discussion

The methodology of the LFRMA responds to the need contrasted with professionals, through the LFA questionnaire, to introduce elements of risk management into the methodology par excellence in development cooperation interventions, the LFA.

The LFRMA methodology consists of two fields of application, the first at the organization level, and the second at the project level. Figure 2 shows the flow of the methodology. The left side shows what would correspond to the introduction of risk management in the organization, through the risk management plan. The step between risk management in the organization and risk management in the project is performed through the TTT analysis (team, project type, and terrain) [6]. In the risk management of a project, it will be necessary to apply everything learned from the risk management of previous projects, and in turn, everything identified in the project in progress must be applied to future projects.

Risk management must be participatory and will affect all phases of the project (identification, design, execution, monitoring, and closure), expressed in different colours. The same colours also show the stages of the LFRMA.

Risk management should be included in the culture of the organization, and it can start with the elaboration of a Risk Management Plan, in which methods, roles, and responsibilities must be established, including the scales to assess the probability of occurrence and the impact of a risk. Risk categories and formats will also be defined. This plan must also address risk response strategies, as well as risk control tools. This would allow the organization to consider risk management not only at the project level but also at the strategic level.

Previous to project definition, risks linked with the management of the project team and with the environment where the project will be developed and should be identified and analyzed. It is recommended to use the analysis TTT (team, typology, and terrain) [6]. Team risk analysis includes not only team members but also other stakeholders such as local organizations, finance entities, or others. The typology analysis is linked with the sector (energy, education, health, TICs, etc.) considering maturity, technology, regulation, etc. The territorial analysis refers to the geographic context for which the project is going to be defined. It is associated with the country’s risks, depending on political stability, the existence of armed conflicts, the lack of transport infrastructure, differences in language and culture, etc.

This analysis is intended to help, from the beginning, to improve risk management, to ensure that the proposed interventions are more effective. It could also help decide the appropriateness of not going ahead with the initiative if the level of risk is very high.

Once risks at the organizational level have been analyzed, LFRMA methodology proposes including a systematic analysis of risks in the project by a participative way in all the steps of the LFA:

First step: stakeholders’ analysis

In the first step of the LFA (stakeholders’ analysis), both direct and indirect beneficiaries are identified, as well as those excluded, neutral, or harmed by the intervention. This first step is one of the most important tasks in the project identification stage.

For all of the stakeholders, the LFRMA suggests studying their specific characteristics and the risks derived from cultural differences that may affect the proper development of the project. To do this, it is recommended to know the terrain and if participants have spent time living with the beneficiary community, as well as contacting local leaders and other people on the ground.

The analysis of the impact of the project on all participants and the interrelation between them is very important. There are always participants that must be considered, especially to identify the risks that may arise from their situation of disadvantage and establish the necessary strategies to know them and keep them informed, thus being able to intervene in the mediation of conflicts in case they arise.

It is also important to identify the agents of change, defining the actors who can play a promoter, neutral, or blocking role, as well as their level of power. The cross between their power (or lack of it) and their disposition in favor of or against change will allow identifying risks in the way of threats but also of opportunities.

Among the tools that can be used, the most recommended are described below.

• Interrelation matrix: paying special attention to how intervention changes the interrelationships between stakeholders. Any change is itself a source of risk, so it is recommended to conduct a study in two scenarios, before and after the project. Results of the analysis become a source of information on possible risks. For example, in a project to study the empowerment of rural women, the change in beneficiary women will entail changes in their interrelations within their family environment, as well as in the community, since they go from being economically dependent to having autonomy, and even suppose competition for other businesses in the area. These situations are sources of risk, generating, as has been proven, risks such as social rejection.
• Power–interest matrix: Identifying the power and interest of stakeholders may help us to identify risks link with a commitment to the project and with communication.
• Multi-stakeholder analysis: to consider the different approaches of each stakeholder and the risks that could particularly affect each one, as well as the risks due to their interaction.

Second and third steps: problem tree and objective tree analysis

The LFRMA proposes introducing risk management in Steps Two and Three of the LFA (problem tree and objective tree), focusing on identifying as many risks as possible in a participatory process. Having the opportunity to work together with the community in the search for the most appropriate problems and solutions allows, in addition, knowing the interests of the community to identify first-hand a series of risks based on the observation of behaviors and decisions of the participants, which is of great value to the risk management. Additionally, through simple questions, one could obtain an idea of the probability of occurrence of these risks and their impact, or even about the possible strategies for responding to risks.

These studies make it possible to identify the key elements of the context in which the intervention is to be carried out; therefore, paying attention to the risks associated with this context will help to identify the risks that could affect activities in the area.

The use of questions related to risks in the process of carrying out these dynamics is recommended. Examples: what if...? What change would be taking place in the community if…? Who/what would it affect...? What would it take to avoid...?

Fourth step: alternatives analysis

For the fourth step of the LFA (alternatives analysis), the LFRMA recommends including the risk criteria to help to ensure that the interventions carried out have considered RM from their conception, betting on the projects with the most assumable risks by the organization to improve their efficiency levels.

Among the tools that can be used, multi-criteria analysis is recommended, including risks as one more criterion.

It is important to have this analysis of alternatives available during the development of the project, given that, as the project progresses through its life cycle, and considering the risk management process, an initiative that was initially discarded may be worth considering again.

Fifth step: the logical framework matrix

The LFRMA proposes to add the risk column and analyze the assumptions column as a source of risks to the previous LFM.

The risks detected for each activity/result/objective will be shown in that column. Additionally, the new LFM or LFRMA matrix must be accompanied by the register of these risks, which will include, at least, their importance (as a result of their probability of occurrence due to their impact) and their response, if considered necessary because the risk is not acceptable (see Figure 3). The aim is to achieve a more adequate, useful, and realistic use of the column of assumptions, as well as to leave documented evidence of the existence of risks to ensure their management. These risks should then be included in the Risk Register.

3.1. The LFRMA Like a Participative Process

In the LFRMA, the identification, analysis, and response to risks associated with each of these stages are proposed as participatory, as the LFA itself is supposed to be. Like any participatory process, it is recommended to carry it out in two steps, first in a segregated manner and concluding with a joint search for consensus. The different stakeholders to work with would be, for example, the project team, the beneficiary community initially segregated by gender, and then, jointly, local leaders. After this first work in groups, the last joint session is recommended in which consensus is sought through multi-stakeholder dialogue. It is possible to count on all the participants who have previously worked separately or with the people designated in those processes as their representatives.

For participatory processes to be beneficial, it is recommended to use very simple risk management techniques and previously introduce the concepts with which you are going to work.

The tools that can facilitate this process are the Risk Breakdown Structure and the Probability–Impact Matrix, simplified according to the needs of the participants, for risk assessment.

It is highly recommended, in addition, to have the judgment of experts in the area and documentation related to previous projects carried out and their lessons learned.

Risk response strategies should have been defined in the Risk Management Plan as well as the level of acceptance of risks.

When implementing responses to unaccepted risks, it is recommended to use the Risk–Response Matrix, which represents the importance of risk versus the difficulty of its response and which determines which responses should be prioritized and which ones can be postponed or rethought.

Among the responses to prioritize would be those that depend on the performing organization itself and that do not require too many resources. For example, given the risk that the participants do not attend the planned training sessions because they need to attend to their current work that is their source of income, the training sessions could be set at sunset.

For those responses that require large resources or these that are less important or not imminent, it is convenient to think about them and consider other options, for which it is useful to establish multi-stakeholder dialogues to learn different points of view, in case any of them could raise a new one or a more affordable response. All those responses that are easy for the organization to attend should be allowed, no matter how low the risk is.

The cost of all risk responses should be included in the project budget.

3.2. Risk Control and Project Life Cycle

During project development, new risks may appear, and existing ones may vary in some of their components (probability or impact) or disappear, so it is important to monitor their evolution throughout the project.

Documenting risks will not only help in this process but will also allow each organization to have valuable information for new projects. Although each project is unique, relying on previous risk registers can greatly facilitate RM in new projects.

The periodicity for risk control and the person responsible for controlling them, as well as the format of the risk register used, must be defined in the Risk Management Plan of each organization.

It is very useful to analyze the risks linked with each project’s life cycle phase. To do this, it is proposed to place the risk categories that contain the most important risks on the life cycle diagram selected by the organization.

An example is proposed in Figure 4, in which the three categories with the most important risks can be seen represented at each phase. This is obtained by adding the importance scores obtained for each risk within each category and allows knowing at a glance what type of risks affect the project the most in each of its phases. It also includes a transversal layer dedicated to the project management, in which risks must also be included.

Again, the information obtained with this simple tool allows the organization to have very useful information for the improvement of the design and development of new projects.

3.3. Methodology Validation

Validation of the methodology is made with eight professionals who had previously answered the questionnaire on the LF. These experts were chosen from among the thirty who had expressed their interest in future consultations, trying to achieve the representativeness of all the actors and including both Spanish (5) and Colombian (3) professionals. All of them were previously consulted about their availability for reading and subsequent comments on the methodology, all accepting the commitment to also carry it out within the established time of one week.

The panel of experts (E) was composed of:

• (E1): head of the Intermon-Oxfam International Cooperation Department.
• (E2): professor of projects at a public Spanish university and independent evaluator of International Development Projects.
• (E3): coordinator of energy projects in Spanish NGDO ONGAWA.
• (E4): technician responsible for development cooperation of the Government of Cauca (Colombia).
• (E5): director of the Acciona.org foundation.
• (E6): university researcher at the Center for Innovation and Technologies for Human Development.
• (E7): coordinator of the Territorial Office of Nariño, Cauca, and Putumayo of the United Nations Development Program (UNDP).
• (E8): coordinator of the Bank of Projects of the Regional Autonomous Corporation of Santander (Colombia).

All participants were asked separately about the four open-ended questions that are summarized below.

Question 1: Do you think this methodology can help to improve the effectiveness of Development Cooperation Projects and therefore the impact on the beneficiary community? Why?

All the people asked agreed that this methodology will help to improve the levels of project success, recognizing that many of the current projects do not take into account the risks, but, above all, that there is no effective learning from a project to another.

With the risk register, it is possible to give, firstly, a “continuity to the evolution of risks, beyond the initial identification and, above all, an identification of risks of great value for future projects” [E1 and E2]. It highlights that “it has great potential in being able to be systematized through computer programs that allow risks to be monitored through the risk register presented” [E8].

In addition, “it would allow better choice and definition of interventions during the identification and design phase” [E3], and “it would help to better deal with risks during the implementation phase by having a structured contingency plan” [E3 and E4], which “would mean a reduction in costs and delays in interventions” [E3] and “to improve project results, directly impacting the beneficiary community” [E5]. It also provides the “novelty of counting on the community to identify risks, which is very valuable given that the people of the territories are the ones who have a better vision and knowledge of the territory, and this will allow us to better identify some risks that are not usually perceived by the teams who design the project” [E4].

Just because “the fact of introducing a risk management improves project planning per se, by allowing a deeper knowledge of the interventions” [E6], “risk management should be essential in any cooperation project, as it is in other sectors, and the best way to internalize it is to do so through a very common tool in this sector such as LFA” [E5].

“In Colombia, particularly, not paying due attention to risks has caused large-scale IDP to be forgotten without causing any favorable impact” [E7]. This methodology “can also be complemented in the case of Colombia with Corruption Risk Maps”, recently published at the national level [E8].

Question 2: What do you like the most about the methodology? Why?

All the people surveyed highlighted the simplicity of the methodology and the ease of integration with other approaches, especially valuing the incorporation of analyzing risks internally in the organization and, specifically, the TTT model presented [6].

The “integration in the LFA’s steps” [E5] and its “compatibility with other approaches and with the theories of change” [E1] and the “participatory nature of the methodology, including the beneficiary community” [E1 and E4] were valued.

Among the tools presented, the TTT [6] analysis stands out, “especially because it includes multi-actor aspects” [E6], but “the risk association diagram for each phase of the project life cycle” [E1 and E3] is also highly valued. Also concerning the previous question, “having a risk register that also makes it possible to consolidate lessons learned for future projects is of great value” [E4 and E7].

The fact that it is a “markedly visual” methodology [E3] consolidates its “ease of understanding” [E8] “through simple steps that connect very well with the logic of the projects” [E7], highlighting “the pragmatic way of integrating risk management into the LFA matrix” [E5].

However, above all, the fact was valued that, with very little additional effort and without requiring many more resources, the LFA planning tool [E2, E5, and E7] will be greatly improved.

Question 3: What do you like least about the methodology? Why?

The main concern expressed by four of the participants was related to the effort required to carry out such an exhaustive risk analysis in the initial stages of the project where it is not even clear that the intervention will be carried out. In response to this concern and the valuable comments provided by some of the participants, some steps recommended in the initial proposal were eliminated, and their use was simplified in the steps of the logical framework, reducing the incidence of the integration of risk management in some steps of the LFA, as in the problem and objective tree. This managed to provide the requested “balance” “between the role of the risk management and the project management itself” [E2].

Another weakness of the initial proposal was the “lack of examples” [E1, E7, and E8], which was improved with the inclusion of some simple examples.

Apart from that, one of the comments given was very interesting regarding the need to “emphasize in the beneficiary communities that the search for risks is to make the intervention more effective, but that the project is going to be carried out, since, if this is not clear, people can be placed in a defensive position and not collaborate in the identification of risks for fear that their discovery will make the project not carried out” [E4].

Another negative point has to do with the difficulty of solving, through a single improvement, the “stiffness limitations of the LFA” [E6]. Although the LFRMA methodology proposes to attend to the evolution of risks and respond to them promptly, indeed, it is only the most accepted improvement, but the LFA could admit more.

Question 4: Would you apply it, or would you recommend its application? Why?

All of the people asked would apply and recommend the application of this methodology, considering that “the third case of failure of a project is the poor consideration of risks”, the other two possible being poor formulation or deficient implementation [E2].

Most highlight the need to apply it in the initial stages of the project (identification and design), given that “any change not considered in the initial planning generally means waste of time and resources” [E4], since “the frameworks, the rigid rules of cooperation, as well as the capacity of many NGDOs, make it more difficult to manage risks during project execution” [E3].

Some experts also make a distinction for its partial application in interventions of short duration or with very little scope [E3, E5, and E6].

The proposal to “demonstrate the virtues of the application by putting real cases of what happens when you don’t apply it” [E1] is also very suggestive. Although the methodology is new and, therefore, there are no results of its application to allow comparisons, there are many and known IDPs that have failed due to not having good risk management; these examples can be used when disseminating the use of the methodology.

After consulting these professionals, more emphasis has been placed on the differences between applications of the methodology, on the one hand, for the organization and, on the other hand, for the project management itself, given that the part dedicated to the organization is one of the most appreciated. With this, its understanding is facilitated and the need to integrate the risk management culture in the world of development cooperation is enhanced. Likewise, more examples have been added for a good understanding of the proposed tools.

3.4. Discussion and Conclusions

The LFRMA methodology is an original and innovative contribution, tested with professionals, that will improve the effectiveness of IDPs by increasing their success rate and their sustainability. Additionally, since the LFA is a framework that represents a common language in the international development sector, authors believe that it would not be appropriate to abandon this methodology but rather to improve it. The introduction of Risk Management in a simple and user-friendly way in all steps of the LFA is the best approach to finally include Risk Management in IDPs, so demanded by international organizations and other authors [25,31].

Another innovation is that the LFRMA’s scope of application is not only limited to project management but also to the organization itself, since, without the latter’s commitment to risk management, any change introduced would have less impact. This duality gives the methodology a valuable holistic character, as recognized by the professionals with whom it has been tested.

Aware that the problems of effectiveness and sustainability of IDP are difficult and wicked problems, whose complexity cannot be solved by a single solution, risk management contributes to their improvement in the sense of structuring the daily knowledge of the existence of risks and allowing decision making following them. In the search for effectiveness and sustainability of the LFA improvement, the co-creation of the proposed solution has also been pursued.

During the first phase of the process (problem statement), some of the RQs and RHs were answered satisfactorily.

The initial literature review helped to deduce that the effectiveness and sustainability of IDPs could be improved, thus responding to RQ1.

The following questionnaire pointed out that the LFA needs to be improved (RH1) and risk management was the best ally to improve IDPs’ effectiveness and sustainability (RQ2 and RQ3) [15].

During the second phase of the process (case studies), the projects analysed in the field, and the work with their teams made it possible to establish a univocal relationship between good risk management in projects and project success, understood as the achievement of objectives and sustainability of actions over time [39]. This relationship thus responds to RH2.

Finally, during the third phase of the process (design and contrast of the LFRMA), the full support of sector professionals for the methodology confirms the fact that it can improve the effectiveness and sustainability of IDPs. This support responds to RH3.

The use of LFRMA is recommended primarily to organizations that previously worked with the LFA. Since these organizations are mainly involved in the management of IDPs and their area of expertise is very broad (from hospital construction to power generation plants), it could be said that it covers all sectors.

Concerning the methodology designed, and all the research on which it is based, of course, it has limitations. As always when working with samples, if the number of participants was increased, the results obtained on the opinion of LFA practitioners could be different, just as there could be differences in the results obtained on the direct relationship between the use of risk management and the success of projects, based on the projects studied in the field. In any case, the size of the sample of LFA practitioners, the projects selected for study, and the panel of experts with whom the LFRMA was consulted do seem to the authors to be representative, given that they were people with a wide experience in the sector and, in many cases, leading recognized and stable organizations dedicated to development cooperation.

Regarding future works and thinking in the applications of the LFRMA to the greatest possible number of IDPs, the authors propose the design of a computer tool accompanied by an application for mobile devices, in which each organization can use the tools presented and keep a record of the IDPs executed. The more organizations that use it, the greater the risk register available for each area and type of project, thus establishing a collaborative risk register that could be shared among the different organizations.