The RI grant life cycle comprises four major phases that are described below and summarised in Fig. 3.1. The first phase is the pre-grant award phase, which involves (i) soliciting research applications or proposals from the research community; (ii) assessing and reviewing applications; (iii) making funding decisions; and (iv) preparing grant award letters. The second phase includes the grant award phase, which entails approving the (i) funding decisions; (ii) communicating outcomes to all applicants; and (iii) preparing the legal document for the funding agency to contract with the grant holder by specifying the terms and conditions relating to the awarding of the grant which is presented in the document termed Conditions of Grant Award. The third phase is the post-grant award phase which is triggered by the staff within the funding agency issuing the Condition of Grant Award to the grant holder. Thereafter numerous monitoring and evaluating activities commence that include (i) financial expenditures; (ii) adherence to the management plan that was submitted by the grant holder as part of the original application; and (iii) reporting on the key performance indicators (KPIs). The fourth and final phase is the project close-out phase, which requires the grant holder to submit a final report on the financials, programme, grants-related activities, successes and challenges. At this stage representatives from the funding agency may need to conduct a site visit or technical audit to ensure that the grant holder has complied with all the conditions related to the grant award (Kwak & Keleher, 2015).

Fig. 3.1
figure 1

Grant lifecycle (adapted from: Kwak and Keleher, 2015)

Full size image

3.1 Pre-grant Award Phase

Competitive processes are employed to solicit applications or proposals for RI grants. When a call for applications is announced, a deadline is also specified by the funding agency for eligible applicants from eligible research institutions to prepare their applications against the pre-defined requirements. Completed applications with the necessary endorsements in the form of signatures of both the applicant and the research institution’s research management representative or designated authority are submitted within the specified timeframe to the specified funding agency contact person.

A pre-screening process follows, where the funding agency (i) logs the applications received and provide a summary of each application; and (ii) conducts an assessment to ensure that all applications meet the minimum eligibility criteria for both the applicant and the research institution. If the eligibility criteria is not met, the application does not proceed to the next phase of evaluation. In instances where there may be minor administrative gaps in the application, the funding agency may provide applicants with the opportunity to revise their applications within a stipulated time frame so that the eligibility criteria are met. In such instances, once all eligibility criteria have been revised and met, the application proceeds to the next phase of evaluation (Table 3.1).

Table 3.1 Example of a pre-screening spreadsheet
Full size table

3.2 Peer Review

A common approach for conducting peer review processes is by either (i) panel review; (ii) mail review; or (iii) both. Both review processes are based on insights and recommendation of well-informed experts on various quality dimensions of research, as guided by a scorecard (Ruegg & Feller, 2003). The following section provides a detailed discussion and comparison of the panel and mail review processes.

3.2.1 Panel Review

In a panel meeting, reviewers are co-opted by the funding agency and a formal meeting is convened. There is usually an appointed chairperson who ensures that all applications, as logged and pre-screened by the funding agency, are reviewed with clear recommendations provided by the panel. The role of the chairperson is to facilitate the discussion on an application and guide the panel towards a consensus decision to either “fund” or “not fund” a specific application. The chairperson will also ensure that an appropriate length of time is allowed for the evaluation of each proposal. In addition to having an appointed chairperson, there is also an appointed assessor who ensures that personal biases from any appointed reviewer is minimised. The assessor’s role is also to ensure that the processes adopted during the meeting are fair and transparent and that the same criteria are applied consistently by all the panel members for the evaluation and scoring of all applications. In essence, the role of the assessor is to ensure procedural consistencies are applied when evaluating proposals. At the end of the panel meeting, both the chairperson and the assessor will submit a jointly written report which will be used by the funding agency to either improve or retain specific review processes. Supporting the chairperson and assessor in a panel meeting is a rapporteur whose role is to capture the proceedings of the meeting on a verbatim basis. This is an important process as it ensures transparency as well as provides a reference point for contestations that may arise from time to time, especially if researchers were unsuccessful in their application to obtain funding and require detailed feedback.

The role of reviewers is to make recommendations to the funding agency on whether each application, when considered in their entirety, should be funded or not. The panel is required to use the prescribed scorecard from the funding agency as a guide for evaluating the applications. The panel reviewers are required to submit a completed reviewer evaluation form at the end of the meeting that can also be used by funding agency staff to provide feed back to the applicant. This report must outline the successes, challenges and areas for improvement in the submitted application. During the panel review, usually two reviewers present a research proposal to the rest of the participants of the peer review group (Braun, 1998). This opens the floor to dialogue and opposing views by the other panel participants. There is a tendency in this review method for those reviewers evaluating a proposal to have the prerogative in the decision on whether or not a project is successful (Lee & Harley, 1998). Although a peer review can gain consensus on proposals that are either outstanding or poor, it is difficult to reach a consensus on proposals that score in the middle range which is a major limitation associated with the peer review system (Kostoff, 1994). At this stage, the role of both the assessor and chairperson becomes of paramount importance, especially in terms of ensuring that the key purpose of a peer review is to support outstanding proposals and reject those proposals that are deemed poor.

The drawbacks associated with the panel review method are cost implications and an inherently subjective decision making process that depends on the interests, experience and knowledge of the evaluators (Lee & Harley, 1998). Furthermore, the quality of the review can never go beyond the competence of the reviewers (Kostoff, 1994). It is, therefore, essential that the reviewer profile of the panel includes a combination from different countries and research backgrounds that span the spectrum of disciplines shortlisted in the pre-screening process, e.g. physical sciences and biological sciences. The use of international reviewers that host and manage mega-RIs should be identified as potential reviewers. These reviewers not only provide an independent and objective expert perspective but also guide the funding agency on best practices, risks, opportunities and challenges relating to the investment in RIs. A drawback to the use of international reviewers is their lack of understanding of local or national imperatives and context.

3.2.2 Mail Review

Funding agencies also employ a mail or postal review system where referees or reviewers decide on the credibility of the proposal and the research applicant in accordance with the guidelines and a scorecard prescribed by the funding agency. In the mail review system, the referee or reviewer makes an independent decision without being exposed to the opinion(s) of other reviewers (Lee & Harley, 1998). Usually two or three mail reviewers are requested on the same project proposal in order to balance the views of proposals. One of two processes can unfold post the submission of mail review reports.

Firstly, the reports can be anonymised and subsequently fed as source documents into the panel review meeting. These mail review reports provide an alternate perspective on the proposals to be evaluated at a panel meeting. If this process is undertaken, the panel reviewers have the final decision relating to whether or not a project is successful. Secondly, the reports are used by the funding agency staff to make the final decision on the outcomes of the application (Braun, 1998).

The general experience in the South African context is that the poor quality of the postal review reports do not provide adequate information for a decision to be made by either the funding agency or panel reviewers on whether or not an application should be funded. Hence the consensus is that the panel review be exclusively employed which aids in reducing (i) the complexity related to awarding RI grants; and (ii) the conflict(s) of interest that may emerge due to the small pool of reviewers available in the country.

3.3 Developing a Suitable Scorecard

All reviewers, both panel and postal, evaluate the merit of RI grant applications against the various funding agency-defined evaluation dimensions as presented in a scorecard. The awarding of research equipment grants should be based on a robust scorecard that, in turn, is informed by the national research strategies, scientific excellence and potential research impact. For example, reviewers for the United States National Science Foundation (NSF) use a four-criterion process to assess proposals, viz. (i) researcher performance competence; (ii) intrinsic merit of the proposed research; (iii) utility or relevance of the research; and (iv) the effect of research on the infrastructure of science and engineering (Kostoff, 1994). In the case of the Public Health Services projects, the criteria for the reviewers include (i) significance and originality of the proposal from a scientific and technical point of view; (ii) adequacy of the methodology to carry out research; (iii) qualification and experience of the principle investigator and staff; (iv) availability of resources; (v) justification for the proposed budget; (vi) duration of the projects; and (vii) other discipline-specific regulatory approvals such as ethics approvals when the project involves human or animal subjects and biohazards (Kostoff, 1994). Similar scorecards are utilised by other funding agencies across the globe.

  • In essence, the scientific case must drive and underpin the justification for any research equipment.

For example, the RI scorecard used by the NRF as a guide for reviewers could include the following essential criteria:

  • Feasibility of the proposed management plan (see Management Plan section);

  • Scientific merit of the proposed research to be undertaken if the equipment is procured;

  • Researcher’s track record in terms of (i) scientific publications using similar equipment; and (ii) human capital development (HCD) including training post-graduate students, postdoctoral fellows and young and/or emerging researchers; and

  • Proposed research collaborations which will be the indicator of how access to the equipment will be promoted to other researchers. This proposed plan for research collaborations needs to be calibrated by the track record of the applying researcher in terms of historic collaborations that they have undertaken, nurtured and sustained (National Research Foundation, 2018b) (Table 3.2).

    Table 3.2 Example of a RI scorecard and the associated evaluation dimensions (National Research Foundation, 2018b)
    Full size table

3.4 Grant Award Phase

This phase of the grant life cycle involves (i) finalising and approving the funding decisions; (ii) communicating the outcomes of the review process to all applicants; and (iii) receiving the signed conditions of grant award from successful applicants that are thereafter referred to as grant holders (Kwak & Keleher, 2015).

3.4.1 Funding Decisions

Post the evaluation process, funding decisions need to be approved by senior management within the funding agency which summarises the list of applications or proposals that were submitted post the closing of a call. It also specifies all those applications that were:

  • Submitted during a call and were either:

    • Rejected at the pre-screening stage due to their not meeting the eligibility criteria; or

    • Approved for further review at the pre-screening stage

  • Submitted for peer review and were either:

    • Recommended for funding; or

    • Not recommended for funding

In some instances, an additional category may be included in the funding decision spreadsheet, e.g. in instances where budgetary constraints prevent funding agencies from fully supporting the list of applications that are recommended for funding by the peer review committee. This, therefore, warrants the inclusion of a separate category, usually entitled: “Recommended for funding, but not awarded due to budgetary constraints. These applications must be awarded if additional funds are made available by the funding agency”. This category then becomes a priority list for approval of funding should additional funds become available in support of RI grants (National Research Foundation, 2018b). A summary of the processes involved in grants management is presented in Fig. 3.2.

Fig. 3.2
figure 2

Summary of the review processes utilised to evaluate applications for RI funding (National Research Foundation, 2018b)

Full size image

Once the funding decision spreadsheet has been approved, the funding agency communicates review outcomes to all applicants. A grant award is sent to successful applicants who have to comply with the requirements set forth in the Conditions of Grant Award which is a governance and risk management tool (National Research Foundation, 2018b). Communication is also sent to applicants that were not successful in soliciting grant funds, with detailed feedback on the gaps and the areas in the application that require strengthening.

3.5 Post Grant Award Phase

This phase refers to the monitoring and evaluation activities employed by the funding agency in an oversight capacity to assess financial expenditures, adherence to the work plan and reporting on key performance indicators (KPIs) as prescribed in the Conditions of Grant Award (Kwak & Keleher, 2015). The funding agency plays a proactive role in tracking performance and identifying red flags against the following indicators:

  • Programme-related indicators which include performance against management plan deliverables in line with the KPIs set forth by the funding agency. This includes, but is not limited to, drop-out rates of students, timelines for achieving pre-defined activities, amongst others (Kwak & Keleher, 2015).

  • Management-related indicators which relate to any special conditions against which grants were award. This includes the development of an institutional plan for risk management which includes, but is not limited to, change of grant holder, loss of technical staff (either through retirement, resignation or death), challenges with supplier support, and other support capabilities including building infrastructure, required for the functionality of the research equipment (Kwak & Keleher, 2015).

  • Financial indicators which refers to the drawing down of the grant in a timely manner as defined in the management plan (Kwak & Keleher, 2015). These will be described in detail under Monitoring and Evaluation.

In the event of red flags materialising, the funding agency must comply with a consequence management framework that puts in place measures such as (i) a recall of the grant investment from the funding agency; and (ii) prohibiting the research institution for a minimum period of three years from applying for additional research equipment grants or until such time that the institution fully addresses the red flag(s).

3.6 Project Close Out Phase

This is the final phase of the grant life cycle which requires the funding agency to (i) undertake a site visit to the research facility of the grant holder; and (ii) receive a project close-out report that summarises the financials, programme and grants-related activities, successes and challenges related to the RI grant (Kwak & Keleher, 2015).

3.7 Summary

Given the complexity, a limited number of reviewers, and a lack of experience and/or expertise of the reviewers or researchers on the use and management of equipment that are available within a country, the panel review process is recommended in the review of RI grants. The continued use of a panel review is further motivated by the fact that the international reviewers are able to (i) train national reviewers on how the peer review process is managed within their respective countries; and (ii) gain exposure to the researchers in the developing country, which can aid in the establishment of collaborations, mentorship programmes and staff and/or student exchange or sabbatical programmes at a later stage. Compared to panel reviews, selected cases in South Africa have shown that the quality of reports submitted by remote reviewers are below par. Caution and additional measures should be taken into account when considering this approach.

Finally, in order to improve and increase the number of exceptional reviewers, it is recommended that the funding agencies facilitate training courses on: (i) the objectives of RI funding instruments, (ii) the national contextual perspective and (iii) imperatives for new reviewers; and build strategic partnerships with experts and institutions across the continent and abroad.