Analytical Framework for Performance Evaluation of Research Organizations

Performance evaluation is an important measure of the total quality management, which can be used to assess the performance of an individual or an organization with respect to set goals and targets. The metrics/parameters used for evaluating the performance and the way in which these are measured by using appropriate tools and techniques play a major role in the evaluation process. Performance evaluation is even more challenging in the case of R and D organizations, where the outcome/output may not be tangible/measurable and varies from one organization to another, depending on the nature, vision, charter and character. A methodology is proposed to arrive at a framework that can help in objectively assessing or evaluating the performance of each of the laboratories of CSIR (Council of Scientific and Industrial Research) based on four knowledge portfolios which are appropriately given weightages according to the impact they have on the four goods viz-Public, Private, Social and Strategic Appropriate parameters have been identified which can help in objectively evaluating the performance of the laboratory. The proposed analytical framework will facilitate quantification of performance of an R and D organization to enable resource allocation in a rational manner.


INTRODUCTION
R and D organizations, whether public or private, are and must be, interested in developing and deploying effective performance measurement and management systems, since it is only through such systems that they can remain high-performance organizations.In a rapidly changing and competitive globalized world, federally or public funded R and D institutions need to monitor, develop and improve their performance continuously to ensure competitive advantage.Organizations need to determine their current status of performance constantly and identify organizational strengths and areas where improvements can be made.One must therefore have appropriate mechanisms to evaluate the performance of the organization by using well-accepted methodologies and tools.Periodic organizational self-assessment by identifying performance parameters and using rational methodologies to compute these parameters are required.This will allow to measure the current management and operational policies, practices and procedures, in order to enhance overall R and D performance and provide focus.
The performance of any organization can be assessed through its stakeholders.The stakeholders being the scientists and other technical supporting staff, its clientele and any other indirect beneficiaries.Thus stakeholder satisfaction/participation may be one of the many parameters that form part of the performance evaluation.
R and D organizations funded either by the government or any industry have a responsibility that they deploy their resources in a very optimal way so that there is a proper pay-off or appropriate Return on Investment (RoI).The parameters of performance can serve as a yard stick to ensure that either the resources are utilized appropriately or are leveraged suitably.This is easily said than done because the quantification or measurement of the parameters is a challenge in itself.In the case of mission oriented organizations where the outputs or outcomes are well defined, the evaluation of the performance or measurement of the parameters may be straightforward.This may not be so, in the case of R and D organizations where the output or outcome is not directly measurable or is not tangible in view of the wide bandwidth of the R and D areas in science and engineering.The degree of complexity varies depending on the type of R and D, its funding, its mandate and so on.The purpose of this study is to: • Identify parameters of performance • Provide methodologies to quantify/measure the parameters to make the evaluation more objective • Arrive at an analytical framework that can help in effectively measuring the parameters of performance through a suitable methodology Number of studies and methodologies have been proposed to evaluate the performance of R and D organizations.In the foregoing section a detailed stateof-the art review is presented on some of the strategies and methodologies proposed for performance evaluation of R and D organizations.In the section subsequent to state-of-the art review is the proposed methodology and the actual measurement of performance.This proposed methodology is further evaluated though a case study of the Council of Scientific and Industrial Research (CSIR) including discussions on how this can be flexibly used to measure different parameters of performance.

LITERATURE REVIEW
The challenges and problems concerning the evaluation and measurement of R and D output/outcome have been observed and studied by many researchers in a large number of earlier studies.Although researchers have come out with different measurement parameters of performance, only those that are closely relevant to the R and D work of especially publicly funded R and D organizations are cited here.Gold (1989) defined six major types of contributions which R and D programs could provide and then discusses their bearing on evaluating: • R and D performance alone • R and D contributions to the overall performance of the firm • The R and D performance of a given firm in comparison with that of its competitors The author recommends in conclusion the role of management in developing more effective bases for evaluating the quality and effectiveness of the performance of the R and D organization.Brown and Gobeli (1992) presented the top ten R and D productivity indicators on the basis of classification to measurements of resources, project planning and management, outputs, outcomes, etc.The authors concluded that further tests still are necessary to come up with reliable parameters admitting that measurement of performance parameters is complex.Werner and Souder (1997) presented an example of an integrated metric that combines several objective and subjective metric including the effectiveness index, timeless index, future potential index, etc. Baglieri et al. (2001) suggested the measurement of R and D performance using intangible assets based on value created through R and D. However, the framework suggested is based on the assumption that there is a strong relationship between R and D contribution to stakeholder value and the operational performance of R and D activities.Ojanen and Tuominen (2002) presented a simplified approach of selecting and developing performance evaluation methods for measuring the overall effectiveness of R and D with respect to the Telecom Sector.They attempted to define factors and factors that influence the overall effectiveness of R and D through an analytical approach.However, the attempt is restricted to part of larger framework and the authors admit that this is a starting point.Jang and Vonortas (2002) reviewed the general evaluation practices in the United States and in the European Union and extensively examined the evaluation practices of the Office of Science, US Department of Energy and the Framework Programme of the EU.A large number of evaluation indicators adopted in these evaluation routines have been identified.Moreover, these indicators have been juxtaposed to those currently adopted for the evaluation of over 200 national R and D programs in Korea.Large-scale survey was conducted by Bowon and Heungshik (2002) covering over 1200 R and D scientists and engineers in Korean R and D organizations.Authors observed that a fair performance evaluation should utilize more behavioral and qualitative measures such as leadership and mentoring for younger researchers and bottom-up (e.g., R and D researchers' evaluation of their own bosses, say, R and D managers) as well as horizontal (e.g., peers and or colleagues) evaluation schemes.Authors found that there are two critical dimensions constituting an R and D performance evaluation system, evaluators and criteria.Ojanen and Vuola (2003) attempted to identify the necessary steps in the early phase of the selection process of R and D performance indicators.This phase prior to the actual selection of measures includes recognition and careful consideration of the measurement needs with the help of the five main dimensions of R and D performance analysis the perspectives of performance analysis, the purpose of R and D performance analysis, the type of R and D, the level of the analysis and the phase of the innovation process.Ojanen and Vuola (2006) presented a methodology for selecting R and D performance measures.They examined design/methodology/ approach with five different perspectives for determining case-specific R and D performance metrics.The metrics were derived from: • Literature survey • Discussions with firms and participationobservation i.e., analysis perspective • Purpose and level • R and D type • Innovation phase to individual elements The studies however, lacked a holistic approach.Deen and Vossensteyn (2006) addressed the issue of applied research performance measurement with specific attention paid to general quality measurement of research in the Netherlands, developments in the Dutch HBO sector and recent experiences in a number of other countries.The main conclusion from this report is that measuring applied R and D performance is an emerging field of study.Authors concluded that overlooking all developments one can see a lack of consensus about what should or should not be used as indicators of applied research performance.The problems may be associated with difficulties of measurement, imbalances between different disciplines and difficult definitions of concepts like the impact of commercialization, innovation and strategic development.Nevertheless, the general tendency is that research output is no longer strictly limited to publications and qualitative academic review.They further conclude that the measurement of research gradually starts to integrate more indicators that measure applied research efforts as well, either within the university evaluation mechanisms, national statistics or in "non-university" institutions.Though this area is still in its infancy, the report showed that advancement is being made towards a more balanced treatment of academic and applied research.Baek (2006) investigated the publishing productivity of US Academic Scientists and Engineers using Data Envelopment Analysis (DEA).The paper demonstrates DEA to be a practical productivity measurement tool and how it selects most productive researchers through various empirical examinations.The methodology holds promise and can be adopted for various other types of applications and measures.Cebeci and Sezerel (2008) attempted to integrate the Balanced Scorecard (BSC) and Analytical Hierarchy Process (AHP) in order to develop an analytical approach to evaluate the performance of R and D departments.The model uses weights and ranks main indicators affecting the R and D performance and is applied to an organization with an R and D experience of more than 15 years.However, identification of parameters have been broad-based and there is scope to improve upon these parameters with more objectives for quantification.Nishimura and Okamuro (2010) examined the effects of the "Industrial Cluster Project" (ICP) in Japan on the R and D productivity of participants, using a unique dataset of 229 small firms and discuss the conditions necessary for the effective organization of cluster policies.However, there is no effort made to holistically measure the performance of the organization.Chiesa et al. (2008) conducted an exploratory study on R and D performance measurement practices based on an empirical survey of Italian R and D intensive firms through a reference framework defined through empirical analysis.The framework was arrived at by focusing on particular R and D intensive organizations considering the investment and strongly committed R and D activities.Martin et al. (2009) discussed proper and improper forms of performance evaluation including the demerits of h-index.The study showed that realistic societal value on a scientist's performance cannot be placed by using an ostensibly objective algorithm and an immediate form of evaluation called 'rule based peer review' was proposed to evaluate larger number of scientists.This approach tends to become subjective and hence this results in empirical evaluation or is based on intuition.This is one study that is for the first time mentions the need to develop methodology to measure social values or intangible assets.Ghameri et al. (2012) investigated on the reporting of intangible assets.The authors considered assets such as R and D, design, organizational capital, human capital and brand equity.The authors further observe that these parameters can be significant bases and roots of success of an organization.
As can be seen from the above, a number of methods of evaluation have been proposed to evaluate the performance of R and D organization in the past by several researchers.The measures to determine the performance provide directions and guidance to focus its future research and determine the rate of success or achievement of the organization.Despite so much reported work on this topic, there is clear scope to improve upon the methodologies reported.Invariably the measures that have been selected are those that have a visible outcome or output and the methods employed or proposed by researchers have been based on either: However, the above methods have not considered outcomes/outputs which are intangible and cannot be directly measured.
An attempt has therefore been made in the present study to determine the parameters of performance in a more holistic way and arrive at a methodology through which more deterministic values can be assigned to the parameters enabling a rational method of evaluation.

PROPOSED METHODOLOGY
A typical R and D activity has three main dimensions: • The R and D input, which is the front-end for innovation and this input is the resources available with each of the laboratories in terms of its expert manpower, facilities, sophisticated equipment etc., • The process that handle different resources and finally • The output/outcome based on which the evaluation takes place Figure 1 depicts the above three dimensions of an R and D activity.This evaluation can be objective only if all the influencing parameters have been identified exhaustively and these parameters are effectively evaluated based on the outcomes, whether these are long term or short-term and are tangible or intangible.One of the most challenging aspects is the selection of a suitable set of appropriate measures for the right subjects of measurement.Further, there is also the problem of determining the right norms to make comparisons.
In this study, it is attempted to identify the critical parameters (both tangible and intangible) depicting/showcasing the performance of the laboratory, analyze these performance parameters and map them into four knowledge portfolios (Vijayalakshmi and Iyer Nagesh, 2011).Research produces new knowledge, products, or processes.Research publications reflect contributions to knowledge, patents indicate useful inventions and citations on patents to the scientific and technical literature indicate the linkage between research and practical application.R and D encompasses a range of activities: from fundamental research in the physical, life and social sciences; to research addressing such critical societal issues as global climate change, energy efficiency and health care; to the development of platform or general-purpose technologies and new goods and services.These four knowledge portfolios are essential since any R and D will always lead to at least one of the forms of knowledge that is: As in any R and D organization, the performance of the scientists will lead to the performance of the organization.Therefore, the contribution of the scientists in terms of publications, patents, products, technologies etc., is captured.These are then mapped to the respective knowledge portfolios as described below.Strategic: Strategic goods are those that are visible in the activities directly related to achieving self-reliance and services that meet the national/indigenous needs including national security for which no solution is available and enables creating technological options and 'resource centers', 'spin-offs', etc.Each of the knowledge portfolios described above are given suitable weightage based on the character of the laboratory and their direct or indirect influence on the four goods viz.public, private, societal or strategic.
In accordance to this mapping, suitable algorithms are formulated to assign weightages to the performance parameters.Values are arrived at based on the survey carried out and information compiled over a ten year period from each of the Laboratory to these parameters in order to bring in clarity in defining the concepts.Thus, each knowledge portfolio is assigned a knowledge coefficient called κ. Table 1 presents a summary of parameters identified by each individual laboratory that they feel are critical to their performance.It is to be noted that not all the parameters need to be critical to all the R and D laboratories.The four knowledge portfolios are always related to at least one of the four goods described as per the Table 1.
The above description of the parameters, though are not exhaustive, consider most of the probable occurrences for each parameter.The criticality of the parameters and significance of each can vary from one R and D laboratory to another R and D laboratory of depending on the nature and character and also its mandate/charter.Once the parameters are identified, each of these are mapped to any one or multiple combinations of the four knowledge portfolios depending on the attributes associated with the parameters.The above framework is relevant as this provides the basis for not only identifying the performance parameters but also in developing/ formulating methodology to quantify the performance parameters.The actual performance evaluation or performance as envisaged through this framework is detailed below.
Figure 2 depicts the process of performance assessment beginning from the research inputs, parameter identification and grouping into knowledge portfolios and then subjective evaluation through quantification and arriving at the laboratory score.
As is evident from Fig. 2, the four knowledge portfolios play a major role in the computation of lab score.Further, steps developed/formulated in the present study to compute the overall laboratory performance of a publicly funded R and D institution can best be described through a case study as we use realistic values and information to determine the same.
Case study: A typical scenario with regard to Council of Scientific and Industrial Research (CSIR), India, one of the premier Research Organizations in the country with 37 constituent R and D laboratories across the country with strength of about 4600 scientists has been chosen in this study as it truly represents the character of public funded R and D institutions.CSIR and its constituent laboratories are involved in world class research with corporate social responsibility.The research areas include Biological Sciences, Physical Sciences, Chemical Sciences, Engineering Sciences and Mathematical and Information Sciences.Each laboratory is grouped among other laboratories in such manner that they form a group cluster that represents one of the science discipline described.Many industries, institutes of national/international/multinational importance and premier private organizations approach CSIR for their problem solving needs in the form of Contract Research, Consultancy and specialized testing services.The technologies developed at CSIR laboratories are being used by various industries in the country.More details about the organogram, vision, goals, roadmap, major R and D initiatives, etc., can be seen by visiting the CSIR website -http://csir.res.in.
CSIR is continuously self-introspecting and ever evolving organization with a cherished desire to make large national and international impact through its contributions to society, economy and environment.It is one of the most transparent organizations through its approach with best practices adopted for performance appraisal.With a view to improve its effectiveness and efficacy, the organization has periodically subjected itself to peer reviews.This organization is ever committed to enhanced contribution to the nation.It has been in the front-end of research and is constantly engaged in innovative and cutting edge research.
The mandate of each of the 37 CSIR laboratories is different.A list of 37 CSIR laboratories is presented in Table 2.The contribution to public, private, strategic  and societal goods from each laboratory are in varying proportions and thus, there is no uniformity to come out with a common generalized approach.It is therefore difficult to quantify the output/outcome of research and compare one laboratory with the other on the same platform.The Change Team (CSIR Report, 2005) was appointed by CSIR to arrive at an implementation strategy, which will help in the evaluation of the laboratory score.The computation of the scores, either empirical or semi-empirical is presented in the report submitted by the Change Team.An attempt was made to derive a global average of stakeholder focus for the entire CSIR as an organization.The distribution is given in Table 3. Normally, the performance of laboratory is assessed based on select parameters such as publications, patents, strategic/societal contributions through External Cash Flow (ECF) etc.The parameters depend on the type of research engaged in viz.Basic Research, Applied Research etc.While it is possible to quantify contributions such as journal publications, ECF generated, patents filed, technologies transferred and also the impact on public or private goods, it is not so with parameters which are towards societal or strategic goods.
For example a laboratory may be involved in developing technologies which contribute to the society at large such as ferro-cement water tanks, manhole covers etc., which are neither patented nor quoted in any literature.These are the outputs/outcomes of R and D that get classified under intangible benefits.Similarly in the case of strategic sector, the contribution from many organizations cannot be directly acknowledged, whereas these organizations would have played a major role through their research contributions.
The one unifying factor for all these laboratories is that they serve as knowledge resources/repositories, which generate knowledge, transfer knowledge or engage themselves in knowledge management and are recognized for the knowledge they possess and create.
Also, the character of any laboratory can be determined by its charter and by its proportional commitment to various goods such as, public goods, private goods, societal goods and strategic goods.For example, a laboratory conducting high-end research will have sufficient contributions and achievements in public goods that are characterized by high impact factor journal publications and highly cited papers than other goods while another laboratory can thrive on its problem solving capabilities and its involvement in many consultancy projects, thereby generating external cash flow.
For instance, many scientists of CSIR laboratories especially the engineering laboratories serve on various committees of the Bureau of Indian Standards (BIS) and other agencies such as Indian Roads Congress, Directorate General of Civil Aviation, Food Safety and Standards Authority of India that define the guidelines, rules/regulations, practices.These are the highest bodies in the country that are responsible for development of rules/guidelines/code of practices in various disciplines for design, construction/fabrication, manufacturing, etc., which are mandatory for compliance.The scientists of CSIR are involved in the revision of existing codes or formulation of new codes of practices based on the R and D contributions of the laboratory and experience.These codes are being used in the design of new buildings, bridges or any other infrastructure or product manufacturing, packaging etc.In a country such as India, where infrastructure plays a major role, the contribution of such scientists is enormous.However, it cannot be directly factored or measured while evaluating the performance of that particular engineering laboratory in general and the scientists of the laboratory in particular.In the same vein, we encounter the indirect contributions of scientists involved in natural sciences, physical, mathematical and information sciences.As mentioned earlier, even though the laboratory and the concerned scientists are well-recognized and well known and act as experts/peers, such and similar contributions are extremely difficult to measure.It is therefore obvious that in public funded R and D institution such as CSIR that has scope to contribute in all four goods, the number of publications in high impact factor journal or number of citations alone cannot be the deciding factor for performance evaluation of a laboratory.Thus, there are parameters, which are measureable or tangible and also other parameters that cannot be quantified or are intangible in nature.An objective evaluation has to be done through a more holistic approach that will not only recognize, but also provide methodologies to compute measures of performance considering all these.A typical case study of two of the laboratories of CSIR is presented below.It is designated as laboratory A and laboratory C for the sake of confidentiality and propriety, without identifying the laboratory.The values of four goods as per Table 3 are presented in Table 4.It may be noted as mentioned earlier that these values for four "goods" are arrived at based on the survey carried out earlier (CSIR Report, 2005) over a 10 year period.
The values of the different parameters of performance have been collected for a 10 year period and these values have been grouped into four different knowledge portfolios as discussed in Table 1.The values are represented in Table 5 to 8. Referring to these tables, the values under the columns A and C represent the actual values based on available data while N A and N C indicate the normalized values.The normalized values are arrived at by taking the ratio of the actual value to the maximum among the corresponding cluster of laboratories.In the present study, two laboratories considered belonging to the Life Sciences cluster coincidently possess the maximum value among themselves in many cases.However, in few cases where this is not true, the maximum value is indicated in bold but within brackets along with the description of the indicator.This can be seen in the Table 5 to 8 where the maximum values are marked in bold.This case is intentionally chosen for bringing in better clarity of the proposed methodology and algorithm.
The normalized values from Table 5 to 8 are now summed up against the corresponding knowledge portfolios and these are summarized in Table 9 for laboratories A and C.
While the quantitative numbers play a major role in arriving at the weights, such as the number of publications, number of products etc., the qualitative factors such as the quality of publications (in terms of impact factor, citation index etc.) must be also be considered.The utilization of resources in terms of manpower, infrastructural facilities and other assets must also be a guiding factor for these weightages.Another main factor for computing the weightages is the impact of the contributions of the laboratory either for the public/private/societal/strategic sector or suitable combination of any of the four goods.The character of the laboratory must be kept in mind at all times, while fixing the weightages.From the different surveys conducted among scientific publicly funded R and D institutions in India, it is found that the weightages generally range from 15 to 45% which is  We propose here the corresponding score S due to different knowledge portfolios as follows: The proposed (overall) laboratory score, S L is proposed as the sum of all the knowledge portfolios: where subscript L represents Laboratory and subscripts G, T, M and R correspond to Knowledge Generation, knowledge transfer, knowledge management and knowledge recognition respectively.Thus, the score for each of the laboratories A and C in the present case is computed as: Table 11 depicts the computation of the scores of laboratory A and C depending on the knowledge coefficient assigned to the corresponding knowledge portfolios represented vide Table 10.
On a scale of 100, thus the scores of Laboratories A and C work out to 90 and 94 respectively.These scores will help in objectively evaluating the performance of the individual laboratory and can also compare performance of one laboratory with another.Since the knowledge coefficients and parameters have been appropriately assigned depending on the character of the laboratory, the score would serve as a tool in assessing the performance of the laboratory as a whole.
The proposed methodology with suitable modifications can also be used as a bench mark to assess the individual performance of the scientists in the laboratory.

CONCLUSION
The performance of a laboratory should not be evaluated just by considering tangible outcomes and outputs, but also by quantifying and objectively evaluating the intangible benefits of research, so that laboratories are evaluated in a more holistic manner for their performance and this can facilitate in allocation of resources in a more rational manner.The purpose of this study is to: • Identify parameters of performance for the evaluation of a publicly funded R and D institution based on its character and contributions to the four goods, namely, public, private, strategic and societal.• Provide methodologies to quantify/measure the parameters to make the evaluation more objective.• Arrive at an analytical framework that can help in effectively measuring the parameters of performance through a suitable methodology and the four goods are appropriately combined into the four knowledge portfolios namely knowledge generation, knowledge transfer, knowledge recognition and knowledge management.
The portfolios relating to knowledge generation and knowledge transfer are essentially output based whereas the portfolios concerned with knowledge recognition and knowledge management are outcome based.
The entire process as stated in the above is demonstrated through case study of two laboratories belonging to life sciences cluster of CSIR.In the present study, it is successfully demonstrated that it is possible to estimate reliably and quantify not only the four goods but also the four knowledge portfolios, if an effort is made to collect and compile relevant information and data over a longer period.In the case study selected a ten year data and information is used to arrive at the estimates and also define the corresponding knowledge co-efficient, κ.An expression is proposed to compute the knowledge score as a product of the knowledge portfolio and square of the corresponding knowledge coefficient.Finally, an equation is proposed to determine the laboratory score which is an absolute value of performance or normalized value.This value can therefore be used to compare with any other R and D institution, if necessary.
The methodology and algorithm proposed through an analytical framework will facilitate each laboratory to critically assess its current state in terms of its stakeholder focus.As is expected from any organization, RoI can be a determining factor to assess the performance.This is necessary to ensure continuing stakeholder satisfaction as well as relevance.The laboratories must also, ensure that investment of resources (human, capital and financial) is aligned to the desired focus.Further, it is proposed as a future extension to arrive at methodology to evaluate the performance of scientists with suitable modifications and adaptations.One can also look at improving the algorithms further by different way of normalizing the indicators.

Table 1 :
Summary of parameters identified by individual laboratories Technical events organized such as conferences, seminars, workshops and capacity building programs, PG, doctoral and post-doctoral student guidance, human resource development, extramural R & D etc.RiSocially relevant S&T contributions through drafting of codes/codal provisions, manuals, handbooks, chapters, R&D reports etc.Knowledge recognition KR SocietalParticipation in policy decisions, international projects, inter-institutional MoU/collaborations/visits, framing of new standards, policies, procedures/ awards and recognitions for scientists etc. Mi Development/management of facilities, prestigious clientele, management of intellectual property through patents, copyrights etc.PrivatePrivate: Industrial training programs, consultancy services, certification and testing services and sponsored research are considered as private goods as beneficiary preferences is reflected in their willingness to pay for these services.Intellectual property, particularly patents, technologies, products, processes and copyrights are in the private domain, but public funds have been used both at their generation (project) stage and at the patenting stage.Social/societal: Social/Societal goods element is evident in activities, which generate livelihood opportunities to people located in far-off regions or to poor as in development of technologies, which use traditional knowledge and use of local resources endowments.Examples include natural hazard/disaster mitigation, environmental benefits from development and use of technologies, such as for coal-washing, mine safety, eco-friendly products and processes, pollution prevention and abatement.

Table 2 :
List of CSIR laboratories

Table 4 :
Distribution of four goods in CSIR laboratories A and C Laboratory

Table 5 :
Performance indicators for the knowledge generation (KG) portfolio ' indicates no significant measurable contribution; *NMITLI: The New Millennium Indian Technology Leadership Initiative (NMITLI) is the largest public-private-partnership effort within the R&D domain in the country.NMITLI evolved largely networked projects in diverse areas viz.agriculture and plant biotechnology, general biotechnology, bioinformatics, drugs and pharmaceuticals, chemicals, materials, information and communication technology and energy

Table 7 :
Performance indicators for the knowledge management (KM) portfolio

Table 8 :
Performance indicators for the knowledge recognition (KR) portfolio Bhatnagar award is the highest scientific recognition offered by CSIR and accepted by all institutions/organizations in the country as one of the most prestigious honor (it is equivalent to life time achievement) Table 10 represents the most probable knowledge coefficients for the two laboratories: