Abstract
This paper examines the theoretical issues and research themes of business and management impact. Our empirical setting is the UK Research Excellence Framework 2014 (REF 2014) and the focus is on the nature of research impact. Stakeholders, including Governments, now expect academic outputs to translate to real world benefits beyond the narrow bibliometric type metrics.
Despite decades of academic literature devoted to business and management research impact, current theories cannot explain the apparent disconnect between academic, economic and societal practice. Adopting a UK Business and Management perspective to frame our investigation, we consider the highly contested rhetorical question – What are the current themes and impacts of Business and Management research?
We propose a definition for research impact and consider its measurement. Then, using the 410 Impact Case Studies submitted to REF 2014 #x2013; Unit of Assessment 19, business and management, we examine how high impact unfolds. The implications for business and management research impact from the perspectives of economic, knowledge and responsibility impacts are considered.
Keywords
Citation
Phillips, P.A., Page, S. and Sebu, J. (2023), "Business and management research themes and impact", Emerald Open Research, Vol. 1 No. 4. https://doi.org/10.1108/EOR-04-2023-0008
Publisher
:Emerald Publishing Limited
Copyright © 2020 Phillips, P.A. et al.
License
This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background
UK business schools (BSs) contribute £3.35bn to the UK economy and provide financial cross subsidies to other university schools ( Bradshaw, 2017). Taking note of its business model, BSs were initially praised as a success story of higher education ( Alajoutsijärvi et al., 2015). BSs remain influential and possess the ability to attract huge numbers of premium fee-paying students on their undergraduate, postgraduate and executive MBA programs.
Yet a series of staff inspired polemical critiques of the nature of BS research have raised questions regarding whether such organizations have lost their way, due to criticisms by authors of a perceived lack of rigor and relevance in their staff research ( Irwin, 2019), along with an inward focus on high quality scientific research that has little application to the real world ( Kapstein & Yip, 2011), a feature polemicized by Caplan’s (2018) debate on the value of Universities. Van de ven (1989) reminds us that the central mission of management scholarship is contributing to management practice. Expectations exist that outputs from the academy should translate into tangible real-world benefits including societal impacts, environmental and economic impacts ( Pettigrew & Starkey, 2016; Watermeyer, 2014).
However, the last decade has witnessed a Tayloristic shift with geopolitical forces reshaping business schools. Calls for global action initiatives such as The United Nations Sustainable Development Goals (SDGs) place increasing pressure for BSs to demonstrate impact to their wider economic and societal audiences ( Weybrecht, 2020). According to the Association to Advance Collegiate Schools of Business (AACSB) more than 800 BSs are signatory to the Principles for Responsible Management Education (PRME) with increasing numbers looking beyond SDG 4 (providing a quality education).
Such observations illustrate the internal challenge that BSs’ face with two principal audiences for their research: academics and practitioners ( Kapstein & Yip, 2011). The blame varies from academics being caricaturized as being obsessed with theory and long-time horizons ( Rynes & Bartunek, 2017; Trank, 2014); physics envy ( Thomas & Wilson, 2011); the academy possessing a myopic view on what constitutes managerial scientific practice ( Seal, 2012); academics seeing impact as a dilution of rigor and threat to academic autonomy ( Johnson & Orr, 2019). Counter-arguments view practitioners as academic phobic, short-term natured and susceptible to the latest management fads advocated by management consultants ( Phillips et al., 2019). The different contexts of academics and practitioners have been acknowledged, so the gap is hardly surprising. These range from reward systems (e.g. Phillips et al., 2018); belief systems (e.g. Kieser & Leiner, 2009); communications flows (e.g. Hughes et al., 2017); lost in translation from theory to practice (e.g. Shapiro et al., 2007).
Research problem
This paper seeks to unravel what we know about business and management research impact and propose some more granular definitions. Then, with this impact agenda we seek to delve into this apparent disconnect to disentangle the practical issues of how high impact unfolds. We adopt a pluralistic approach ( Aguinis et al., 2019) and assess BS impact by examining Research Excellence Framework 2014 (REF 2014) impact case studies (ICS). Following on from these discussions we formulate our research questions as follows:
How should research impact be defined, and how is it measured?
How does high quality business and management ICS research impact unfold within the academy?
What should the role of BSs be in society, and what type of impact should they have?
The paper now considers the definition and measurement of research impact, including theoretical aspects of the concept of impact, REF 2014 and ICSs to contextualise the paper. The methodology employed to examine the ICSs and key findings is then presented. We then outline the implications, conclusions and consequences for policy development.
Definition and measurement of research impact
Definition of research impact.
Existing definitions of research impact are wide ranging making the construction of a working definition of impact complex. Definitions have varied depending on the focus of the researcher, which Penfield et al. (2014) divided into academic impact or external socio-economic impact. The UK Economic and Social Research Council 1, has indicated that academic research has impact if it is able to demonstrate some contribution to society and the economy, or more simply put it generates impact beyond academic stakeholders. The ESRC has traditionally framed this, like many other research councils in the UK do, in terms of developing pathways to impact where ‘high quality Pathways to Impact will include explicit awareness of principles and practices of knowledge exchange – including the application of principles and practices of co-production – as opposed to dissemination’ ( http://www.esrc.ac.uk/research/impact-toolkit/developing-pathways-to-impact/). However, in March 2020, UK research and innovation (including the research councils) advocated dropping the pathways to impact, with impact now included as a core element of the entire grant application process, illustrating the enhanced importance of impact as it is now embedded in the research funding process. In the case of the ESRC, research outcomes that affect society, culture, the environment and economy are deemed to have had impact beyond academia. Academic impact, in contrast, has often been measured in relation to citations of articles. The entire area of academic impact is one which is highly challenging (e.g. Amara & Landry, 2012; Garfield, 1979; Pendlebury, 2009), with its qualitative and quantitative measures of impact.
Lejeune et al. (2019) prefer a more granular approach towards impact, developing a three-fold categorisation comprising economic impact, knowledge impact and responsibility impact as illustrated in Table 1. We proffer some definitions to capture the importance of some element of a transfer of value in all three categorisations. We believe this is crucial as it moves the dial away from activities, and focuses on real impact among actors.
Measurement of research impact.
Analysis of academic ‘impact’ has traditionally adopted quantitative measures, focusing on frequency of research citations and the journal impact factor (IF) ( Hall & Page, 2015; Holmberg et al., 2015). A common complaint is the disconnect between the knowledge produced by academics and that which is consumed by practitioners ( Aguinis et al., 2019; Jackson et al., 2014), a gap that arguably is widening. The focus on traditional scholarly impact is depicted in terms of an internal exchange within the field of academia ( Aguinis et al., 2019). This internal focus normally considers journal evaluation techniques such as journal impact factors, including the H-Index, SCImago Journal Rank (SJR), Source Normalized Impact per Paper (SNIP) and Eigen factors ( Mingers & Yang, 2017). But, these methods of measuring impact only help to ascertain the benefit of research within academia and so are only ever going to be a partial analysis of impact ( Bornmann & Marx, 2013). The main challenge remains expanding the external reach of BS research beyond academia to permeate practitioners, given the scope of many BSs missions to prepare students for the world of work and engage with businesses to help create economic value.
New frameworks have been developed as outlined by Penfield et al. (2014), which advocated a mixed method approach incorporating the case study method as a way of collating all available information, data and evidence to coherently summarise impact. Aguinis et al. (2012) suggested an alternative approach to impact, adopting the number of pages indexed by Google to assess the impact of academic research outside academia as opposed to the use of citation data. Aguinis et al. (2012) adopted five steps and decision points to ensure the validity of their measure. Using a sample of 384 highly cited management scholars over the past 30 years, they demonstrated that scholarly works only affected stakeholders within academia and not stakeholders outside academia, a worrying feature given the external reach embodied in many BSs missions. Aguinis et al. (2012) advocated that academic impact is a multidimensional construct and not easily captured and incorporated through conventional academic measures.
There are two underpinning concepts which REF 2014 introduced to guide the assessment of research impact in terms of the significance of the research to the wider community and how far it had reached these end users captured in two documents submitted by each UOA 2. The impact score accounted for 20% of the total score in RE2014 and it is rising to 25% in REF 2021. REF 2014 defines research impact as “the effect that research has had on the economy, policy, culture, the environment or quality of life outside the academic sphere”. Research performance being measured on a scale from 4* (world-leading) to unclassified. With 4* Quality world leading; 3* internationally excellent; 2* internationally; 1* nationally.
Theoretical aspects of the concept of business school impact
Theoretical challenges.
As the growing number of papers on this subject posit, a number of major theoretical challenges for the academy are prominent. For example, Teagarden et al. (2018: 304) argue that it is important to rise to the theorization challenge in business research as ‘contextualizing…[helps] to achieve research rigor and practical relevance is a challenge faced by all sub-disciplines.’. This means looking at research through a different lens and adopting multiple levels of analysis to incorporate theory and impact. This tension between rigor and relevance as embodied in the debate on research impact has become a substantial problem ( Head & Alford, 2015) (i.e. one which is where there is incomplete theoretical knowledge, and so is impossible to address because of conflicting objectives) as the debates over the role of BSs continue to emphasize (see Hardy & Everett, 2013 for a more detailed discussion). This problem led Hamet & Michel (2018) to explore the theoretical explanations of the problem further, arguing that it is a consequence of multiple factors such as the mission of BSs (i.e. the education-research nexus), the favouring of positivist epistemologies, while impactful research is recognized as lacking rigor. This is inherent in the research-practice debates which led Kieser et al. (2015) to suggest the causes of the research-practice debate was associated with various streams of thought including:
A transmission problem, with poor dissemination to the wider communities of practice who popularise the research
Institutional causes, where narrow disciplines do not collaborate with business reflecting the weaknesses of the academy in practice
An overconcentration on theoretical research and high-level journals, to the detriment of action research methodologies, especially qualitative methodologies.
Beer’s (2001) arguments would seem to support these explanations, on the reasons why so little business and management research have any impact (i.e. they are unimplementable). This is attributed to the limited success of new management epistemologies and an auto-referential system of knowledge production that reinforces the positivist paradigm. Yet these explanations appear to not be supported in other disciplines, since in Science the knowledge acquisition process by non-academic stakeholders has overcome the schism between researchers and practitioners.
This has led Narasimhan (2018) to argue that the reward structure of BSs has placed a premium on high level scholarship to the detriment of the real world experience of academics. Yet as we argue in this paper, research impact as espoused by the REF exercise actually promotes rigor (i.e. the intellectual quality of the research which is highly subjective) and relevance (i.e. the production of useful knowledge that can address real world issues). This has led Van de Ven & Heath (2007) to promote the concept of the engaged scholar. In a similar vein, Birnik & Billsberry (2008) suggest the need for a shift away from pure self-interest of the academic to address a crisis of relevance in BSs, by adopting a more pluralistic approach where self-interest and altruism is fostered. In many countries the state has promoted ‘Grand Challenges’ to stimulate altruistic thinking and to enhance the application of research to well-being and the public good.
All of these activities need to be undertaken by the researcher to develop a multifaceted and outward facing process. In some cases, innovations in teaching (see Antonacopoulou, 2010) suggest the solution to making BSs more impactful is phronesis. This concept, from Ancient Greek, extols the virtues of wisdom, good judgement and common sense in management practice, as a basis for extending the theoretical debate on rigor and relevance. Such a seed of change is almost calling for a theoretical shift away from the conventional disciplines and their theoretical bases, as a means to address business-related problems away from their narrow disciplinary focus. Instead, research becomes more impactful by embracing a more critical social science range of specialisms and harnessing these with external stakeholders, which begins to move us away from theoretically anchored multidisciplinary approaches to research and education and towards new modes of knowledge production around critical social science.
There is a long history in research, and consequently a vast research stream debating if BSs should impact society, and what their impact can be. Recent literature has considered such issues. One school of thought suggests the modern BS is an overloaded institution responsible to multiple stakeholders and for multiple purposes ( Irwin, 2019). A core driver in the reward mechanisms for BSs is the playing and winning the “indicator game”. Universities vie for accreditations and rankings and this distorts behaviour and outputs. Nevertheless, despite BSs vision and mission statements conceptualizing themselves as “for and about business” the reality is much different. As Bennis & O’Toole (2005) retort in their seminal article on this “dirty little secret”, academics are preoccupied with their self-serving attitude with scant regards to stakeholder needs. BSs should think and act more pro-actively about wider societal goals, as opportunities will be lost to set the agenda globally beyond the “indicator game”. In response to this situation, Parker (2018) contends that if Universities have responsibilities to society, BSs have to comply or be shut down.
REF 2014 and Impact case studies.
The empirical setting for this study in terms of REF2014 and ICS has been described in Phillips et al. (2020) in relation to the field of Tourism, Sport, Leisure and Hospitality and their outcomes in REF 2014. Business and Management was one of the largest Units of Assessment (UOA19) in contrast to the area examined by Phillips et al. (2020) which was smaller and more niche in focus. In this paper we delve into the themes and impacts of BSs.
In accordance with UK Research and Innovation, REF 2014 impact for this study is broadly defined “as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”. Its measurement is potentially based on a peer review system, with scores being determined by a combination of factors, including quality of publications, quantity and value of research income, team profile and characteristics, spin offs created and evidence of esteem among influential stakeholders ( Chowdhury et al., 2016).
In the £250m REF 2014 exercise 6,975 ICSs were submitted for all UOAs and in UOA19, 410 case studies were submitted by BSs and 37.7% of them were judged to be outstanding (4*) with 42.5% judged as very considerable (3*) and this was only 3% below the average score for impact for all of the REF 2014 ICSs. Although this could be used as evidence and reflection of some excellent impact generated by BS’, key questions that the REF 2014 does not answer are: how have BS’ affected their main constituents? And have they produced distinct and disparate pathways to impact? While there are a variety of impact outputs, the main route to impact for BS’ given their mission should logically be businesses, or a contribution to government policies that directly improve business performance and so economic activities 3. A significant question is what do BS’ see as their primary and secondary mandates, and how is the creation of impactful research informing and influencing the management of these organizations? In other words, what does a 4* UOA19, high-quality ICS look like?
We must also recognise that in REF 2014 ICS’ were often chosen centrally by participating institutions (mainly universities). So, this makes it challenging to draw general conclusions on BS impact for the following reasons:
There was a very limited data set of cases from each institution.
The entries were not necessarily those advocated by BS’.
Many ICS’ mapped to REF 2014 criteria and frequently went beyond economic impact.
The impact of BS was excluded.
ICS’ did not necessarily include the magnitude of impacts.
Following the publication of the results of REF 2014, several studies have attempted to identify what makes a good ICS and how to improve one’s scores for the next REF (e.g. Hughes et al., 2017; Saltera et al., 2017). But there has been no study that has provided an in-depth analysis of ICS for UOA19 despite the scale and significance of this UOA, second only to fields allied to Medicine.
Methods
The study is rooted in a two-stage sequential mixed methods approach, adopting a pragmatic research paradigm. The reason for adopting a pragmatic paradigm is the decision to combine the exploratory analysis of a large body of secondary data that is predominantly based on qualitative data with a more detailed quantitative investigation of outcomes. The first stage adopts a qualitative analysis of the themes based on secondary data generated by the REF 2014 process associated with ICSs. The second stage was to use a set theoretic approach to ascertain which configurations arise from the ICS’ as a body of qualitative textual data. The purpose was to find an analytical technique capable of combining different attributes, which create successful or unsuccessful outcomes. This would identify some of the paths that would explain high and low levels of external impact.
Secondary analysis of REF 2014 ICS’
The first stage in the research process was to undertake a descriptive analysis of all 410 ICS’ submitted to UOA19, gathered by downloading them from the REF 2014 website. Initial grade point average (GPA) scores exist for the impact and the output scores for all 101 submitting institutions. This was then used to initially find the relationship between the impact score as the dependent variable and the output score as the independent variable by way of a scatter plot as an initial starting point which was developed in Kellard & Śliwa’s (2016) study.
A scatter plot of the relationship between the impact GPA and output GPA scores is shown in Figure 1, a feature explored in terms of staff interest in each by Saltera et al. (2017). The plot shows a positive relationship between the two GPA scores which can also be inferred from the slope of the regression line (0.86). The slope implies that an uplift in the output GPA score by one unit increases the impact GPA score by approximately 0.86. However, the R-squared shows that the output GPA score only explains about 37% of the variation in impact GPA score. This implies that there are other factors together with the output GPA score that explain the score of ICS. The scatter plot clearly shows an outlier in the data. This is the University of York St. John which had the minimum impact GPA as observed earlier. To prevent further analysis being influenced by this outlier, University of York St. John was taken out of the data. Therefore, the University of Chester then becomes the institution with the least impact GPA score of 1.60.
The second stage was to identify three clusters to group institutions into top, middle and bottom groups based on their impact GPA scores. 10 institutions were then selected from each group, extracting their “summary of impact” from the ICS. The sample of 30 ICS was selected so as to confirm the results of the Kellard & Śliwa’s (2016) study to ensure a consistent methodological approach 4. The first top 10 institutions with the highest impact GPA were chosen and these are presented in Table 2. The middle and bottom ten institutions with the least impact GPA score were also obtained and presented in Table 3 and Table 4.
The middle ten was obtained by first obtaining the median which was the 49th term occupied by University of Central Lancashire. We moved four steps up and five steps down to get all the middle ten institutions. In all cases, where institutions had a similar impact GPA score, their respective output GPA score was used to rank them. For example, in selecting the bottom 10 institutions, University of West of Scotland, University of Westminster, University of Wolverhampton and University of Sunderland had similar impact GPA scores of 2.3. Counting from the bottom, only two institutions from this list were needed to complete the bottom ten. Arranging them in descending order of their output GPA score, University of Wolverhampton (2.0) and University of Sunderland (1.13) were the least. However, the UOA19 impact case studies for University of Wolverhampton could not be located among the case studies submitted on the REF website. Hence, University of Westminster, which has the next highest output GPA of 2.34, was used in place of Wolverhampton.
Set theoretic approach
The structure of a higher education institution (HEI), like any major organization, is made up of a complex set of attributes that work together in order to achieve an outcome. Organizations possess different skills, resources, structures, and environments that all come together to influence what could be achieved depending on how they are used together. Hence, in any analysis of performance or outcome, one should examine an organization in a systematic and holistic way. No one attribute or characteristic of an organization is able to achieve an outcome without the influence of other attributes, since it involves the interconnection among various attributes or characteristics in that organization. An appropriate piece of supporting theory is configuration theory. According to Fiss (2007) a configurational approach predicts that organizations can be understood to consist of clusters of interconnected structures, profiles or patterns and practices.
Unlike traditional multivariate techniques that mostly assume a linear relationship between a set of independent variables and a dependent variable, configuration theory assumes a complex interconnection between variables, attributes or characteristics. These interconnections can be in the form of a direct relationship, a reverse causality or have an inverse relationship between factors. The absence or presence of some factors could enable an increase or decrease in the outcome. Hence, different organizations may yield similar results using different strategies or even facing different conditions and factors. These configurations are a combination of different attributes, which could yield different or similar outcomes.
To empirically test how configurations affect an outcome, the set theoretic (ST) approach is used. This study employs Fiss’ (2007) application of a ST approach in business and management research, which uses the aggregated relationships and configurations which can be traced back to the development of set theory in mathematics. Set theory postulates that different cases could influence a certain outcome. The ST approach adopts Boolean algebra to determine which configurations, or the combination of organizational factors combine to produce a particular outcome, like performance. The factors or attributes are converted into truth values, true or false, using a value of 1 and 0 respectively.
The configurations in this study are made up of attributes that interconnect for HEIs to produce impactful research, where the impact scores obtained for an impact research evaluation is the outcome or dependent variable. The first stage of the research method is to identify factors or attributes that could interrelate to produce impactful research. This must be based on existing knowledge of the possible factors influencing the outcome. For this reason, we examined the existing research impact literature to identify what variables could influence outcomes (see Table 5).
Data for these variables was obtained from the study by Kellard & Śliwa (2016). These were a combination of their Table 6, Table 8 and Table 10, which were the ICS metrics of the top ten, middle ten and bottom ten groups. Table 2, Table 3 & Table 4 present the members of each group and their respective scores in terms of impact and output GPA scores. The data was calibrated into fuzzy set scores for use in the fuzzy set qualitative analysis (fsQCA) software ( Ragin, 2006) that undertakes ST analysis. Fuzzy sets are an extension of the crisp sets, which allows membership scores ranging from 0 to 1 ( Ragin, 2006). Three-value fuzzy sets were used in this study. Three thresholds were determined to help with the calibration and the ratio-scale and interval scale variables for fuzzy membership scores. The thresholds included 0.95 for full membership (high impact score), 0.5 for cross-over point and 0.05 for full non-membership (low impact score). Table 6 displays the calibration thresholds chosen for the conditions in the data.
Following the calculation of calibration thresholds, the next step was to create a truth table. The truth table stipulates the outcome for each configuration and determines which configurations to include in the analysis. The next step involved specifying the causal configurations and outcomes to be minimized. The truth table was then generated which included 1024 configurations. These were all the possible configurations for our data. The output of the truth table recorded 1s and 0s, which indicate different corners of the vector space defined by the fuzzy set causal conditions. Configurations linking a frequency threshold of cases of at least one was set as the rule for classifying which configurations were relevant. A consistency threshold of 0.9 was used to identify outcomes (impact) that will be given a 1 and 0 for those configurations with consistency score of less than 0.9 and this is shown in Table 7 below.
Results
Boolean logic was applied to determine commonalities among configurations that lead to the outcome and to generate logical statements. The Quin-McCluskey algorithm is applied to simplify ST statements using the fsQCA software package ( Roig-Tierno et al., 2016). The parsimonious solutions generated from the analysis are reported in Table 8. Five solutions were obtained from the analysis revealing several paths to achieving a high impact score. Solution one shows that the average grant listed in thousands of pounds was the most important factor for achieving a high impact factor. Secondly, achieving a high impact factor is a product of high number of ICS’ cases submitted and a low percentage of national reach. This may seem contradictory as it was expected that a high average percentage of national reach would produce a high impact score. The third solution revealed that a product of having spent several years in post and a lesser percentage of national reach also produce a high impact factor. For the fourth solution a product of a high number of ICS’ submitted and a high average number of key researchers produced a high impact factor. The last solution is a product of a high number of ICS’ submitted and longevity of time in post as shaping the high impact score.
The solution output included measures of coverage and consistency for each of the configurations and the solution as a whole. The consistency measures according to Ragin (2006) are the measure of the degree to which each configuration term and the solution as a whole are subsets of the outcome. For the individual solutions, a solution is said to be consistent if membership in the solution is less or equal to membership in the outcome. The fourth solution has the highest consistency score of 0.95 and the first solution has the least consistency of 0.78. The solution consistency is 0.85 and this measures the degree to which membership in the solution is a subset of membership in the outcome.
The raw coverage measures the number of or percentage of outcomes of cases in the outcome explained by each term of the solution. Solution coverage measures the proportion of memberships in the outcome that are explained by the complete solution ( Ragin, 2006). According to Elliott (2013) the consistency and coverage scores should be in ranges that validate the solutions. Hence, with both the solution consistency and solution coverage score being high and relatively close, the results could be said to be valid.
Main findings
Theoretical implications.
As such, our results contribute to the “black box” by outlining five solutions in terms of paths of achieving a high impact score. Evidence of a significant level of grants was the most influential factor, which in itself can be a due to a myriad of characteristics including past productivity, collaboration with stakeholders, BS environment, and research strategy. It was noted that being part of a large research group with several years in post contributed to a high ICS score. The results illustrate that a track record of winning research grants, size of research team, number of ICS produced, reach of impact, together with a low staff turnover of key staff were helpful in terms of ICS success. As can be seen, many of these attributes refer to internal impact. Relationships across boundaries reside at the heart of successful ICS and these can include business, professions, policy, and civil society. If relationships are an important driver of success, then further analysis is required to operationalise research impact with interested stakeholders. We will now reconsider our definitions and categorisations of economic impact, knowledge impact and responsibility impact.
Economic impact.
Interestingly, issues that can be explained as external impact are not significantly made in the business domain. The significant point from this ICS study, in terms of research impact themes, is that BSs possess a limited focus on business despite this being a core element of their mission. There appears to be a failure of practicing what they preach. From the theoretical debates on the rigor-relevance debate, it is clear that this is not a philosophical debate around epistemologies. This is pertinent to the debate, illustrating a lack of what Van de Ven & Heath (2007) term as engaged scholarship which incorporates both rigor and relevance. So, an apt question to ask is what do they focus on? From an external perspective, the ICS provide a novel mechanism to examine the policy implications for BS research and engagement with stakeholders.
More importantly, there was the expectation that as a business and management UOA, most of these themes would be economic impact related with an input-output focus. However, the evidence clearly shows that this was not the case. The implication of recognising the effect of this lack of business focus should not be lost in terms of contribution to the UK long-term macroeconomic performance.
Knowledge impact.
BSs can engage with the National Industrial Strategy, which outlines some of the major pillars (and its successor launched in July 2020 - The Research and Development Roadmap) with one of its key objectives to ‘become world-class at securing the economic and social benefits from research’ (i.e. impact). The Industrial Strategy and its successor include key points. BSs have research activity that span many of these themes such as supporting businesses to start and grow, developing skills, encouraging trade and investment and improving procurement. So BSs understand the nature of these changes. Research England’s Knowledge Exchange Framework could be a driver for change, as it seeks to make available performance information about Universities knowledge exchange activities. The metrification of this activity will reinforce the importance to both universities and BSs.
BSs aiming to deliver value creation strategies via knowledge impact need to consider the issue of value contextualisation (conceptualisation, construction, delivery and realisation). Then consider if value is symbolic or tangible. This will provide some stimulus for their knowledge impact. For example, is the beneficiary of the proposed value offer willing to pay a premium price? Branding is crucial, as the more symbolic and abstract the knowledge the higher will be the perceived knowledge and fee.
Responsibility impact.
BSs are inherently multidisciplinary, producing complexity in its constituent parts, and impact is a very serendipitous activity, based on networks and collaborative behaviour of individuals and groups. Stakeholders now expect BS staff to be influential externally as well as within their internally oriented academic community. BSs need to acknowledge that influential studies such as Giddens (1998) The Third Way, have shifted the expectation to also focus on citizenship and the development of a more active civil society ( Edwards, 2013) which is an emergent research strand for BSs (see Page et al., 2017) around business engagement and grand societal challenges.
The modern plurality focus of BSs together with more demanding stakeholders necessitate consideration of the third way and civil society agenda. The UK Goverment with its number of outputs from the Department for Business, Energy and Industrial Strategy (see Building our Industrial Strategy: green paper and the Made Smarter Review) illustrates the challenge and opportunities for BSs. Phillips et al. (2020) articulate this in greater detail. Nevertheless, the apparent gulf between theory and practice suggest the need to develop new mechanisms for producing the outcomes to close the gap. But the capability if not the capacity is prevalent in BS, and there is need for BSs to show how their research can influence policy and practice. Research is now gathering pace with the Economic and Social Research Council (ESRC) and UK Research and Innovation (UKRI) announcing rapid response calls for coronavirus disease 2019 (COVID-19), which is the current enduring problem for HEIs.
Conclusions
This paper suggests that BSs research strategies that have primarily focused on high quality scholarship and outputs are not enhancing the engagement agenda for economic impact. The challenge for universities that wish to see their BSs engage more fully with stakeholders, unlike many other faculties, is the large student population and revenue generating role they have for their organizations. It means prioritising day-to-day operational and strategic foci on delivering the annual surplus, predominantly from teaching income. Therefore, repositioning BSs to make their research more highly engaged with stakeholders will need a major rethink, especially about the mission and implications for the theoretical approaches and focus on the academy.
A more liberal approach to what is deemed rigorous and relevant needs recognising and given a greater weighting in reward structures. There is no disputing BSs commitment to undertaking high quality scholarship, but that alone will not serve it or its diverse stakeholders and the research climate it is operating in. The focus of output excellence over the last three decades on research assessment exercises cannot easily be reversed even with the growing significance of ICS’. Developing ICS’ is a new skill for many academics to adopt, understand and implement with external partners that places their research at the heart of the triple helix if executed skilfully ( Ekzkowitz, 2008). We have demonstrated that a ST approach to the analysis of ICS may help in understanding what pathways we need to adopt to reach that central point in the triple helix so as to create more impactful research not only for ICS’ but more generally that meet the twin objectives of rigor and relevance. These findings illustrate that there is strength in clustering research around areas of excellence based on longevity of service, research income and teams which can be applied to address research problems over a sustained period of time. Putting the ‘business’ back into BS research appears to be the greater challenge from the analysis of ICS’ if these are representative of the wider BS academy and impactful research.
In terms of knowledge and responsibility impacts, BSs have a national contribution to make towards policy and R&D, as well as society ( Morsing & Rovira, 2011) but the ICS evidence does not immediately align BSs with the needs of national policy where the underlying driver of the industrial strategy is improving productivity and industrial growth. These issues are not peculiar to BSs and to an extent affect the perception of Universities in the social sciences, where their contribution is not always directly associated with cause and effect research relationships that are more visible in science, where there is more sustained engagement with business.
Nevertheless, this study does provide BSs and Universities with a critical debate around the extent to which they produce impactful research, how they evidence that and more importantly, communicate that to external stakeholders, drawing upon the theoretical development in Science and how they have become successful (see Greenwood et al., 2002). It is the communication and relationship building to articulate relevance synergies with business that may need a rethink in the 21st century. This goes to the very essence of BSs purpose, particularly their intended outward-facing mission so that business engagement is reinvigorated to give it meaning, where it is evidenced through impactful research and applied problem-solving capabilities locally, regionally, nationally and internationally.
Data availability
Source data
This link leads to information on the output and impact scores of each submitting institution: https://results.ref.ac.uk/(S(fpqtu01uy4jjci5ubrrqdp5x))/Results/ByUoa/19
This link leads to the 410 case studies that were submitted to the Business and Management Unit of Assessment: https://impact.ref.ac.uk/casestudies/Results.aspx?UoA=19
Publisher’s note
This article was originally published on the Emerald Open Research platform hosted by F1000, under the ℈Quality Education for All and Responsible Management℉ gateway.
The original DOI of the article was 10.35241/emeraldopenres.13987.1
Author roles
Phillips PA: Conceptualization, Data Curation, Methodology, Project Administration, Supervision, Writing - Original Draft Preparation, Writing - Review & Editing; Page S: Conceptualization, Methodology, Supervision, Writing - Original Draft Preparation, Writing - Review & Editing; Sebu J: Data Curation, Investigation, Methodology, Software, Writing - Original Draft Preparation
Funding statement
The authors wish to thank the Kent Business School, University of Kent and Hertfordshire Business School, University of Hertfordshire for funding towards the research reported in this study.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests
No competing interests were disclosed.
Reviewer response for version 1
Ludo Waltman, Centre for Science and Technology Studies, Leiden University, Leiden, Netherlands
Competing interests: No competing interests were disclosed.
This review was published on 29 April 2021.
This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendation: reject
First of all, I need to acknowledge that I have no specific expertise in business research. The paper extensively discusses the role of business schools, in particular in the UK. While this seems to be an important discussion, I don’t have the expertise to evaluate this element of the paper. I will focus my review primarily on the methodology adopted by the authors.
I appreciate the authors’ idea of using a methodology based on configuration theory (as opposed to a more commonly used methodology such as regression analysis). The authors take a set theoretical approach. Unfortunately, however, the paper does not provide enough methodological information to understand this approach in sufficient detail. Since the set theoretical approach taken by the authors is not very common, readers cannot be assumed to be familiar with this approach. It is therefore important to provide relatively detailed methodological information. Let me give a few examples of the lack of clear methodological information:
“Three-value fuzzy sets were used in this study. Three thresholds were determined to help with the calibration and the ratio-scale and interval scale variables for fuzzy membership scores. The thresholds included 0.95 for full membership (high impact score), 0.5 for cross-over point and 0.05 for full non-membership (low impact score).”: This is not clear to me. What do the values 0.95, 0.5, and 0.05 represent? Do they represent percentiles? Also, what happens when a value is below the lowest threshold (0.05)? Shouldn’t this also be interpreted as non-membership?
“The next step involved specifying the causal configurations and outcomes to be minimized.”: What do you mean by minimizing the outcomes? This is not clear to me. The rest of this paragraph is also hard to understand. The methodological information needs to be expanded. Providing an example may help to clarify the methodological details.
“The Quin-McCluskey algorithm is applied to simplify ST statements using the fsQCA software package (Roig-Tierno et al., 2016).”: It would be helpful if some discussion could be provided explaining what exactly the Quin-McCluskey algorithm is doing. Providing a full technical explanation may not be possible, but as a reader I would like to get at least a proper intuitive understanding of what the algorithm is doing.
“The consistency measures according to Ragin (2006) are the measure of the degree to which each configuration term and the solution as a whole are subsets of the outcome. For the individual solutions, a solution is said to be consistent if membership in the solution is less or equal to membership in the outcome.”: I don’t understand this. This requires a more detailed explanation.
In the first paragraph of the Results section, the authors should not use the term ‘impact factor’. The use of this term is confusing, because the term could easily be understood incorrectly as the journal impact factor.
Finally, the authors need to carefully reread their paper to fix some typos and minor inconsistencies in the text. Some sentences could also be phrased in a more elegant way (e.g., “a large body of secondary data that is predominantly based on qualitative data” and “theoretical debates on the rigor-relevance debate”).
- Is the work clearly and accurately presented and does it cite the current literature?
Partly
- If applicable, is the statistical analysis and its interpretation appropriate?
Partly
- Are all the source data underlying the results available to ensure full reproducibility?
Yes
- Is the study design appropriate and is the work technically sound?
Partly
- Are the conclusions drawn adequately supported by the results?
Partly
- Are sufficient details of methods and analysis provided to allow replication by others?
Partly
I have expertise in scientometrics and research evaluation, but I do not have expertise in business research.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.
Reviewer response for version 1
Julie Davies, Manchester Metropolitan University, Manchester, United Kingdom
Competing interests: No competing interests were disclosed.
This review was published on 03 December 2020.
This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendation: approve
Feedback:
Thank you for inviting me to review this article. Overall, it provides a useful overview of debates in the literature about tensions between management scholars and management practitioners. It draws on configuration theory to frame the analysis from a systemic perspective, specifically set theoretic configurational methods. The sample of REF 2014 impact cases in the UK is clearly explained. The findings are particularly interesting in terms of the context of impact case teams with a focus on economic, knowledge, and responsibility. This paper provides clear definitions and explanations about measuring research impact. It also offers emphasizes the need to demonstrate greater impact with businesses with insights that are generalizable beyond the UK.
The paper would benefit from reflections on its limitations. I would also like to see the authors’ thoughts on potential specific interventions. What future agenda on research impact related to communications and relation building do you propose in the light of consultations about REF 2027 and in the context of the pandemic?
Abstract:
There is scope to sell the contribution of this paper as the only in-depth analysis of cases for business and management studies, the second largest unit of assessment. It would be helpful to mention the set theoretic (ST) methodology, configuration theory, and specific Tourism, Sport, Leisure and Hospitality case. Can you provide some indication of the answers to your question here: ‘what are the current themes and impacts of business and management research?’ Refer to your key findings about key factors that support impact case study success which are important takeaways: primarily winning significant research grants, as well as a large research team, reach of impact, low turnover of key staff. The insight that there is a need to put ‘business’ back into BS research is an interesting finding. This is also an important point to include in the abstract: ‘relationships across boundaries reside at the heart of successful ICs.’
Introduction:
It would be useful to include an introduction that indicates the problem statement, key questions, theoretical framing, methodological approach, and structure of the paper.
Background:
You provide a clear overview about debates relating to management academics and management practitioners as audiences.
Can you include a citation about your point that business schools are influential? It might be helpful to start with the broader picture first and then focus on the UK later in terms of justifying the choice of context and sample.
Include a citation for Tayloristic.
You might cite Bennis and O’Toole in reference to business schools losing their way.
SDG4 – note that this is about providing a quality education.
Consider the requirement in AACSB standards to demonstrate impact and EFMD’s BSIS in the business school sector.
Research problem:
The research questions are clearly stated. It would be useful to see some insights into why the research problem matters, e.g. for individuals’ careers, the legitimacy of business schools in society, and some indication perhaps about consultations regarding the REF after 2021 in terms of a greater focus on research teams (Rob Blackburn is a good point of contact) to alleviate current problems.
Definition and measurement of research impact:
You might refer to the Australian context as this is where a national policy on evaluating research impact began. The UK approach appears to be based on PESTLE.
Theoretical challenges:
You may find that these articles relevant here:
Johnson S. and Orr K. (2020) What is business school research for? Academic and stakeholder perspectives, politics and relationality 1.
Ployhart, R.E. and Bartunek, J.M. (2019). Editors’ comments: There is nothing so theoretical as good practice—a call for phenomenal theory[ref2].
REF 2014 and impact case studies:
It is worth noting the number of impact cases selected and that not all UK academics are engaged in the REF impact case studies agenda.
Methods:
These are clearly explained drawing on an approach in a previous study.
Results:
This should be in an earlier section: “Boolean logic was applied to determine commonalities among configurations that lead to the outcome and to generate logical statements. The Quin-McCluskey algorithm is applied to simplify ST statements using the fsQCA software package ( Roig-Tierno et al., 2016).”
Main findings:
It would be useful to present these diagrammatically as a configuration. What are the limitations of the methodology and your analysis?
Conclusions:
What practical solutions might be offered in terms of implications for practice looking forward to post REF 2021? How might publishers, business journalists and other stakeholders support communications and relationship building for impact?
Further literature you might consider in relation to drivers for performativity in UK HEIs, the need for systemic change in UK business schools to support research impact, and multidisciplinary approaches through integrating the business school better within the university:
Watermeyer, R. (2019) Competitive accountability in academic life: The struggle for social impact and public legitimacy 3.
Davies, J., Yarrow, E. and Syed, J., 2020. The curious under‐representation of women impact case leaders: Can we disengender inequality regimes? 4.
Currie, G., Davies, J. and Ferlie, E., 2016. A call for university-based business schools to “lower their walls:” Collaborating with other academic departments in pursuit of social value 5.
Presentation points:
There are quite a few grammatical and punctuation points to correct. For example:
Keywords: Impact Set Theoretic Approach, Research performance – be consistent with using lower case and there should be a comma after impact.
Add ‘impact to’ before “their wider economic”.
“its business model” - for plural BSs needs to be amended.
“Series” should be followed by ‘has not have’
“so the gap” – no comma.
Insert comma before “we formulate”.
Omit comma: “But, these...”.
“Growing number” should be followed by singular verb.
Number is repeated.
No comma before “or the reasons”
“To not be supported” – change to not to be
“Favouring” – use US spelling consistently
“Do you mean hard, physical science when you talk about Science/Such a seed…” - this sentence is very long.
“Towards” – use toward in US spelling, same for “regards”.
Insert comma before “we delve”.
BS’ and ICS’ needs to be corrected throughout.
Use either bullet points or a), b) etc., but not both.
Delete “the” before “Kellard”.
Use ‘an’ not “a” before ST.
“Categorisation” should be ‘categorization’, same for “conceptualisation” – US spelling.
“To the UK long-term” – use UK’s.
“Goverment” – misspelt.
- Is the work clearly and accurately presented and does it cite the current literature?
Yes
- If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
- Are all the source data underlying the results available to ensure full reproducibility?
Yes
- Is the study design appropriate and is the work technically sound?
Yes
- Are the conclusions drawn adequately supported by the results?
Yes
- Are sufficient details of methods and analysis provided to allow replication by others?
Yes
The business and management education field, impact cases, UK context
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
References
1. Johnson S, Orr K: What is business school research for? Academic and stakeholder perspectives, politics and relationality. Studies in Higher Education. 2020; 45 (3): 557-578
2. Ployhart R, Bartunek J: Editors' Comments: There Is Nothing So Theoretical As Good Practice-A Call for Phenomenal Theory. Academy of Management Review. 2019; 44 (3): 493-497
3. R Watermeyer: Competitive accountability in academic life: The struggle for social impact and public legitimacy. 2019.
4. Davies J, Yarrow E, Syed J: The curious under-representation of women impact case leaders: Can we disengender inequality regimes?. Gender, Work & Organization. 2020; 27 (2): 129-148
5. Currie G, Davies J, Ferlie E: A Call for University-Based Business Schools to “Lower Their Walls:” Collaborating With Other Academic Departments in Pursuit of Social Value. Academy of Management Learning & Education. 2016; 15 (4): 742-755
Figures
Definition of research impact.
| Nature of impact | Definition | Examples in Literature |
|---|---|---|
| Economic impact | A deliberate managed process across university boundaries that creates value through connections, and delivers financial benefits to external communities | Azmat et al., 2018; Cooke & Galt, 2010; Kelly & McNicoll, 2011; Valero & Van Reenen, 2019; |
| Knowledge impact | A deliberate managed process across university boundaries that creates value through connections, and results in knowledge exchange that enables external communities to move forward | Amara et al., 2016; Bager, 2018; Tho, 2017; Tho & Trang, 2015. |
| Responsibility impact | A deliberate managed process across university boundaries that creates value through connections, and delivers societal oriented benefits to external communities | Murcia et al., 2018; Snelson-Powell et al., 2020 |
Top 10 institutions.
| Institution | Impact | Output |
|---|---|---|
| University of Cambridge | 3.84 | 3.19 |
| University of Aberdeen | 3.80 | 2.77 |
| University of Strathclyde | 3.75 | 2.81 |
| University of St Andrews | 3.73 | 2.88 |
| University of Brighton | 3.73 | 2.57 |
| University of Bristol | 3.70 | 2.87 |
| University of Ulster | 3.67 | 2.82 |
| London School of Economics and Political Science | 3.64 | 3.29 |
| Bournemouth University | 3.63 | 2.37 |
| University of Sheffield | 3.60 | 2.91 |
Middle 10 institutions.
| Institution | Impact | Output |
|---|---|---|
| University of Kent | 3.12 | 2.80 |
| University of Exeter | 3.10 | 2.76 |
| University of Birmingham | 3.10 | 2.66 |
| Kingston University | 3.10 | 2.65 |
| University of Central Lancashire | 3.10 | 2.19 |
| Middlesex University | 3.10 | 2.51 |
| Robert Gordon University | 3.10 | 2.42 |
| University of Sussex | 3.08 | 2.74 |
| London Business School | 3.06 | 3.28 |
| Royal Holloway, University of London | 3.06 | 2.95 |
Bottom 10 institutions.
| Institution | Impact | Output |
|---|---|---|
| University of Westminster | 2.30 | 2.34 |
| University of Sunderland | 2.30 | 1.13 |
| Keele University | 2.23 | 2.46 |
| University of Surrey | 2.18 | 2.82 |
| University of Bedfordshire | 2.10 | 2.49 |
| University of Northampton | 2.00 | 1.81 |
| Teesside University | 1.90 | 2.48 |
| University of Worcester | 1.90 | 1.64 |
| University of Hertfordshire | 1.80 | 2.16 |
| University of Chester | 1.60 | 1.57 |
Factors influencing impactful research.
| Variable name | Description |
|---|---|
| Number of case studies | Number of impact case studies submitted by each institution. |
| Average number of outputs | Average number of underpinning research output listed to support each constitutional impact case study. |
| Percentage of journals | Percentage of the underpinning research outputs represented by journal articles |
| Average ABS score | Average Association of Business School 2012 (ABS2010) Quality Guide ranking of each journal in which underpinning research was published |
| Average listed grant (£k) | Average listed grant amounts for each institutional impact case study |
| Average number of key researchers/ Number of researchers | Average number of key researchers for each institutional impact case study |
| Average longest time in post (years) | Average length of service for longest serving key researcher for each institutional impact case study (i.e. the length of time a key researcher has been working in the institution prior to REF2014) |
| Percentage of women key researchers | Percentage of women key researchers for each institutional impact case study |
| Average percentage public of public interaction | Percentage of an institution’s case studies stemming from primary interaction with public and non-profit organisations |
| Average percentage national reach | Percentage of national reach which represents the proportion of an institution’s case studies where the impact was generated within one country only |
Sources: Kellard & Śliwa (2016) and Chowdhury et al. (2016)
Calibration thresholds for conditions.
| Calibration | Threshold | ||
|---|---|---|---|
| Factors | non-membership (0.05) | cross-over (0.5) | Full membership (0.95) |
| 1. Number of cases | 2 | 3 | 8 |
| 2. Average number of outputs | 4.5 | 5.475 | 6 |
| 3. Percentage of journals | 47 | 70.795 | 100 |
| 4. Average ABS score | 1.67 | 2.67 | 3.45 |
| 5. Average grant (£k) | 0 | 125.43 | 954.85 |
| 6. Average number of key researchers | 1 | 1.9 | 4.67 |
| 7. Average longest time in post (years) | 5 | 17.25 | 31.67 |
| 8. Percentage of women key researchers | 0 | 31.37 | 100 |
| 9. Average percentage public | 0 | 77.5 | 100 |
| 10. Average percentage national reach | 0 | 50 | 100 |
| 11. Impact | 1.8 | 3.09 | 3.8 |
Truth table for the outcome high impact score.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | Number of cases | Impact | Raw consist. | PRI consist. | SYM consist |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0.993711 | 0.96875 | 0.96875 |
| 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 0.992565 | 0.973684 | 0.973684 |
| 1 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 0.992395 | 0.962963 | 0.962963 |
| 0 | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0.991416 | 0.960784 | 0.960784 |
| 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 0.991379 | 0.958333 | 0.958333 |
| 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0.978903 | 0.166665 | 0.166665 |
| 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0.977848 | 0.925532 | 0.977528 |
| 0 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0.972549 | 0.916667 | 0.916667 |
| 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0.972308 | 0.871428 | 0.938462 |
| 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0.964444 | 0.619048 | 0.684211 |
| 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 0.951542 | 0.153846 | 0.153846 |
| 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0.9375 | 0.238095 | 0.294118 |
| 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0.734884 | 0.033898 | 0.033898 |
| 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0.728745 | 0.014706 | 0.014706 |
| 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 2 | 0 | 0.698171 | 0.282608 | 0.282608 |
| 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0.667897 | 0.021739 | 0.021739 |
| 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 0.662447 | 0.012346 | 0.012346 |
Note Numbers 1 to 10 in the first row correspond to the numbering of factors in Table above.
Parsimonious solution for high impact score.
| Solutions | Raw coverage | Consistency |
|---|---|---|
| Average listed grant k | 0.743539 | 0.781882 |
| No. cases*~average percentage national reach | 0.455268 | 0.823741 |
| Average longest time in post (years)*~average percentage national reach | 0.434725 | 0.786571 |
| No. cases*average no. of key researchers | 0.46786 | 0.948925 |
| No. cases*average longest time in post (years) | 0.432074 | 0.832695 |
| Solution coverage: | 0.851557 | |
| Solution consistency: | 0.723536 |
1The Economic and Research Council (ESRC) is one of the UKs major public funders of social science research, including studies related to Business and Management and the disciplines that are often located within Business Schools.
2According to HEFC ( http://www.ref.ac.uk/2014/media/ref/content/pub/REF%20Brief%20Guide%202014.pdf), Impact case studies comprised four-page documents which described impacts that had occurred between January 2008 and July 2013. The submitting university must have produced high quality research since 1993 that contributed to the impacts. Each submission included one case study, plus an additional case study for every 10 staff. An impact template was also submitted. This document explained how the submitted unit had enabled
impact from its research during the period from 2008 to 2013, and its future strategy for impact.
3What is notable is that few successful ICS’ can be observed in REF 2014 from UOA19 that went down this route.
4The sample was designed to replicate this study because we cannot access the score for each of the 410 case studies. Instead it adopts the same methodology as Kellard and Śliwa because they looked at the institutional level rather ICS level.
References
Aguinis, H., Ramani, RS, Alabduljader, N. et al., “A pluralist conceptualization of scholarly impact in management education: Students as stakeholders”, Acad Manag Perspect, (2019), Vol. 18 No. 1, pp. 11-42, doi: 10.5465/amle.2017.0488.
Aguinis, H., Suárez-González, I., Lannelongue, G. et al., “Scholarly Impact Revisited”, Academy of Management Perspectives, (2012), Vol. 26 No. 2, pp. 105-132, doi: 10.5465/amp.2011.0088.
Alajoutsijärvi, K., Juusola, K. and Siltaoja, M. “The legitimacy paradox of business schools: losing by gaining?” Academy of Management Learning & Education, (2015), Vol. 14 No. 2, pp. 277-291, doi: 10.5465/amle.2013.0106.
Amara, N. and Landry, R. “Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science”, Scientometrics, (2012), Vol. 93 pp. 553-581, doi: 10.1007/s11192-012-0729-2.
Amara, N., Halilem, N. and Traoré, N. “Adding value to companies' value chain: Role of business schools scholars”, J Bus Res, (2016), Vol. 69 No. 5, pp. 1661-1668, doi: 10.1016/j.jbusres.2015.10.035.
Antonacopoulou, EP “Making the Business School More ‘Critical’: Reflexive Critique Based on Phronesis as a Foundation for Impact”, British Journal of Management, (2010), Vol. 21 No. s1, pp. s6-s25, doi: 10.1111/j.1467-8551.2009.00679.x.
Azmat, G., Murphy, R., Valero, A. et al., “Universities and industrial strategy in the UK (No. 539)”, Centre for Economic Performance, LSE, (2018), available at: Reference Source.
Bager, T. “Knowledge exchange and management research: barriers and potentials”, European Business Review, (2018), Vol. 30 No. 2, pp. 169-182, doi: 10.1108/EBR-02-2017-0047.
Beer, M. “Why management research findings are unimplementable: An action science perspective”, Reflections: The Sol Journal, (2001), Vol. 2 No. 3, pp. 58-65. available at: Reference Source.
Bennis, W. and O’Toole, J. “How business schools have lost their way”, Harv Bus Rev, (2005), Vol. 83 No. 5, pp. 96-104. 15929407.
Birnik, A. and Billsberry, J. “Reorienting the business school agenda: The case for relevance, rigor, and righteousness”, J Bus Ethics, (2008), Vol. 82 No. 4, pp. 985-999, doi: 10.1007/s10551-007-9607-x.
Bornmann, L. and Marx, W. “How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons”, EMBO Rep, (2013), Vol. 14 No. 3, pp. 226-30. 23399654 doi: 10.1038/embor.2013.9, 3589094.
Bradshaw, D. “Rethinking business education: Fit for the future”, The Chartered Association of Business School. London, (2017), available at: Reference Source.
Caplan, B. “The Case Against Education”, Princeton University Press, (2018), available at: Reference Source.
Chowdhury, G., Koya, K. and Philipson, P. “Measuring the impact of research: Lessons from the UK’s Research Excellence Framework 2014”, PLoS One, (2016), Vol. 11 No. 6, p. e0156978. 27276219 doi: 10.1371/journal.pone.0156978, 4898824.
Cooke, A. and Galt, V. “The impact of business schools in the UK”, The Association of Business Schools. (2010).
Edwards, M. “The Oxford Handbook of Civil Society”, Oxford University Press, Oxford, (2013), available at: Reference Source.
Ekzkowitz, H. “The Triple Helix: University-Industry-Government Innovation in Action”, Routledge, New York, (2008), available at: Reference Source.
Elliott, T. “Fuzzy set qualitative comparative analysis”, Research Notes: Statistics Group, UCI, (2013), available at: Reference Source.
Fiss, P. “A set-theoretic approach to organizational configurations”, Acad Manage Rev, (2007), Vol. 32 No. 4, pp. 1180-1198, doi: 10.5465/amr.2007.26586092.
Garfield, E. “Citation indexing: Its Theory and Application in Science, Technology, and Humanities”, John Wiley, New York, (1979), available at: Reference Source.
Giddens, A. “The Third Way: The Renewal of Social Democracy”, Polity, Cambridge, (1998), available at: Reference Source.
Greenwood, R., Suddaby, R. and Hinings, C. “Theorizing Change: The Role of Professional Associations in the Transformation of Institutionalized Fields”, Acad Manage J, (2002), Vol. 45 No. 1, pp. 58-80, doi: 10.5465/3069285.
Hall, CM and Page, S. “Following the Impact Factor: Utilitarianism or academic compliance?” Tour Manag, (2015), Vol. 51 pp. 309-312, doi: 10.1016/j.tourman.2015.05.013.
Hamet, J. and Michel, S. “Rigor, relevance, and the knowledge “market”, European Business Review, (2018), Vol. 30 No. 2, pp. 183-201, doi: 10.1108/EBR-01-2017-0025.
Hardy, G. and Everett, D. “Shaping the Future of Business Education: Relevance”, (2013), doi: 10.1057/9781137033383.
Head, BW and Alford, J. “Wicked problems: Implications for public policy and management”, Administration & society, (2015), Vol. 47 No. 6, pp. 711-739, doi: 10.1177/0095399713481601.
Holmberg, K., Didegah, F. and Bowman, T. “The different meanings and levels of impact of altmetrics”, Accepted for oral presentation at the 11th International Conference on Webometrics, Informetrics and Scientometrics & 16th COLLNET Meeting, 26-28 November, New Delhi, India, (2015).
Hughes, T., Webber, D. and O’Reagan, N. “Achieving wider impact in business and management: Analysisng the case studies from REF2014”, Studies in Higher Education, (2017), Vol. 44 No. 3, pp. 1-15, doi: 10.1080/03075079.2017.1393059.
Irwin, A. “Re-making ‘quality’within the social sciences: The debate over rigour and relevance in the modern business school”, Sociol Rev, (2019), Vol. 67 No. 1, pp. 194-209, doi: 10.1177/0038026118782403.
Jackson, SE, Schuler, RS and Jiang, K. “An aspirational framework for strategic human resource management”, Acad Manag Ann, (2014), Vol. 8 No. 1, pp. 1-56, doi: 10.1080/19416520.2014.872335.
Johnson, S. and Orr, K. “What is business school research for? Academic and stakeholder perspectives, politics and relationality”, Stud High Educ, (2019), Vol. 45 No. 5, pp. 1-22, doi: 10.1080/03075079.2018.1564901.
Kapstein, M. and Yip, G. “The future of Business School research”, In Morsing, M. and Roviura, A. (Eds.): Business Schools and Contribution to Society, Sage, London, (2011) pp. 114-124.
Kellard, NM and Śliwa, M. “Business and Management Impact Assessment in Research Excellence Framework 2014: Analysis and Reflection”, Brit J Manage (2016), Vol. 27 No. 4, pp. 693-711, doi: 10.1111/1467-8551.12186.
Kelly, U. and McNicoll, I. “Through a glass, darkly: Measuring the social value of universities”, National Co-ordinating Centre for Public Engagement, Bristol, (2011), available at: Reference Source.
Kieser, A., Nicolai, A. and Seidl, D. “The practical relevance of management research: Turning the debate on relevance into a rigorous scientific research programme”, Acad Manag Ann, (2015), Vol. 9 No. 1, pp. 143-233, doi: 10.1080/19416520.2015.1011853.
Kieser, A. and Leiner, L. “Why the Rigour–Relevance Gap in Management Research Is Unbridgeable”, J Manage Stud, (2009), Vol. 46 No. 3, pp. 516-533, doi: 10.1111/j.1467-6486.2009.00831.x.
Lejeune, C., Starkey, K., Kalika, M. et al., “The Impact of Business Schools: Increasing the Range of Strategic Choices”, Management International/International Management/Gestión Internacional, (2019), Vol. 23 No. 2, pp. 88-98, doi: 10.7202/1060033AR.
Mingers, J. and Yang,, L. “Evaluating journal quality: A review of journal citation indicators and ranking in business and management”, Eur J Oper Res, (2017), Vol. 257 No. 1, pp. 323-337, doi: 10.1016/j.ejor.2016.07.058.
Morsing, M., Rovira, A. Business Schools and Their Contribution to Society, Sage, London, (2011), doi: 10.4135/9781446250822.
Murcia, MJ, Rocha, HO and Birkinshaw, J. “Business schools at the crossroads? A trip back from Sparta to Athens”, J Bus Ethics, (2018), Vol. 150 No. 2, pp. 579-591, doi: 10.1007/s10551-016-3129-3.
Narasimhan, R. “The fallacy of impact without relevance – reclaiming relevance and rigor”, European Business Review, (2018), Vol. 30 No. 2, pp. 157-168, doi: 10.1108/EBR-01-2017-0005.
Page, SJ, Hartwell, H., Johns, N. et al., “Case study: Wellness, tourism and small business development in a UK coastal resort: Public engagement in practice”, Tour Manag, (2017), Vol. 60 p. 466-477, doi: 10.1016/j.tourman.2016.12.014.
Parker, M. “Shut down the business school”, University of Chicago Press Economics Books Chicago, (2018), available at: Reference Source.
Pendlebury, D. “The use and misuse of journal metrics and other citation indicators”, Arch Immunol Ther Ex, (2009), Vol. 57 No. 1, pp. 1-11, doi: 10.1007/s00005-009-0008-y.
Penfield, T., Baker, MJ, Scoble, R. et al., “Assessment, evaluations, and definitions of research impact: A review”, Res Evaluat, (2014), Vol. 23 No. 1, pp. 21-32, doi: 10.1093/reseval/rvt021.
Pettigrew, A. and Starkey, K. “From the Guest Editors: The Legitimacy and Impact of Business Schools—Key Issues and a Research Agenda”, Acad Manag Learn Edu, (2016), Vol. 15 No. 4, pp. 649-664, doi: 10.5465/amle.2016.0296.
Phillips, PA, Moutinho, L. and Godinho, P. “Developing and testing a method to measure academic societal impact”, High Educ Q, (2018), Vol. 72 No. 2, pp. 121-140, doi: 10.1111/hequ.12154.
Phillips, PA, Subramanian, KV, Newman, V. Management Consultancy Through an Academic and Practitioner Perspective, Notionpress.com, India, (2019), available at: Reference Source.
Phillips, PA, Page, SJ and Sebu, J. “Achieving research impact in tourism: Modelling and evaluating outcomes from the UKs Research Excellence Framework”, Tour Manag, (2020), Vol. 78 p. 104072, doi: 10.1016/j.tourman.2019.104072.
Ragin, CC “Set relations in social research: Evaluating their consistency and coverage”, Polit Anal, (2006), Vol. 14 No. 3, pp. 291-310, doi: 10.1093/pan/mpj019.
Roig-Tierno, N., Huarng, KH and Ribeiro-Soriano, D. “Qualitative comparative analysis: Crisp and fuzzy sets in business and management”, J Bus Res, (2016), Vol. 69 No. 4, pp. 1261-1264, doi: 10.1016/j.jbusres.2015.10.089.
Rynes, SL and Bartunek, JM “Evidence-based management: Foundations, development, controversies and future”, Annu Rev Organ Psych, (2017), Vol. 4 pp. 235-261, doi: 10.1146/annurev-orgpsych-032516-113306.
Saltera, A., Saladrab, R. and Walker, J. “Exploring preferences for impact versus publications among UK business and management academics”, Res Policy, (2017), Vol. 46 No. 10, pp. 1769-1782, doi: 10.1016/j.respol.2017.08.005.
Seal, W. “Some proposals for impactful management control research”, Qualitative Research in Accounting Management, (2012), Vol. 9 No. 3, pp. 228-244, doi: 10.1108/11766091211257461.
Shapiro, DL, Kirkman, BL and Courtney, HG “Perceived causes and solutions of the translation problem in management research”, Acad Manage J, (2007), Vol. 50 No. 2, pp. 249-266, doi: 10.5465/amj.2007.24634433.
Snelson-Powell, AC, Grosvold, J. and Millington, AI “Organizational hypocrisy in business schools with sustainability commitments: The drivers of talk-action inconsistency”, J Bus Res, (2020), Vol. 114 pp. 408-420, doi: 10.1016/j.jbusres.2019.08.021.
Teagarden, MB, Von Glinow, MA and Mellahi, K. “Contextualizing international business research: Enhancing rigor and relevance”, J World Bus, (2018), Vol. 53 No. 3, pp. 303-306, doi: 10.1016/j.jwb.2017.09.001.
Tho, ND and Trang, NTM “Can knowledge be transferred from business schools to business organizations through in-service training students? SEM and fsQCA findings”, J Bus Res, (2015), Vol. 68 No. 6, pp. 1332-1340, doi: 10.1016/j.jbusres.2014.12.003.
Tho, ND “Knowledge transfer from business schools to business organizations: the roles absorptive capacity, learning motivation, acquired knowledge and job autonomy”, J Knowl Manag, (2017), Vol. 21 No. 5, pp. 1240-1253, doi: 10.1108/JKM-08-2016-0349.
Thomas, H. and Wilson, AD “‘Physics Envy’, Cognitive Legitimacy or Practical Relevance: Dilemmas in the Evolution of Management Research in the UK”, Brit J Manage, (2011), Vol. 22 No. 3, pp. 443-456, doi: 10.1111/j.1467-8551.2011.00766.x.
Trank, CQ “Reading” Evidence-Based Management: The Possibilities of Interpretation”, Acad Manag Learn Edu, (2014), Vol. 13 No. 3, pp. 381-395, doi: 10.5465/amle.2013.0244.
Valero, A. and Van Reenen, J. “The economic impact of universities: Evidence from across the globe”, Econ Educ Rev, (2019), Vol. 68 pp. 53-67, doi: 10.1016/j.econedurev.2018.09.001.
Van de Ven, AH “Nothing is quite so practical as a good theory”, Acad Manage Rev, (1989), Vol. 14 No. 4, pp. 486-489, doi: 10.5465/amr.1989.4308370.
Van de Ven, AH and Heath, V. “Engaged Scholarship: A Guide for Organizational and Social Research”, OUP Oxford, (2007), available at: Reference Source.
Watermeyer, R. “Impact in the REF: Issues and obstacles”, Stud High Educ, (2014), Vol. 41 No. 2, pp. 199-214, doi: 10.1080/03075079.2014.915303.
Weybrecht, G. “The Launch of a Blueprint for SDG Integration in Business Schools”, (2020), available at: Reference Source.