Perspectives on relevance: The relevance test in the constructive research approach

https://doi.org/10.1016/j.mar.2016.07.001Get rights and content

Abstract

Interventionist research (IVR), such as the constructive research approach (CRA), has been suggested as a method to improve the relevance of management accounting (MA) research. Although literature identifies several perspectives on relevance, the current assessment of CRA focuses on practical relevance. Moreover, an overreliance on pragmatism in assessing CRA research in the form of CRA market tests has been criticized. This article analyses the challenges inherent in conducting and assessing CRA research, both conceptually and with a CRA case example. In order to overcome these possible CRA challenges, we suggest analyzing CRA relevance from multiple perspectives. The perspectives in question are those of practical value relevance, legitimative decision relevance, academic value relevance, and instrumental decision relevance. Further, we suggest that indications of relevance in CRA studies can be analyzed during the research project. In particular, we introduce the relevance test as an explicit part of the CRA research process. We suggest a new tool, the Relevance Diamond would facilitate conducting the relevance test and aid the analysis of CRA relevance from multiple perspectives. Furthermore, we suggest new interpretations of what should constitute a pass in the CRA market tests under special circumstances, thereby contributing to CRA methodology, and especially to the analysis of relevance and ‘battlefields’ of different interests in CRA/IVR projects.

Introduction

Many scholars have called for accounting research that is relevant to practice (Johnson and Kaplan, 1987, Kasanen et al., 1993, Labro and Tuomela, 2003, Malmi and Granlund, 2009, Modell, 2014, Westin and Roberts, 2010). Malmi and Granlund (2009) suggest that a suitable way of creating both practical and theoretical relevance is to solve practical case problems alongside practitioners and theorize the findings using interventionist research (IVR) approaches, such as the constructive research approach (CRA) of Kasanen et al. (1993). In CRA, the researcher actively participates in the innovation of management accounting (MA) constructions (also called constructs), such as new accounting tools. According to the market test idea of Kasanen et al. (1993), the key pragmatic test of constructions is whether they work in practice, as exemplified by whether they are adopted, widely used, or create benefits (see also Jönsson and Lukka, 2005, Labro and Tuomela, 2003). When an organization adopts the construct created in the CRA process, a weak market test has been passed, suggesting that the construct has practical value. Stronger indications of practical usefulness require passing the semi-strong or strong market test, which respectively requires that the construct is used in other organizations, or that financial benefits of the use of the construct apply to multiple businesses.

The success or validity of the CRA method is largely based on the practical usefulness of the construction in an organization or beyond it. However, evaluating the success of CRA projects using the market tests can be challenging: it is not always clear what constitutes ‘usefulness’ or ‘working in practice’ (see Labro and Tuomela, 2003, Lukka, 2000, Piirainen and Gonzalez, 2013). Piirainen and Gonzalez (2013) note that the use of a construction may be a problematic measure of success because the adoption of the construction does not actually measure its qualities. For example, sub-optimal constructions may be adopted purely to confer legitimacy. In addition, constructions adopted but soon abandoned may be difficult to assess in terms of relevance, which, conceptually, is not an unambiguous term (see Lukka and Suomala, 2014).

We suggest that CRA constructions might be analyzed from several relevance perspectives, and also based on the potential relevance and relevance over time. IVR projects tend to involve conflicts between various interest groups, such as practitioners and academics, who often have different views and opinions (e.g., Suomala et al., 2014). This suggests that several, even contradictory, perspectives on the relevance of CRA constructs may emerge during the research process. The perspectives on relevance suggested in earlier research include practical and theoretical (or academic) relevance, alongside several other categories, such as value relevance, decision relevance as well as societal, instrumental and ‘legitimative’ relevance (see Barth et al., 2001, Järvinen, 2006, Lukka and Suomala, 2014, Nicolai and Seidl, 2010).

Although market tests for practical relevance are presented in CRA literature, clear testing rules for other perspectives on relevance, such as the academic relevance of CRA contributions, have not been articulated. This study presents a way to analyze multiple perspectives on relevance in CRA research, called the relevance test (including ex-ante analysis, as visualized in the Relevance Diamond tool, see Appendix A). Considering several perspectives on relevance already during the CRA process might increase the theoretical orientation and the publication potential of CRA research (see Lukka and Suomala, 2014). Explicit analysis of relevance perspectives might also assist the CRA researcher to understand and mitigate the effect of possible conflicts over research direction.

It is challenging to evaluate the validity, impact, or relevance of a CRA construction over a short time span because the wider societal or theoretical impacts are only revealed after a substantial period of time (see Jönsson and Lukka, 2005, Labro and Tuomela, 2003, Van der Stede, 2012). Recent value relevance, decision relevance, societal relevance and legitimative relevance discussion establish that the adoption of a new construction does not necessarily generate much value or increase legitimacy in the eyes of various stakeholder groups (Barth et al., 2001, Lukka and Suomala, 2014, Nicolai and Seidl, 2010, Piirainen and Gonzalez, 2013). In IVR, Jönsson and Lukka (2005) separate the ex-ante consideration (such as how a construct is supposed to work) and the ex-post consideration (the actual outcomes and theoretical research contribution). However, the CRA process steps do not explicitly include the analysis of relevance over time or from multiple perspectives during the CRA process. Thus, an aim of this article is to contribute to the analysis of the relevance of CRA constructs over time, that is, both ex-ante and ex-post.

Practical and theoretical relevance perspectives in the IVR context can be complementary (Lukka and Suomala, 2014, Suomala et al., 2014). Suomala et al. (2014) noted that compromises are often required during an IVR process to resolve tensions among different interest groups. This suggests that changes in relevance perceptions (and in circumstances) can occur during the research project, and the eventual CRA construction (and its relevance) may be something other than was anticipated. Thus, while acknowledging the value of CRA, we see the potential to amend the current CRA methodology in assessing CRA relevance over time and in developing the market tests. These amendments offer an opportunity to contribute to both accounting theory and practice, especially under circumstances, where conducting the classic CRA market tests can be challenging (e.g., amidst changing stakeholder views or in the non-profit sector).

This article analyses CRA methodology and provides tools for the assessment of the CRA contribution and the relevance of CRA. We suggest amendments to the ways in which CRA research is conducted. Accordingly, our research questions are: How can the assessment of CRA research (e.g., the market tests) be developed further? How can the relevance of CRA research be assessed during the CRA process while considering multiple relevance perspectives?

Both conceptual analysis and empirical analysis of the CRA research by Sippola (2008) are used to prompt amendments to the CRA market tests established by Kasanen et al. (1993) and Labro and Tuomela (2003). Analysis of multiple perspectives of relevance and the amended market tests are expected to improve evaluation of CRA projects, and boost the theoretical rigor of the CRA methodology, particularly in special circumstances, such as in fast changing business conditions where the classic (ex-post) CRA market tests may not be fully applicable. Fulfilling the conditions of the semi-strong and strong market tests (Kasanen et al., 1993) is not necessarily in the interests of the CRA case organization, which might prefer to block the wider use of a construction so as to maintain competitive advantage. Therefore the academic interests of disseminating information and practical efforts of value creation may conflict in a CRA project, and require consideration if CRA research is to be published. Generally, our discussion touches on the methodology, impact, and relevance of MA case research.

This article starts with a conceptual analysis of relevance and continues with an illustration of an earlier CRA study. In methodological terms, this resembles the approach of developing a case research process by learning from earlier empirical case projects (Ahrens and Dent, 1998, Labro and Tuomela, 2003, Lukka and Suomala, 2014, Suomala et al., 2014, Von Zedtwitz, 2002). This approach may also be referred to as design science, in which a research design is improved with new rules and solutions (Van Aken, 2005). Our empirical data are based on the empirical qualitative material (extensive field notes of the interviews and project meetings) collected during a CRA case study (Sippola, 2008), in which a quality cost measurement system was constructed in a software company. The relevance test introduced in this article could be a one-off event, but in longer projects or after significant project changes, the test might be repeated. The relevance test could also facilitate evaluating the relevance of other IVR research types. By noting the potentially conflicting and multilayered nature of the different perspectives on relevance in the case setting, we also contribute to the recent analysis of potential conflicts of interests in IVR research (called battlefields by Suomala et al., 2014).

Section snippets

Relevance in accounting research

Extant research presents several aspects of relevance: value, theoretical, decision, societal and legitimative relevance (Barth et al., 2001, Lukka and Suomala, 2014, Nicolai and Seidl, 2010). In the IVR context, Lukka and Suomala (2014) highlight the importance of practical, theoretical and societal relevance. Lukka and Suomala (2014) define relevance as “something that is of significance for something else” but note that relevance is often connected with values and usefulness. Lukka and

The constructive research approach (CRA)

Kasanen et al. (1993) devised the CRA to encourage MA academics to take a more active role in improving existing practice. According to Lukka, 2000, Lukka, 2005, the ideal CRA outcome is a solution to the original case problem that works in practice, and has both high practical and theoretical value. Therefore, the value of the construct is an important issue but the guidelines for estimating value relevance in the CRA process might benefit from greater elaboration, for example, in the case of

CRA case backgrounds

This study uses a CRA dissertation project (Sippola, 2008) to illustrate the different relevance perspectives and the challenges of CRA market testing under changing conditions. Sippola (2008) presented the idea of measuring quality costs in real-time (referred to as the real-time quality cost model or RQC) for a case company that develops and produces packaged software (SW) used in embedded products such as devices for public transport payment in buses, trains, ferries, and subways. The case

Discussion and conclusions

This article discusses the relevance of IVR research, particularly the relevance of the CRA. We suggest amendments to the assessment of the relevance of CRA. As far as we know, the ex-ante assessment of relevance and the interplay of the multiple perspectives on relevance have not been considered together in MA literature. In this article, relevance was first divided into value relevance and decision relevance. More specifically we categorized relevance into: (1) practical value relevance; (2)

Acknowledgements

We acknowledge the valuable contribution of the following researchers and scholars: Jukka Pellinen, Marko Järvenpää, Bob Scapens, Hanno Roberts, Eija Vinnari, Sven Modell, Jodie Moll, Nora Munoz-Izquierdo, Maria del Mar Camacho and the two anonymous reviewers. We also thank the audiences at the ENROAC Jyväskylä conference 2013, Manchester Business School Research Seminar (February 2014), EAA 2014 Tallinn Congress and at the Universidad Complutense Madrid Research Seminar (November 2014) for

References (58)

  • H. Nørreklit

    The Balanced Scorecard: what is the score? A rhetorical analysis of the Balanced Scorecard

    Account. Organ. Soc.

    (2003)
  • R.W. Scapens

    Never mind the gap: towards institutional perspective on management accounting practice

    Manag. Account. Res.

    (1994)
  • P. Suomala et al.

    Battlefield around interventions: a reflective analysis of conducting interventionist research in management accounting

    Manag. Account. Res.

    (2014)
  • T. Ahrens et al.

    Accounting and organizations: realizing the richness of field research

    J. Manag. Account. Res.

    (1998)
  • M. Alvesson et al.

    Generating research questions through problematization

    Acad. Manag. Rev.

    (2011)
  • M. Alvesson et al.

    Habitat and habitus: boxed-in versus box-breaking Research

    Organ. Stud.

    (2014)
  • M. Alvesson

    Do we have something to say? From re-search to roi-search and back again

    Organization

    (2012)
  • E. Amir et al.

    A comparison of the value-relevance of U.S. versus Non-U.S. GAAP accounting measures using form 20-F reconciliations

    J. Account. Res.

    (1993)
  • R. Balakrishnan et al.

    Financial benefits from JIT adoption: effects of customer concentration and cost structure

    Account. Rev.

    (1996)
  • M. Bolton et al.

    Ties that do not bind: musings on the specious relevance of academic research

    Public Adm. Rev.

    (2003)
  • P. Chiwamit et al.

    The societal relevance of management accounting innovations: economic value added and institutional work in the fields of Chinese and Thai state-owned enterprises

    Account. Bus. Res.

    (2014)
  • P. Darke et al.

    Successfully completing case study research: combining rigour, relevance and pragmatism

    Inf. Syst. J.

    (1998)
  • P.J. DiMaggio et al.

    The iron cage revisited: institutional isomorphism and collective rationality in organizational fields

    Am. Sociol. Rev.

    (1983)
  • G.S. Easton et al.

    The effects of total quality management on corporate performance: an empirical investigation

    J. Bus.

    (1998)
  • B. Flyvbjerg

    Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again

    (2001)
  • K. Golden-Biddle et al.

    Appealing work: an investigation of how ethnographic texts convince

    Organ. Sci.

    (1993)
  • N. Hanley et al.

    Estimating the monetary value of health care: lessons from environmental economics

    Health Econ.

    (2003)
  • C. Humphrey et al.

    Ac)counting research. The value of holistic understanding

  • T. Hyvönen et al.

    Contract-based budgeting in health care: a study of the institutional processes of accounting change

    Eur. Account. Rev.

    (2006)
  • Cited by (9)

    • Customer-driven menu analysis (CDMA): Capturing customer voice in menu management

      2020, International Journal of Hospitality Management
      Citation Excerpt :

      The study was also exploratory research with limited generalisability. Because of time constraints, the actual fieldwork phase of the CRA process was completed over a relatively short period, which makes it difficult to evaluate relevance and especially the ‘potential future relevance’ of the new MA construct (Rautiainen et al., 2017). The customer preferences were based on general DEVAs and therefore did not allow for a restaurant-specific approach, where the attributes would be determined by the actual customers of the restaurant.

    • Waste analysis framework for lean production: Pilot study in a healthcare setting

      2021, International Journal of Productivity and Quality Management
    View all citing articles on Scopus
    View full text