Perspectives on relevance: The relevance test in the constructive research approach
Introduction
Many scholars have called for accounting research that is relevant to practice (Johnson and Kaplan, 1987, Kasanen et al., 1993, Labro and Tuomela, 2003, Malmi and Granlund, 2009, Modell, 2014, Westin and Roberts, 2010). Malmi and Granlund (2009) suggest that a suitable way of creating both practical and theoretical relevance is to solve practical case problems alongside practitioners and theorize the findings using interventionist research (IVR) approaches, such as the constructive research approach (CRA) of Kasanen et al. (1993). In CRA, the researcher actively participates in the innovation of management accounting (MA) constructions (also called constructs), such as new accounting tools. According to the market test idea of Kasanen et al. (1993), the key pragmatic test of constructions is whether they work in practice, as exemplified by whether they are adopted, widely used, or create benefits (see also Jönsson and Lukka, 2005, Labro and Tuomela, 2003). When an organization adopts the construct created in the CRA process, a weak market test has been passed, suggesting that the construct has practical value. Stronger indications of practical usefulness require passing the semi-strong or strong market test, which respectively requires that the construct is used in other organizations, or that financial benefits of the use of the construct apply to multiple businesses.
The success or validity of the CRA method is largely based on the practical usefulness of the construction in an organization or beyond it. However, evaluating the success of CRA projects using the market tests can be challenging: it is not always clear what constitutes ‘usefulness’ or ‘working in practice’ (see Labro and Tuomela, 2003, Lukka, 2000, Piirainen and Gonzalez, 2013). Piirainen and Gonzalez (2013) note that the use of a construction may be a problematic measure of success because the adoption of the construction does not actually measure its qualities. For example, sub-optimal constructions may be adopted purely to confer legitimacy. In addition, constructions adopted but soon abandoned may be difficult to assess in terms of relevance, which, conceptually, is not an unambiguous term (see Lukka and Suomala, 2014).
We suggest that CRA constructions might be analyzed from several relevance perspectives, and also based on the potential relevance and relevance over time. IVR projects tend to involve conflicts between various interest groups, such as practitioners and academics, who often have different views and opinions (e.g., Suomala et al., 2014). This suggests that several, even contradictory, perspectives on the relevance of CRA constructs may emerge during the research process. The perspectives on relevance suggested in earlier research include practical and theoretical (or academic) relevance, alongside several other categories, such as value relevance, decision relevance as well as societal, instrumental and ‘legitimative’ relevance (see Barth et al., 2001, Järvinen, 2006, Lukka and Suomala, 2014, Nicolai and Seidl, 2010).
Although market tests for practical relevance are presented in CRA literature, clear testing rules for other perspectives on relevance, such as the academic relevance of CRA contributions, have not been articulated. This study presents a way to analyze multiple perspectives on relevance in CRA research, called the relevance test (including ex-ante analysis, as visualized in the Relevance Diamond tool, see Appendix A). Considering several perspectives on relevance already during the CRA process might increase the theoretical orientation and the publication potential of CRA research (see Lukka and Suomala, 2014). Explicit analysis of relevance perspectives might also assist the CRA researcher to understand and mitigate the effect of possible conflicts over research direction.
It is challenging to evaluate the validity, impact, or relevance of a CRA construction over a short time span because the wider societal or theoretical impacts are only revealed after a substantial period of time (see Jönsson and Lukka, 2005, Labro and Tuomela, 2003, Van der Stede, 2012). Recent value relevance, decision relevance, societal relevance and legitimative relevance discussion establish that the adoption of a new construction does not necessarily generate much value or increase legitimacy in the eyes of various stakeholder groups (Barth et al., 2001, Lukka and Suomala, 2014, Nicolai and Seidl, 2010, Piirainen and Gonzalez, 2013). In IVR, Jönsson and Lukka (2005) separate the ex-ante consideration (such as how a construct is supposed to work) and the ex-post consideration (the actual outcomes and theoretical research contribution). However, the CRA process steps do not explicitly include the analysis of relevance over time or from multiple perspectives during the CRA process. Thus, an aim of this article is to contribute to the analysis of the relevance of CRA constructs over time, that is, both ex-ante and ex-post.
Practical and theoretical relevance perspectives in the IVR context can be complementary (Lukka and Suomala, 2014, Suomala et al., 2014). Suomala et al. (2014) noted that compromises are often required during an IVR process to resolve tensions among different interest groups. This suggests that changes in relevance perceptions (and in circumstances) can occur during the research project, and the eventual CRA construction (and its relevance) may be something other than was anticipated. Thus, while acknowledging the value of CRA, we see the potential to amend the current CRA methodology in assessing CRA relevance over time and in developing the market tests. These amendments offer an opportunity to contribute to both accounting theory and practice, especially under circumstances, where conducting the classic CRA market tests can be challenging (e.g., amidst changing stakeholder views or in the non-profit sector).
This article analyses CRA methodology and provides tools for the assessment of the CRA contribution and the relevance of CRA. We suggest amendments to the ways in which CRA research is conducted. Accordingly, our research questions are: How can the assessment of CRA research (e.g., the market tests) be developed further? How can the relevance of CRA research be assessed during the CRA process while considering multiple relevance perspectives?
Both conceptual analysis and empirical analysis of the CRA research by Sippola (2008) are used to prompt amendments to the CRA market tests established by Kasanen et al. (1993) and Labro and Tuomela (2003). Analysis of multiple perspectives of relevance and the amended market tests are expected to improve evaluation of CRA projects, and boost the theoretical rigor of the CRA methodology, particularly in special circumstances, such as in fast changing business conditions where the classic (ex-post) CRA market tests may not be fully applicable. Fulfilling the conditions of the semi-strong and strong market tests (Kasanen et al., 1993) is not necessarily in the interests of the CRA case organization, which might prefer to block the wider use of a construction so as to maintain competitive advantage. Therefore the academic interests of disseminating information and practical efforts of value creation may conflict in a CRA project, and require consideration if CRA research is to be published. Generally, our discussion touches on the methodology, impact, and relevance of MA case research.
This article starts with a conceptual analysis of relevance and continues with an illustration of an earlier CRA study. In methodological terms, this resembles the approach of developing a case research process by learning from earlier empirical case projects (Ahrens and Dent, 1998, Labro and Tuomela, 2003, Lukka and Suomala, 2014, Suomala et al., 2014, Von Zedtwitz, 2002). This approach may also be referred to as design science, in which a research design is improved with new rules and solutions (Van Aken, 2005). Our empirical data are based on the empirical qualitative material (extensive field notes of the interviews and project meetings) collected during a CRA case study (Sippola, 2008), in which a quality cost measurement system was constructed in a software company. The relevance test introduced in this article could be a one-off event, but in longer projects or after significant project changes, the test might be repeated. The relevance test could also facilitate evaluating the relevance of other IVR research types. By noting the potentially conflicting and multilayered nature of the different perspectives on relevance in the case setting, we also contribute to the recent analysis of potential conflicts of interests in IVR research (called battlefields by Suomala et al., 2014).
Section snippets
Relevance in accounting research
Extant research presents several aspects of relevance: value, theoretical, decision, societal and legitimative relevance (Barth et al., 2001, Lukka and Suomala, 2014, Nicolai and Seidl, 2010). In the IVR context, Lukka and Suomala (2014) highlight the importance of practical, theoretical and societal relevance. Lukka and Suomala (2014) define relevance as “something that is of significance for something else” but note that relevance is often connected with values and usefulness. Lukka and
The constructive research approach (CRA)
Kasanen et al. (1993) devised the CRA to encourage MA academics to take a more active role in improving existing practice. According to Lukka, 2000, Lukka, 2005, the ideal CRA outcome is a solution to the original case problem that works in practice, and has both high practical and theoretical value. Therefore, the value of the construct is an important issue but the guidelines for estimating value relevance in the CRA process might benefit from greater elaboration, for example, in the case of
CRA case backgrounds
This study uses a CRA dissertation project (Sippola, 2008) to illustrate the different relevance perspectives and the challenges of CRA market testing under changing conditions. Sippola (2008) presented the idea of measuring quality costs in real-time (referred to as the real-time quality cost model or RQC) for a case company that develops and produces packaged software (SW) used in embedded products such as devices for public transport payment in buses, trains, ferries, and subways. The case
Discussion and conclusions
This article discusses the relevance of IVR research, particularly the relevance of the CRA. We suggest amendments to the assessment of the relevance of CRA. As far as we know, the ex-ante assessment of relevance and the interplay of the multiple perspectives on relevance have not been considered together in MA literature. In this article, relevance was first divided into value relevance and decision relevance. More specifically we categorized relevance into: (1) practical value relevance; (2)
Acknowledgements
We acknowledge the valuable contribution of the following researchers and scholars: Jukka Pellinen, Marko Järvenpää, Bob Scapens, Hanno Roberts, Eija Vinnari, Sven Modell, Jodie Moll, Nora Munoz-Izquierdo, Maria del Mar Camacho and the two anonymous reviewers. We also thank the audiences at the ENROAC Jyväskylä conference 2013, Manchester Business School Research Seminar (February 2014), EAA 2014 Tallinn Congress and at the Universidad Complutense Madrid Research Seminar (November 2014) for
References (58)
- et al.
The relevance of the value relevance literature for financial accounting standard setter: another view
J. Account. Econ.
(2001) - et al.
The roles of accounting in organizations and society
Account. Organ. Soc.
(1980) - et al.
Conceptualizing management accounting change: an institutional framework
Manag. Account. Res.
(2000) - et al.
Diverging views on the boundaries of management accounting
Manag. Account. Res.
(1990) Towards explaining stability in and around management accounting systems
Manag. Account. Res.
(2001)Financial accounting: in communicating reality, we construct reality
Account. Organ. Soc.
(1988)- et al.
The impact of just-in-time manufacturing on firm performance in the US
J. Oper. Manag.
(1995) Exploring management accounting change: the case of balanced scorecard implementation
Manag. Account. Res.
(2002)Management accounting change and stability: loosely coupled rules and routines in action
Manag. Account. Res.
(2007)Towards explaining activity-based costing failure: accounting and control in a decentralized organization
Manag. Account. Res.
(1997)
The Balanced Scorecard: what is the score? A rhetorical analysis of the Balanced Scorecard
Account. Organ. Soc.
Never mind the gap: towards institutional perspective on management accounting practice
Manag. Account. Res.
Battlefield around interventions: a reflective analysis of conducting interventionist research in management accounting
Manag. Account. Res.
Accounting and organizations: realizing the richness of field research
J. Manag. Account. Res.
Generating research questions through problematization
Acad. Manag. Rev.
Habitat and habitus: boxed-in versus box-breaking Research
Organ. Stud.
Do we have something to say? From re-search to roi-search and back again
Organization
A comparison of the value-relevance of U.S. versus Non-U.S. GAAP accounting measures using form 20-F reconciliations
J. Account. Res.
Financial benefits from JIT adoption: effects of customer concentration and cost structure
Account. Rev.
Ties that do not bind: musings on the specious relevance of academic research
Public Adm. Rev.
The societal relevance of management accounting innovations: economic value added and institutional work in the fields of Chinese and Thai state-owned enterprises
Account. Bus. Res.
Successfully completing case study research: combining rigour, relevance and pragmatism
Inf. Syst. J.
The iron cage revisited: institutional isomorphism and collective rationality in organizational fields
Am. Sociol. Rev.
The effects of total quality management on corporate performance: an empirical investigation
J. Bus.
Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again
Appealing work: an investigation of how ethnographic texts convince
Organ. Sci.
Estimating the monetary value of health care: lessons from environmental economics
Health Econ.
Ac)counting research. The value of holistic understanding
Contract-based budgeting in health care: a study of the institutional processes of accounting change
Eur. Account. Rev.
Cited by (9)
Towards interventionist research with theoretical ambition
2022, Management Accounting ResearchCustomer-driven menu analysis (CDMA): Capturing customer voice in menu management
2020, International Journal of Hospitality ManagementCitation Excerpt :The study was also exploratory research with limited generalisability. Because of time constraints, the actual fieldwork phase of the CRA process was completed over a relatively short period, which makes it difficult to evaluate relevance and especially the ‘potential future relevance’ of the new MA construct (Rautiainen et al., 2017). The customer preferences were based on general DEVAs and therefore did not allow for a restaurant-specific approach, where the attributes would be determined by the actual customers of the restaurant.
An exposition of the constructive research approach: a tactical treatise for addressing methodological and practical issues in organisational research
2023, International Journal of Organizational AnalysisWaste analysis framework for lean production: Pilot study in a healthcare setting
2021, International Journal of Productivity and Quality ManagementLegitimacy and relevance of a performance measurement system in a Finnish public-sector case
2020, Qualitative Research in Accounting and Management