“What you measure is what you will get”?: Exploring the effectiveness of marketing performance measurement practices

Marketing accountability and the measurement of marketing performance have long been reported as a significant challenge for management globally. To address the increasing demand for marketing accountability, companies across the world are investing resources in developing and improving their marketing performance measurement (MPM) practices. MPM practices in Irish firms were reported to be less well developed than those reported in other countries. This study aims to investigate if and how MPM practices have changed since then, by means of a comparative analysis which contrasts secondary data collected by O'Sullivan in 2007 and primary data collected in 2015. This study also seeks to examine the impact of MPM practices on firms’ marketing capabilities and firm performance. Based on survey data collected from 210 Irish managers, this study finds that companies adopting comprehensive MPM practices tend to develop enhanced marketing capabilities and achieve better firm performance than those adopting less comprehensive MPM practices. Subjects: Information / Knowledge Management; Strategic Management; Marketing Management


ABOUT THE AUTHORS
Dr Xiaoning Liang is an assistant professor in Marketing at Trinity Business School, Trinity College Dublin, the University of Dublin. Her research interests lie in exploring how marketing performance measurement practices can be effectively deployed to influence firm performance, and understanding how companies can develop marketing analytical capability.
Dr Yuhui Gao is a senior lecturer in Marketing at DCU Business School, Dublin City University. Her current research interests are marketing performance management (marketing metrics), marketing research, and firm leaders' personal values and their relevance to marketing strategy.
Dr Qing Shan Ding is a senior lecturer in Marketing at Huddersfield Business School, University of Huddersfield. His research interests are market entry strategies, consumer perceptions, and brands in emerging economies.

PUBLIC INTEREST STATEMENT
Driven by an increasing demand for marketing accountability, companies across the world are investing resources in developing and improving their comprehensive marketing performance measurement (MPM) practices. One premise underlying this growing interest is that a comprehensive MPM practice allows firms to perform better. This study aims to empirically investigate into this premise. Based on secondary data published in 2007 and primary data collected from Irish firms in 2015, this study finds that companies adopting more comprehensive MPM practices perform better with regard to their marketing capabilities and firm performance than those with less comprehensive MPM practices. The findings suggest companies should adopt comprehensive MPM practices to enhance marketing capabilities and improve firm performance.

Introduction
Marketing performance measurement is a business process "that provides performance feedback to the organisation regarding the results of marketing efforts" (Clark, Abela, & Ambler, 2006, p. 191). Both marketing academics and practitioners have considered the measurement of marketing performance to be a significant challenge for management (Frösén, Tikkanen, Jaakkola, & Vassinen, 2013;Homburg, Artz, & Wieseke, 2012). The Marketing Science Institute (MSI), representing the perspective of marketing academics, for instance, has continuously advocated research on marketing metrics, marketing performance measurement, and marketing accountability (Marketing Science Institute [MSI], 2002[MSI], , 2004[MSI], , 2006[MSI], , 2008[MSI], , 2012. The Chief Marketing Officer (CMO) Council, representing the perspective of practitioners, undertook a large-scale survey and found that 80% of the CMOs surveyed were dissatisfied with their ability to measure marketing performance (CMO Council, 2009). Marketing performance measurement issues have become more challenging due to increasing global competition and the increasingly dynamic business environment. A 2014 industry study reported that 85% of the marketing managers surveyed indicated an increase in the pressure to measure marketing performance (Schwartz, Kim, & Patterson, 2014).
Driven by increasing demands for greater marketing accountability, companies have invested considerable resources in developing and improving their MPM practices (Rust, Ambler, Carpenter, & Kumar, 2004). Such investments are evidenced by the increased use of multiple marketing metrics, such as marketing dashboards (Pauwels et al., 2009) and balanced scorecards (Kaplan & Norton, 1996). However, there is a heated debate on the impact of MPM practices on firm performance (Ittner, Larcker, & Meyer, 2003). Some scholars are of the view that such systems are merely a management fashion without any practical implication for the firm (Voelpel, Leibold, & Eckhoff, 2006). Others, however, believe that MPM practices are beneficial to companies (Clark et al., 2006;O'Sullivan & Abela, 2007). New evidence showed that companies adopting multiple marketing metrics outperformed others with regard to profitability, revenue growth and market share (CMO Council, 2009).
Contributing to this debate, O'Sullivan (2007) examined the marketing performance measurement practices in Irish firms. Based on survey data from 209 Irish marketing managers, he pointed out that MPM practices were positively associated with firm performance. He also noted that Irish firms had less developed MPM practices than those reported in Spain or the United Kingdom (O'Sullivan, 2007). He thus recommended that Irish companies devote resources to improving MPM practices to boost firm performance. This study seeks to establish the extent to which Irish companies have responded to O'Sullivan's (2007) recommendations. It is hoped that this investigation will not only provide an overview of current practices but also seek to establish whether there is a relationship between MPM practices and marketing capabilities. It is hoped that this latter examination provides additional empirical support for the positive role of MPM practice in enhancing marketing capabilities and improving firm performance. O'Sullivan (2007) reported that an over reliance on financial metrics (e.g., sales and profitability) and less frequent tracking and reporting of marketing performance by Irish firms hindered their development of better MPM practices. He suggested that Irish firms should improve their MPM practices in order to survive and succeed in an intensely competitive global marketplace. This study aims to provide insights into how MPM practices have evolved in Irish firms in the intervillage years since O'Sullivan's study was undertaken.

Research objectives
Some scholars have suggested that the reasons why companies are interested in adopting comprehensive MPM practices is because comprehensive MPM practices can drive firm performance through improved organisational learning, market orientation, and decision-making (Homburg et al., 2012;O'Sullivan, Abela, & Hutchinson, 2009). However, empirical studies have mainly supported the positive connection between MPM practices and firm performance (e.g., Homburg et al., 2012;O'Sullivan, 2007;O'Sullivan & Abela, 2007). The hypothesised relationship between MPM practices and drivers of firm performance, for example organisational learning and decision-making, have rarely been examined. An assessment of the relationship is warranted given that management practices do not influence firm performance directly. These practices influence drivers of firm performance (e.g., organisational learning, strategy implementation), thus ultimately influencing firm performance. Amongst a litany of drivers of firm performance, marketing capabilities are considered to be important drivers of firm performance (Vorhies, Orr, & Bush, 2011). Therefore, this study aims to explore the relationship between MPM practices and different marketing capabilities.
This study aims to explore the relationship between MPM practices and these marketing capabilities. To be specific, this study aims to provide an insight into the following questions: (1) What key marketing metrics have been used by Irish firms,

Methodology
A quantitative research method was utilised to examine the research questions. A survey was sent to marketing (senior) managers across Ireland in 2015. All measures in the survey were adopted from previous studies, and were pilot-tested on 6 academics and 11 practitioners to ensure clarity, logic flow, MPM "MPM is a business process that provides performance feedback to the organisation regarding the results of marketing efforts". (Clark et al., 2006, p. 191)

Comprehensiveness of MPM practices
The diversity of marketing performance measures used to align with marketing strategy and to reflect the cause-and-effect relationships between marketing activities and firm performance (Chenhall, 2005;Hall, 2008;Homburg et al., 2012) Architectural marketing capability Planning-related capabilities that are involved in selecting strategic marketing objectives, and implementing-related capabilities that are associated with the achievement of marketing strategies (Slotegraaf & Dickson, 2004;Vorhies & Morgan, 2005) Market-linking capability Firms' ability to sense, detect and anticipate market changes, create and maintain durable relationships with customers, and create and retain strong connections with channel members (Day, 1994;Song et al., 2007) Market-focused learning capability "The capability of the firm to acquire, disseminate, unlearn and use market information for organisational change" (Weerawardena, O'Cass, & Julian, 2006, p. 39) Relationally-focused learning capability "The capacity and extent an organisation acquires knowledge through external linkages or networks, and disseminates, unlearns, and uses such knowledge for organisational change" (Weerawardena et al., 2006, p.39) Adaptive marketing capability Firm's ability to "reconfigure resources and coordinate processes promptly and effectively to meet rapid environmental changes" (Zhou & Li, 2010, p. 225) and relevance. The survey was modified based on the feedback received from these academics and practitioners.

Marketing performance measurement practice
Regarding the key marketing metrics used in Irish firms, respondents were asked to identify which marketing metrics their companies used from a list of 47 key marketing metrics. The list of 47 marketing metrics was adopted from Frösén et al. (2013). Six key marketing metric categories were selected from Ambler, Kokkinaki, and Puntoni (2004) and the digital metric category was added based on the interviews with practitioners. Then the frequency of tracking these seven sets of marketing metrics was measured on a five-point Likert scale (1 = never, 5 = monthly or more frequently). The importance of these marketing metric categories to senior management team was also measured on a five-point Likert scale (1 = not at all important, 5 = very important). Twelve items were adopted from Homburg et al. (2012) and Burney and Widener (2013) to measure the comprehensiveness of MPM practices. The Cronbach's alpha was .88, indicating a good internal reliability.

Market-linking capability
In line with Song et al. (2007), seven widely used items were utilised to measure market-linking capability. In addition, in order to fully capture firms' ability to link with the broader market environment, three items were borrowed from Morgan, Slotegraaf, and Vorhies (2009). The 10 items were analysed together for their reliability as an indicator of market-linking capability. A Cronbach's alpha of .94 indicated a good internal reliability.

Architectural marketing capability
Nine widely used items were adopted from Morgan, Zou, Vorhies, and Katsikeas (2003), Vorhies and Morgan (2005), and Chang et al. (2010) to measure marketing planning capability and implementation capability. Respondents were asked to indicate how their firm performed in terms of marketing planning skills and implementation skills, relative to their major competitors. The Cronbach's alpha for architectural marketing capability was .93.

Relationally-/market-focused learning capability
Fifteen items were initially chosen from O'Cass and Weerawardena (2010) and Weerawardena (2003) to measure relationally and market-focused learning capability. During the pilot testing process, two items were dropped due to their potential to cause confusion. This left seven items to measure relationally-focused learning capability (Cronbach's alpha = .89) and six items to measure market-focused learning capability (Cronbach's alpha = .86).

Adaptive marketing capability
Adaptive marketing capability refers to firm's ability to sense and respond to the external environment in a swift way (Wang & Ahmed, 2007). To measure adaptive marketing capability, six items were adopted from Ma, Yao, andXi (2009) andAkgün, Keskin, andByrne (2012). A Cronbach's alpha of .92 indicated a good reliability.

Firm performance
Firm performance was measured using subjective measures, which were in line with previous studies (Verhoef & Leeflang, 2009;Vorhies & Morgan, 2005). The respondents were asked to indicate how their firm performed relative to their major competitors, with respect to customer satisfaction, market share, sales, acquiring new customers, return on investment and profitability (Cronbach's alpha = .85).

Data collection
Following McKenna (1991), survey data were collected from marketing managers or members of top management teams in Irish firms. The Irish Times Top 1000 Companies database, which consists of 969 1 companies, was used as the main source for recruiting respondents. Three hundred and nine companies were ruled out because they refused to participate in the study due to firm policy or because they had no marketing department in Ireland. Thus, 660 available firms were contacted to participate in this study. In order to accommodate firms that were not on The Irish Times Top 1000 Companies list, marketing managers of Irish firms who are alumni of researchers' university were also invited to participate. Dillman's (2011) Tailored Design Method was used to collect data. A total of 235 responses were received with 210 completed and qualified surveys, resulting in a response rate of 27.01% and a completion rate of 24.14%.  Of the 203 respondents (out of 210) who indicated their job titles, 36.95% were marketing managers, 19.70% were CMOs, 20.69% were Chief Executive Officers (CEOs), and 22.66% were other experienced professionals (titles include business development manager, commercial director, senior marketing coordinator, and director of sales and operation). Respondents had an average of 17.25 years of professional experience.

Respondent profile
Among the 210 responding companies, 74.76% were private companies and 25.24% were public traded. 67.14% were Business to Business (B2B) companies, while 32.86% were Business to Consumer (B2C) companies. Most companies (83.33%) adopted a differentiation strategy, while a small portion (16.67%) adopted either a cost leadership strategy or a mix of cost leadership and differentiation strategies. Our sample also included both manufacturing (18.10%) and service/trade (56.67%) firms.
No significant difference was found between early and late responses to the study, or between the Irish Times Top 1000 Companies responses and Alumni responses, indicating no concerns relating to non-response bias. In order to make sure that these respondents were competent informants, they were asked to indicate their knowledge on MPM and marketing capability related issues on a seven-point Likert scale. An average score of 5.48 indicated their reliability and validity as key informants (O'Cass & Weerawardena, 2010).

What key metrics have been used in Irish firms?
Financial metrics (e.g., sales and profitability) were identified by respondents as being the most commonly used marketing metrics in their firms. As shown in Figure 1, other commonly used marketing metrics in Irish firms were: consumer satisfaction, market share, brand awareness, visitors, return on investment, revenue of new products, loyalty/retention, and conversions (lead to sales). These results are in congruence with extant studies. For instance, sales, profitability, consumer satisfaction, market share, brand awareness, and number of new product have also been found to be the most commonly used metrics in other countries, such as the United Kingdom, China, Finland, Brazil, and Spain (Ambler & Wang, 2003;Ambler et al., 2004;Frösén et al., 2013;Sampaio et al., 2011).

How MPM practices have been changed in Irish firms since 2007?
To answer the second research question as to how MPM practices have been changed in Irish firms since 2007, the primary data collected in this study were compared with those collected by O'Sullivan (2007). The relative importance of marketing metrics to senior management teams was calculated and compared to explore how MPM practices have evolved in Irish companies from 2007 to 2015. (%: The percentage of respondents who report the usage of these metrics) Figure 2 shows the relative importance of marketing performance metrics to senior management teams in 2015 and 2007, respectively. The data collected in 2015 showed that, financial measures had the highest mean importance (4.3), followed by competition metrics (3.7), consumer attitude metrics (3.5), consumer behavior metrics (3.5), and digital metrics (3.5). Trade customer metrics (3.3) and innovation metrics (3.3) were valued less by senior management teams. As shown in Figure 2, financial metrics have been consistently ranked as the most important metrics by management teams. The results also highlight the fact that digital metrics, not investigated in 2007, have now entered the lexicon of marketing metrics used by Irish firms and are deemed as important as consumer attitude and consumer behaviour metrics.
In addition, this study also compared the frequency of marketing performance tracking and data collection in 2007 with that in 2015. The 2015 study revealed that financial measures were more frequently tracked by Irish companies than other marketing performance measures. As shown in Figure 3, the frequencies of marketing performance tracking and data collection have been surprisingly reduced. The results showed that companies tracked marketing performance less frequently in 2015 than they did in 2007. However, due to the increasing popularity of social media and digital marketing, Irish companies reported that they tracked digital marketing metrics quite frequently (3.7).
O'Sullivan (2007) had noted that, Irish companies had an over reliance on financial metrics and had less-frequent marketing performance tracking. These factors, he argued, had hindered the development of better MPM practices. The 2015 study found that Irish companies were still encountering very similar problems. Surprisingly, in 2015, companies were found to track marketing performance even less frequently than they did in 2007. Given that tracking frequency has been found to be positively related to firm performance (O'Sullivan, 2007), it is recommended that companies increase the frequency which they track their marketing performance.

If comprehensive MPM practices influence marketing capabilities and firm performance in Irish firms
A series of t-tests were conducted in order to answer the third research question. First, based on the level of the comprehensiveness of their MPM practices, respondents were mean-split into two groups: low comprehensiveness group and high comprehensiveness group. Seven t-test analyses were conducted to compare marketing capabilities and firm performance in the two groups. Frequency score: 1 = never, 2 = rarely/post hoc, 3 = regularly/yearly, 4 = quarterly, 5 = monthly or more often As shown in Figure 4, a t-test comparing firm performance in the two groups showed that companies adopting more comprehensive MPM practices performed significantly better (M2 = 4.95, SD1 = 1.08) than those adopting less comprehensive MPM practices (M1 = 4.50, SD2 = .98; t = -2.91, p < .01). The results indicated that high-performing companies were more likely to adopt a comprehensive MPM practice. Similarly, companies adopting a more comprehensive MPM practice reported higher levels of marketlinking capability (M1 = 4.71, SD1 = .92; M2 = 5.23, SD2 = 1.01; t = -3.60, p < .01). This result indicated that these companies were more able to build a good relationship with their customers, monitor their competitors, create a tight bond with their channel members, and sense market changes.
To further test the relationship between the comprehensiveness of MPMSs and marketing capabilities and firm performance, a series of linear regression tests were conducted. The results are shown in Table 3. Table 3, the comprehensiveness of MPMSs is found to positive influence subjective firm performance (β = .22, p < .01), market-linking capability (β = .35, p < .001), architectural marketing capability (β = .55, p < .001), relationally focused learning capability (β = .28, p < .001), market-focused learning capability (β = .36, p < .001), and adaptive marketing capability (β = .44, p < .001). The results further confirmed that comprehensive MPM practices were beneficial to companies. The adoption of comprehensive MPM practice could facilitate marketing capabilities and improve firm performance. For example, companies could use MPM practices to make better marketing plans, build better relationships with customers and channel members, and adapt to the dynamic environment more quickly.

Discussion and conclusions
Based on a survey of more than 200 marketers across different industries in Ireland, this study found that the top 10 most commonly used marketing metrics in Ireland were: sales, profit/profitability,  Figure 4. T-test analysis for differences. consumer satisfaction, market share, brand awareness, visitors, return on investment, revenue of new products, loyalty/retention, and conversions (lead to sales). This finding is consistent with previous studies conducted in other countries, such as China, Spain, Finland, Brazil, and the United Kingdom (e.g., Ambler & Wang, 2003;Ambler et al., 2004;Frösén et al., 2013;Sampaio et al., 2011).
The finding that Irish firms prioritised financial metrics is in line with the finding of O'Sullivan's (2007) study. Another finding was that Irish companies tracked marketing performance less frequently than they had done previously in 2007. The relatively low frequencies of tracking marketing performance is worthy of comment given that the frequencies of tracking marketing performance have been found to be positively associated with firm performance (Clark et al., 2006;O'Sullivan, 2007;O'Sullivan & Abela, 2007). Additionally, the independent t-test and regression results showed evidence for a positive relationship between comprehensive MPM practices, firm performance, and various marketing capabilities. These findings provided empirical support for the hypothesised relationship between MPM practices and drivers of firm performance in previous studies. This study is first of its kind to empirically examine the hypothesised relationship between MPM practices and marketing capabilities. This paper concludes that better-performing companies tend to have more comprehensive MPM practices in place (i.e., using a number of key marketing metrics to reflect organisation strategy). Therefore, it is recommended that firms use multiple marketing metrics (i.e., customer attitude metrics, customer behaviour metrics, trade customer metrics, competition metrics, financial metrics, innovation metrics, and digital metrics) to monitor the performance of their marketing activities. It is also recommended that management teams use marketing metrics more frequently to enhance their firms' marketing capabilities and improve marketing and financial performance. Use of marketing performance information to identify problems and provide timely feedback regarding their customers, suppliers and partners, facilitates firms in reacting to the changing environment more quickly and creating better relationships with their customers and channel members.
However, this paper is not without any limitations, some of which may provide future research opportunities. The first limitation lies in the use of a cross-sectional design, which prevents us of testing the causal relationship between the comprehensiveness of MPM and marketing capacities and firm performance. Therefore, it is recommended that future research adopt a longitudinal research design to explore this research phenomenon. In addition, since this study only explored the direct impacts of the comprehensiveness of MPM on marketing capabilities and firm performance, future research can consider the possible mediating mechanisms underlying these impacts (Homburg et al., 2012). The second limitation is the use of single-informant for data collection. Thus, another recommendation for future research is to collect data from multiple informants to increase the credibility of data. Third, this study was conducted under the Irish context. Thus, the research results should be generalised with great caution. Future research undertaken in other contexts might provide even more compelling evidence to this research area.