Original research
Descriptive conversion of performance indicators in rugby union

https://doi.org/10.1016/j.jsams.2018.08.008Get rights and content

Abstract

Objectives

The primary aim of this study was to examine whether accuracy of rugby union match prediction outcomes differed dependent on the method of data analysis (i.e., isolated vs. descriptively converted or relative data). A secondary aim was to then use the most appropriate method to investigate the performance indicators (PI’s) most relevant to match outcome.

Methods

Data was 16 PI’s from 127 matches across the 2016–17 English Premiership rugby season. Given the binary outcome (win/lose), a random forest classification model was built using these data sets. Predictive ability of the models was further assessed by predicting outcomes from data sets of 72 matches across the 2017–18 season.

Results

The relative data model attained a balanced prediction rate of 80% (95% CI – 75–85%) for 2016–17 data, whereas the isolated data model only achieved 64% (95% CI – 58–70%). In addition, the relative data model correctly predicted 76% (95% CI – 68–84%) of the 2017–18 data, compared with 70% (95% CI – 63–77%) for the isolated data model. From the relative data model, 10 PI’s had significant relationships with game outcome; kicks from hand, clean breaks, average carry distance, penalties conceded when the opposition have the ball, turnovers conceded, total metres carried, defenders beaten, ratio of tackles missed to tackles made, total missed tackles, and turnovers won.

Conclusions

Outcomes of Premiership rugby matches are better predicted when relative data sets are utilised. Basic open-field abilities based around an effective kicking game, ball carrying abilities, and not conceding penalties when the opposition are in possession are the most relevant predictors of success.

Introduction

Success in sport can be assessed and quantified with performance indicators (PIs).1 Understanding PI’s that relate to success in sport is important for coaches to improve future technical, tactical and physiological performance.2 Whilst the most meaningful PI's should differentiate between successful and unsuccessful outcomes,1 no consensus can currently be drawn in rugby union regarding PI’s associated with success.4, 6, 8

Based on the available literature, the frequency of ball kicking differentiates success in both domestic and international rugby union matches.4, 7, 8 Winning teams kick the ball more and kick away greater proportions of possession. Match winners also have lower error4, 9 and turnover8, 9 rates compared to losers. In addition, winners have an effective defensive game, with a superior success rate at the tackle8 and make more tackles overall.4 Attacking actions, such as higher distance of average carry8 and making more clean breaks in the opposition’s defensive line,3, 7, 8 are also associated with successful performances. Together with open field actions, set piece performance is important, with winners securing more opposition lineouts9 and a greater effectiveness at the scrum.7 However, some research has failed to uncover significant differences in PI’s between successful and less successful teams. For example, at the 2011 World Cup competition, multiple indicators were examined and no differences were established that explained tournament ranking.5

It is unlikely that the complex, dynamic and interactive games such as rugby union can be represented by simple analysis or frequency data.5 The conflict in current literature with respect to PI’s and match outcome is best represented by Vaz et al.4 They reported significant predictors of match outcome in the Super Rugby competition, but the same PI’s did not differentiate between winners and losers in an International competition. The authors suggested international level differences between winners and losers do not exist or are masked by variations in playing styles that underpin match outcome.

A significant limitation of the above research is the failure to acknowledge that, in rugby union, outcome depends on ability and performance of both teams. Therefore, when considering associations between PI’s and competition results equal emphasis should be placed on data from each team.2 Failure to do so will likely distort any relationships present.1 Processing sports data to consider PI’s as a differential between opponents is suggested as a better descriptor of a sport’s nature10 and a contest’s outcome. In analysing sports data, this type of data processing method has been termed “descriptive conversion” but has not been applied in the literature concerning rugby union. Only isolated data has been considered, ‘isolated’ referring to the PI’s of each participating team considered discretely and not relative to the opposition.

The primary aim of this study was to examine whether accuracy of match prediction outcomes differed dependent on the method of data analysis (i.e., isolated vs descriptively converted data). A secondary aim was to use the most appropriate method to identify the most relevant PI’s for successful outcomes in rugby union and specify how this information can have practical relevance to sports practitioners.

Section snippets

Methods

PI's for the 2016–17 English Premiership Rugby Union regular season and the first 12 rounds of the 2017–18 season were downloaded from the OPTA website (optaprorugby.com). The 2016–17 season data consisted of 22 rounds of 6 matches (132 matches total, 12 teams). As the study assessed the impact of PI’s on a binomial outcome (win/loss), matches that finished with a draw (n = 5) were excluded from analysis. The full set of team PI’s for each match were utilised in the analysis. These PI’s were

Results

The randomForest model based on the isolated data set from the 2016–17 season classified 85 from 127 losses (67%) and 78 from 127 wins (61%), giving an overall accuracy of 64% (95% CI 58–70%, p < 0.05). The randomForest model based on the relative data set predicted 102 of 127 losses (80%) and 101 of 127 wins (80%), with an overall accuracy of 80% (95% CI 75–85%, p < 0.05). The McNemar’s value of 57.7 (p < 0.05) confirmed that the relative model outperformed the isolated model.

When assessing the

Discussion

The primary aim of this study was to investigate for the first time whether a relative (a data set that has undergone descriptive conversion) or an isolated data set best predicted outcomes of rugby union matches. Results indicated relative data was more effective at predicting match outcome compared to isolated data. The model based on the relative data set outperformed the isolated data model in terms of overall accuracy and, as per previous research,24, 25 the balance of prediction was

Conclusions

This study demonstrates the effectiveness of utilising data that has undergone descriptive conversion in predicting match outcomes. It also demonstrates game outcomes are more closely related to open field abilities and basic skills such as ball carrying, kicking and tackling ability than they are to set pieces and, despite the apparent complexity of the game, success can be explained by a small number of basic components.

Practical applications

  • The use of relative data sets rather than isolated data sets, when evaluating match performance.

  • Devising game strategies to maximise average carry and tackles at or over the gainline.

  • Having a focus on defensive strategies that minimise the likelihood of conceding penalties. This would include areas of the game where high numbers of penalties are conceded in matches, for example when defending driving line-outs.

  • Using partial dependency plots to set objective team performance markers.

Acknowledgements

None to declare. There was no financial support for this study. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References (30)

  • Z. He et al.

    Stable feature selection for biomarker discovery

    Comput Biol Chem

    (2010)
  • M.D. Hughes et al.

    The use of performance indicators in performance analysis

    J Sports Sci

    (2002)
  • T. McGarry

    Applied and theoretical perspectives of performance analysis in sport: scientific issues and challenges

    Int J Perform Anal Sport

    (2009)
  • S. den Hollander et al.

    Skills associated with line breaks in elite rugby union

    J Sports Sci Med

    (2016)
  • L. Vaz et al.

    Rugby game-related statistics that discriminate between winning and losing teams in IRB and super twelve close games

    J Sports Sci Med

    (2010)
  • M.T. Hughes et al.

    Performance indicators in rugby union

    J Hum Sport Exerc

    (2012)
  • S. Prim et al.

    A comparison of performance indicators between the four South African teams and the winners of the 2005 super 12 rugby competition. What separates top from bottom?

    Int J Perform Anal Sport

    (2006)
  • E. Ortega et al.

    Differences in game statistics between winning and losing rugby teams in the six nations tournament

    J Sports Sci Med

    (2009)
  • N. Watson et al.

    On the validity of team performance indicators in rugby union

    Int J Perform Anal Sport

    (2017)
  • N.M.P. Jones et al.

    Team performance indicators as a function of winning and losing in rugby union

    Int J Perform Anal Sport

    (2004)
  • B. Ofoghi et al.

    Data mining in elite sports: a review and a framework

    Meas Phys Educ Exerc Sci

    (2013)
  • Evans JS, Murphy MA. rfutilities. R package version 1.0-0....
  • Breiman L, Cutler A, Liaw A, et al. Package “randomForest.”; 2011. Softw available URL...
  • R Core Team. R foundation for statistical computing. Vienna, Austria....
  • Kabacoff RI. R in Action. manning;...
  • Cited by (30)

    • Performance indicators associated with match outcome within the United Rugby Championship

      2023, Journal of Science and Medicine in Sport
      Citation Excerpt :

      The introduction of the 50:22 law in the test data is likely to drive additional kicks within winning teams.21 These findings validate what has been reported in both elite Rugby Union and sub-elite Rugby Union.2–4 Attacking metrics, including metres made, and clean breaks also ranked highly within the reduced relative model, demonstrating the importance of an effective attack and conversely a strong defence.

    • Key performance indicators in Australian sub-elite rugby union

      2020, Journal of Science and Medicine in Sport
      Citation Excerpt :

      Perhaps sub-elite teams may not convert penalty opportunities into points as well as elite teams, though further analysis into whether this is the case is warranted. Similarly, to the elite population,2 penalties conceded when attacking was ranked last. This may suggest a consistent refereeing style across the southern and northern hemisphere in which attacking teams are given appropriate advantage.

    • Who said “there is no ‘I’ in team”? The effects of observational learning content level on efficacy beliefs in groups

      2019, Psychology of Sport and Exercise
      Citation Excerpt :

      Responses to the ATG-T item were rated between 0 (not involved at all) and 100 (highly involved), and responses to the GI-T item were rated between 0 (not close at all) and 100 (very close). Team performance in rugby is multifaceted, as evidenced by the wealth of performance indicators adopted in this sport (Bennett, Bezodis, Shearer, Locke, & Kilduff, 2019). In this study, performance was measured using twenty-three performance indicators that represent overall rugby performance, as collected by the performance analysis unit of the elite age-grade rugby union team.

    View all citing articles on Scopus
    View full text