The effectiveness of South African defined-contribution occupational fund benefit statements to inform and persuade: framework and initial applications

Members of defined-contribution retirement funds may be able to make choices that affect their retirement outcomes. The benefit statement is considered a key resource in this process, and trustees and administrators may design these statements specifically to inform and persuade members to make appropriate choices to improve their retirement outcomes. For a statement to be theoretically effective at informing members it should be effective at communicating the inherent risks, be appropriate for the audience, have meaningful and realistic illustrations, use reasonable and consistent assumptions and show sensitivity to these assumptions, be balanced and complete, include a statement of principal assumptions and definitions of key terms and outline the options available. For a statement to be theoretically persuasive it should use emotion appropriately, identify behaviour to change, identify the member’s needs and create a link between these needs and the behaviour to change, be positively framed, personalised and appropriately timed. These characteristics can be used to develop a framework to assess the theoretical effectiveness of benefit statements. This framework was applied to a small sample of administrator benefit statements to assess their effectiveness. When member data from one fund were analysed, it was found that improving the theoretical effectiveness of the benefit statement for this fund was not sufficient to improve the contribution rate. This merits a larger scale investigation.


INTRODUCTION 1.1
The Retirement Savings Landscape 1.1.1 South Africa has experienced a shift from defined-benefit to definedcontribution retirement funds (Andrew, 2004). Therefore, a large proportion of members now bear the risks associated with retirement funding and there is far more uncertainty surrounding the benefits that these members will receive at retirement (Impavido et al., 2010).
1.1.2 There are a number of possible actions a member is able to take either within the fund or externally to manage these risks (Sundén, 2006), but many members are not even aware of the available options (Groyer & Holtzhausen, unpublished), let alone equipped to make optimal choices (Sundén, 2006). This view is shared by Bernheim (1998) and Impavido et al. (2010).
1.1.3 A study carried out by Atkinson & Messy (unpublished) showed that 52% of South Africans prefer spending to saving for the long term and 60% tend "to live for today and let tomorrow take care of itself ". This attitude represents a significant impediment given the responsibility that members bear in a defined-contribution environment.

The Importance of the Benefit Statement to Members
The necessity of providing members with appropriate information to aid them in their decision-making has been stressed by Clark et al. (2012) and Sundén (2006). Research from the United Kingdom has shown that benefit statements are the primary source of information which members use to make informed decisions (Sykes et al., unpublished) and South African research by Van der Waldt & Van Heerden (2007) has shown that fund members view benefit statements as important.

1.3
Reasons why Funds send out Benefit Statements 1.3.1 Although members may rely on benefit statements to make decisions, funds may have different reasons for distributing benefit statements ranging from regulatory compliance to persuasion.
1.3.2 At the most basic level, funds may communicate with members in order to comply with regulation (Antolín & Harrison, unpublished). In the South African context, under the Pension Funds Act, 1 the trustees must ensure that enough and correct information is passed on to the members about their rights, benefits and duties in terms of the rules of the fund. Trustees fulfil this requirement by sending an annual benefit statement to every member and an explanatory pamphlet to new members as set out under pension fund circular 86. 2 1.3.3 In other cases, funds may communicate to inform members and assist them with retirement planning and savings decisions (Van der Waldt & Van Heerden, 2007). 1.3.4 In some funds, the communication is aimed at modifying both the employee's knowledge and perception of saving (Wiener & Doescher, 2008). Communication

1.5
Contribution to Knowledge 1.5.1 To the authors' knowledge there is scant quantitative research on the effect of benefit statements on member behaviour in general, and no local quantitative research in particular. By developing the framework for measuring effectiveness and providing a methodology for testing a change in member behaviour, this research provides a first step towards local quantitative research in this area. 1.5.2 The qualitative frameworks developed to assess theoretical effectiveness and persuasiveness of benefit statements have been used in two separate studies and have found to be robust over time. They may thus be of use to administrators and trustees when designing new benefit statements or improving existing ones.
1.5.3 At the time of writing there was no local professional guidance for actuaries providing input to the design of benefit statements and the Actuarial Standards Board in the United Kingdom has not issued guidance in this area after the withdrawal of Guidance Note 34. 3 This research represents a summary of principles for actuaries working in this area, and particularly actuaries working as trustees of retirement funds.

1.6
Plan of Development Section 2 sets out the literature review in which the four research questions posed in ¶1.4.2 are discussed in turn. Section 3 sets out the framework for assessing the theoretical effectiveness of the benefit statements. Section 4 sets out the methodology the authors used in a case study to assess whether improving the theoretical effectiveness of benefit statements had an impact on member behaviour. The results are set out in section 5 before the discussion and conclusions are given in sections 6 and 7.
This heuristic lends itself to the construction of a benefit statement in which all the risks and uncertainties are first explained before providing an illustration, to ensure the member has a full understanding of the uncertainty associated with the illustrations.
2.1.1.4 People respond better to personalised information when dealing with complex information (Nyce, 2005;Sykes et al., unpublished). This suggests the use of retirement income projections, rather than generic economic information or investment reports, for example. However, Maslowska et al. (2011) found that personalisation is only helpful where the recipient has a high need for uniqueness. The need for uniqueness theory posits that individuals who are made happier by believing they are unique are better able to remember information that helps them distinguish themselves from others (Maslowska et al., 2011 Sykes et al. (unpublished) argue that understanding the attitudes of readers towards retirement and the benefit statements is also important. For example, those who perceive themselves as too close to retirement to make meaningful changes or too far from retirement for it to be relevant were largely indifferent to benefit statements (Sykes et al., unpublished). Bassett et al. (1998) note that in addition to age and earnings, years of service also affect members' perceptions of the importance of the benefit statement.
2.1.2.3 Many members have both a low level of interest in the benefit statement and a low knowledge base (Duklan & Martin, 2002;Sykes et al., unpublished). Hence, it is important that sufficient detail is given to improve understanding (Sykes et al., unpublished) without too much detail creating confusion (Sze, 1993).
2.1.2.4 The appropriateness for the audience depends on the language used in benefit statements (Sykes et al., unpublished). Gluckman & Kamionsky (1997) suggest that the preferred language of the member should be used if possible. Irrespective of the language adopted, words used should be simple and sentences should be short and direct (Sze, 1993;Duklan & Martin, 2002). The use of familiar language as opposed to technical jargon, and concrete language rather than abstract concepts is preferred (Duklan & Martin, 2002).

mEaningful and rEalistic illustrations
2.1.3.1 Illustrations should be meaningful and realistic. 5 Apart from the illustrations relating to risks discussed in ¶2.1.1.2, the projected level of the retirement benefit can also aid members in their retirement planning ). 2.1.3.2 Sze (1993 stresses that the illustrations must be understood by the recipient. In order for the results to be understood correctly it is essential that information is given in a suitable context (Duklan & Martin, 2002). This reinforces the importance of the statement being appropriate for the audience as discussed in section 2.1.2.

2.1.3.3
The way in which the illustrations are presented is critical. Groyer & Holtzhausen (unpublished) hypothesise that members relate their expected benefits to income needed in retirement and for this reason it is appropriate to have projected expected benefits as a monthly income rather than a lump sum. Although this contributes further to the uncertainty of the projection due to the need for an annuity type and price assumption (McGill, 1984), it avoids the problem of wealth illusion (Sundén, 2006). Wealth illusion refers to the preference people have for lump sums over regular income streams, either due to overestimation of the lump sum or underestimation of the income stream (Fetherstonhaugh & Ross, 1999).
2.1.3.4 Sykes et al. (unpublished) also argue that incorporating key explanations with any forecasts aids members in their interpretation of the values and figures and hence make illustrations more meaningful.

rEasonablE and consistEnt assumptions
Any assumptions used in calculations should be reasonable and consistent. 6 The reasonability of assumptions should contribute to the realism of the illustrations (Sze, 1993;Rinaldi & Giacomel, unpublished). Consistency is also stressed by Gluckman & Kamionsky (1997).
2.1.5 sEnsitivity to assumptions dEmonstratEd 2.1.5.1 The benefit statement should describe or illustrate the sensitivity of the results to the assumptions. 7 There are many available approaches to illustrate sensitivity, including stochastic analysis 8 and comparison of illustrations performed using different assumptions (Rinaldi & Giacomel, unpublished;Gluckman & Kamionsky, 1997;Duklan & Martin, 2002). However, Sykes et al. (unpublished) found that members themselves preferred single figure forecasts rather than ranges because the ranges were viewed as difficult to interpret and too generalised and impersonal.
2.1.5.2 Another possible approach as suggested in Guidance Note 34 is to quote the effect of a change to a key assumption such as mortality or investment return. 9 However, Sundén (2006) notes that a member may be able to understand why benefits reduce with lower investment returns but may have a more difficult time understanding why a lighter mortality assumption results in a decreased expected monthly benefit.

balancEd and complEtE
Balanced communication is described as "taking everything into account; fairly judged or presented" 10 and complete communication is defined as "having all the necessary or appropriate parts". 11 Duklan & Martin (2002) and Sykes et al. (unpublished) propose that a narrative approach can improve understandability. This is balanced, in the sense that Duklan & Martin (2002) outline that this requires three clear sections to the communication: a beginning, which sets the scene and allows readers the chance to understand the background to the story as well as motivates the reasons for the story being told; a middle section, which contains the most critical elements; and an ending, which brings together any loose ends and presents the moral of the story. Duklan & Martin (2002) further suggest that complete communication requires that numbers, words and pictures need to be incorporated so as best to engage the reader and portray the message.

statEmEnt of principal assumptions and dEfinition of kEy tErms
The statement of principal assumptions and definitions of key terms are important parts of disclosure. 12 The definitions may limit the scope for different interpretations (Tversky & Kahneman, 1974;Covello & Sandman, 2001) and reduce the confusion that may arise due to terms having different meanings for those drawing up the benefit statements and the average person reading them (Jardine & Hrudey, 1997). For example, Sykes et al. (unpublished) found that most members do not understand the meaning of the term 'today's prices'. The statement of principal assumptions allows for the reasonability of assumptions to be assessed which is discussed in ¶2.1.4.

outlining thE various options availablE to mEmbErs
2.1.8.1 Effective communication, as described by Duklan & Martin (2002: 11), should "engage the audience to understand and actively participate" as well as elicit the right questions from members. Thus, beyond merely informing members, effective communication should encourage members to adopt an active approach towards their retirement savings (Sundén, 2006). To do this, members need to then have their options outlined so that they may weigh up their possibilities (Groyer & Holtzhausen, unpublished) and choose the correct course of action given their situation (Rinaldi & Giacomel, unpublished). The member should be reminded of options such as raising contributions, changing retirement age and altering investment portfolios (Groyer & Holtzhausen, unpublished;Sykes et al., unpublished;Sundén, 2006). This prevents members viewing their annual benefit statements as a routine formality, which is the case for most members, as opposed to a reminder to review their decisions, (Sykes et al., unpublished).
2.1.8.2 When making options available to members, the limitations of the benefit statement needs to be made clear (Rinaldi & Giacomel, unpublished). Also, setting out the various options to the members, particularly the option of additional contributions, should not give the impression of selling or lecturing (Rinaldi & Giacomel, unpublished). If this is done, then not only do members take these recommendations less seriously, but they might look at the entire benefit statement in a more sceptical light (Rinaldi & Giacomel, unpublished).

What makes a Benefit Statement effective at persuading Members?
The literature identifies seven properties of benefit statements that improve their effectiveness to change behaviour, namely: appropriate use of emotion; identification of the behaviour to change; identification of needs; identification of the link between needs and behaviour; positive framing of information; personalisation; and appropriate timing.
These concepts are expanded upon below.

appropriatE usE of Emotion
Emotive content has been shown to have a greater impact on financial decisions than using factual information alone (Hauff et al., 2014). Wentzel et al. (2010) confirm that narrative communication, which follows a storyline, is likely to have a greater emotional impact than pure statement of fact. This heightened emotional impact is likely to make the communication more persuasive (Wentzel et al., 2010). However, if messaging is too emotive, this may cause members to focus on the emotion rather than the relevant information in the message (Mullainathan & Shleifer, unpublished). Studies have shown that people recall communications that reinforce their beliefs and ignore those that do not (Mullainathan & Shleifer, unpublished). The implication is that by aiming for an emotional response, the message that members need to hear could be sacrificed for the one that members want to hear (Mullainathan & Shleifer, unpublished). Wentzel et al. (2010) concluded that if it were obvious to the reader that the communication was being used to manipulate or persuade, they would view the message with scepticism. The meaning of the communication to the recipient may thus be altered (Wentzel et al., 2010). Hence, although the use of emotion is valuable, too much can detract from, change and discredit the key message.

idEntification of thE bEhaviour to changE
The need to clearly identify member's options to enable them to make decisions was outlined in section 2.1.8. Wiener & Doescher (2008) identified three key areas where persuasion to modify behaviour could be useful: the level of the contribution rate; the investment decision; and -the retirement age.
Where participation in a retirement fund is voluntary, Nyce (2005) also found that participation rates could be affected by persuasive communication.

idEntification of nEEds
The level of income required in retirement is a retirement need that has been shown to be useful in persuading members to modify their retirement savings behaviour . On a psychological level, members have needs for self-esteem, security and social approval (Wiener & Doescher, 2008).

idEntification of thE link bEtwEEn nEEds and bEhaviour
Wiener & Doescher (2008) found that persuasive communication should indicate to members that their psychological needs can be met through retirement savings. This linkage could be created through emotion, relatable spokespeople and family imaging such as grandparents in retirement (Wiener & Doescher, 2008).

positivE framing of information
2.2.5.1 Wiener & Doescher (2008) suggest that higher participation in savings plans are seen when people perceive saving to be easy. Conversely, where people are concerned about their retirement outcomes, savings rates can be negatively affected (Wiener & Doescher, 2008). Positively framed messages about retirement increase savings rates through the former effect and negatively framed messages can reduce savings rates via the latter effect (Wiener & Doescher, 2008). This is supported by research by DeBono (1987), who found that if the messaging induced a favourable thought, then persuasion worked, but if more negative thoughts were induced, then persuasion would not work and should not be used.
2.2.5.2 However, it is important to always ensure that the communication is in the member's best interests, rather than being used to only tell people what they want to hear (Mullainathan & Shleifer, unpublished). An overly positive message may elicit scepticism which may discredit the key message as discussed in ¶2.2.1.

pErsonalisation
According to Van der Waldt & Van Heerden (2007), personalised, interactive communication seems to be more effective at getting people to increase their savings when compared to general communication. Nyce (2005) and Bochet et al. (2006) concur. The importance of personalisation in general communication was discussed in ¶2.1.1.4.

appropriatE timing
Wiener & Doescher (2008) discuss the importance of the timing of the persuasive communication. Sending out persuasive communication aimed at increasing contribution rates is best done at salary increase time to avoid a reduction in living standards (Wiener & Doescher, 2008). Put differently, the timing of the communication should minimise the perceived discomfort resulting from the behaviour change (Wiener & Doescher, 2008).

What, apart from Communication, could influence Contribution Rates?
To be able to test whether more effective communication increases contribution rates, it is important to control for confounding factors. Age, state of the economy, gender and marital status, salary earned, employment tenure and employer's contribution have all been shown to influence contribution rates, as discussed below.

agE
2.3.1.1 According to Holden & VanDerhei (2001) and Huberman et al. (2007), age is a significant factor influencing contribution rates. They agreed that age is a proxy for the stage in the life savings cycle. At younger ages, people tend to save less because of study loans and acquiring assets. As they get older, these expenses decrease leaving more room for saving (Holden & VanDerhei, 2001). However, these studies reached different conclusions about the relationship between age and the rate of contribution. Holden & VanDerhei (2001) concluded that contribution rates increase with the member's age, while Huberman et al. (2007) found that contribution rates have a negative correlation with ages below 40 and a positive correlation with ages above 40.
2.3.1.2 In contrast, Munnell et al. (2001) argued that age is not an important factor in determining contribution rate.
2.3.1.3 The different results may stem from the way age was modelled in each study. Huberman et al. (2007) modelled age as a continuous variable, whereas Holden & VanDerhei (2001) rounded the age variable down to the nearest 10, thus creating five different age groups. Munnell et al. (2001) incorporated age in their model as a categorical variable with only three age bands. The different treatment of the age variable in these three studies would change the statistical models used and could account for the different results.

statE of thE Economy
During recessions, contribution rates from 401(k) plan members decline (Butrica & Smith, 2016;Muller & Turner, unpublished). This may be motivated by a greater need for liquidity when the risk of unemployment is higher and by poor prospects for investment returns from the market (Butrica & Smith, 2016;Muller & Turner, unpublished). Huberman et al. (2007) found that female retirement fund members tend to contribute more than males for following two reasons: -Females, in general, value saving for retirement more than their male counterparts.

gEndEr and marital status
-Married women's salaries are secondary to their husbands, which consequently allowed women have more salary available to contribute. This suggests that marital status is also an important factor.

salary EarnEd
The salary earned by members significantly influences their contribution rates (Munnell et al., 2001;Holden & VanDerhei, 2001). Munnell et al. (2001) found that contribution rates are negatively affected by the salary earned. This is because there is an index-linked limit on the amount the member contributes that is imposed by tax legislation. 13 However, Holden & VanDerhei (2001) found that the contribution rate is positively correlated to salary earned until the contribution amount reaches the limit after which the contribution rate becomes negatively correlated to the salary amount earned.

EmploymEnt tEnurE
The tenure, or employment tenure, of a member is defined as the number of years of service for their current employer. Huberman et al. (2007) found contribution rates are always positively correlated with the member's tenure. However, Holden & VanDerhei (2001), found that for the first 18 years, contribution rates rose with service and declined with service thereafter.

EmployEr contribution
2.3.6.1 Holden & VanDerhei (2001) showed that as the size of employer contribution rate increases, the employee's contribution rate increases. In contrast, Munnell et al. (2001) found that the size of the employer's contribution is significantly negatively correlated to their employees' contribution rates.
2.3.6.2 Both of these findings are based on results from analysing occupational retirement funds under 401(k) plans in the United States of America. Under 401(k) plans, employers decide on the employer's match rate, which is the percentage of the amount contributed by the employee that the employer contributes towards the employee's retirement savings account (Holden & VanDerhei, 2001). Both the studies by Holden & VanDerhei (2001) and Munnell et al. (2001) take into account the employer's match rate, which is controlled for in their analysis, but it is important to note that this is not a feature of South African defined-contribution funds.

2.4
Does making a Benefit Statement more theoretically effective at informing and persuading increase Contribution Rates? 2.4.1 A number of studies have found that communication can result in increased contribution rates. Groyer & Holtzhausen (unpublished) found that many South African retirement fund members opt to make additional contributions once they realise their projected fund shortfall and the possibility of contributing more is explained to them. Mayer et al. (2011) found that adding information relating to income needs in retirement increases retirement savings significantly. Nyce (2005) found that making communication more personalised has a positive effect on contribution rates, which was also found by Clark & Schieber (1998). Thus, there is some empirical evidence that more theoretically effective communication should increase contribution rates.
2.4.2 However, some authors suggest that no change at all in contribution rates should be expected. Madrian & Shea (2001) explained that members have a tendency to 13 https://www.efile.com/ira-401k-maximum-contributions-limits/, Accessed 23/09/2019 contribute at the default contribution rate either as a result of participant inertia or employees' perception that the default contribution rate is implicit advice to them from their respective employers.

3.1
A framework for assessing the theoretical effectiveness of a benefit statement at informing members is shown in Table 1.  3.3 Although a well-timed benefit statement that corresponds with salary increases would be ideal as discussed in ¶2.2.7, this can be practically difficult. Anecdotally, benefit statements are often linked to the financial year end of the retirement fund which may be on a different annual cycle to the employer's remuneration review. Even if the two were co-ordinated, there is often a delay between the effective date of the benefit statement and their circulation due to the timing of the receipt of contributions and data checks.

3.4
The criteria for effectiveness have a maximum total score of 16 whereas those for persuasiveness have a maximum total score of 13. For the purposes of this research, the two scores were normalised so that equal weight was given to both factors. In order to avoid spurious accuracy, they are expressed as a percentage with one decimal place.

3.5
The frameworks were tested on four different benefit statements gathered in 2012 as an initial pilot. Scores ranged from four to ten out of 16 for effectiveness and three to eight out of 13 for persuasiveness. Using equal weightings, the composite scores for benefit statement effectiveness ranged from 24.0% to 58.2% with an average score of 39.7%. This suggested that there were considerable differences in the effectiveness of benefit statements and considerable scope for improvement.

3.6
In 2013, a further two benefit statements were collected and were found to have an average score of 48.0%. These benefit statements were from administrators who did not provide benefit statements in the 2012 study. Hence, one cannot tell if the improved score is as a result of an improvement over time or because of different approaches adopted by different administrators. This suggested a more controlled experiment into the improvement of benefit statements over time is warranted.

Overview
The aim of the case-study experiment was to ascertain whether a change in the theoretical effectiveness of a benefit statement had an impact on observed contribution rates. It is given as an example of a methodology that can be applied to other funds, as opposed to providing conclusions to apply to members in general. A defined-contribution occupational retirement fund that had changed the format of their benefit statement was identified. Information was then gathered about the change in the benefit statement and when the benefit statements were distributed to members. The contribution rates before and after these changes were tracked as well as other factors known to influence contribution rates, as discussed in section 2.3. The data were cleaned and checked. Statistical tests were then carried out to ascertain whether receiving a more theoretically effective benefit statement had an effect on contribution rates.

Identifying Appropriate Funds
In 2016, four large South African retirement fund administrators were approached for permission to access data from funds that allowed for member choice in contribution rates and had had a recent change in benefit statement format. The administrators were asked to identify such a fund and permission to access the fund data was requested from the trustees. This was done by means of introductory letters together with a questionnaire regarding the fund. Only one of the administrators agreed to participate in the study. They identified a fund meeting the study criteria and provided the requested data at an appropriate level.

4.3
Data sourced 4.3.1 The following information was requested relating to the benefit statements and choices: the current and previous format of the benefit statement; the date at which the format of the benefit statement changed;

EFFECTIVENESS OF SA DEFINED-CONTRIBUTION OCCUPATIONAL FUND BENEFIT STATEMENTS | 15
the contribution rate options available to members; the months in which members are allowed to change their contribution rate; and the month in which the benefit statement is received by most members.

4.3.2
In addition, for each member in the fund, monthly data were requested. Data for the fields in Table 3 were required from 11 months before the last benefit statement on the old format was issued until the present. 14 Date member joined the fund As compulsory participation in a retirement fund is usually a condition of service in South Africa, in most cases this date is the same as the date the member joined the employer. This is used to calculate a proxy for tenure, which was discussed in ¶2.3.5. Date the member joined the employer Used to calculate tenure, which was discussed in ¶2.3.5.
Exit date (where applicable) Used to assess whether the observation for a member was truncated.

4.3.3
As discussed in ¶2.3.2, the state of the economy also influences the contribution rate. Maria-Dolores & Vázquez (2008) stated that the growth rate of GDP provides the better overall view of an economy compared to other existing measures. The South African quarterly GDP over the relevant period was retrieved from the INET database. This particular fund changed the format of its benefit statement in October 2015. These statements are distributed to members in October each year through the employer.

4.4.3
The fund had a default gross contribution rate of 14%, which could be varied by the member in any month. Any changes had to be done through the employer. The employer in turn communicated the requested changes in contribution rate to the administrator through the contribution deduction schedule. The administrator received this schedule every month from the employer. Any requested changes were effective within a month from the date the administrator received the contribution deduction schedule. Annual bonuses were paid in December of each year and were included in the pensionable earnings. Thus, contributions in rand terms were always higher in December. Although it might have been theoretically possible for the member to lower their contribution rate in the month of December to keep the contribution level, this was not found in practice and the authors attributed this to the administration process around changes to the contribution rate.

4.4.4
For each member in the fund, the administrators provided information about the gender, gross annual salary excluding bonus, the date the member joined the fund and their gross monthly contribution for each month between October 2014 and February 2016. They also provided a copy of their benefit statements under the old and new format. They were unable to provide any information on the marital status nor information about the breakdown between employee and employer contributions.

4.5
Data Cleaning 4.5.1 The data set was checked and cleaned allowing only for individuals aged between 18 and 64 years actively contributing to the fund. Members who joined or left the fund during the period of investigation were also excluded from the sample. After the data check and cleaning process, there were 956 members left. 4.5.2 The sample was significantly smaller in size than the samples used by Choi et al. (2002) and Holden & VanDerhei (2001). However, the sample size was similar to that used by Munnell et al. (2001). Hence, this sample may not be representative of the general population, but there is no reason to believe that the fund and members exhibit characteristics that bias the outcome of the analysis (Nyce, 2005) 4.5.3 Given the incomplete bonus information received, it was decided to exclude the data for the months of December 2014 and December 2015. A sustained increase in contribution rate is required to improve retirement outcomes and because the contribution rates for the months before and after these observation points were accurately available, this should have limited impact on the results.

4.6
Descriptive Statistics 4.6.1 Table 4 lists summary statistics for the sample. Of the 956 members, 39.43% were male and 60.57% were female. For the pooled sample, the average contribution rate increased very slightly after the format of the benefit statement changed. Most members were found to contribute at the default contribution rate and this did not change throughout the period.

4.6.2
The average age and salary were higher after the benefit statement changed. This was to be expected given salary inflation, members aging over time and no new entries being allowed in the sample. Analysis of the quarterly GDP values during the period of investigation showed that GDP was relatively constant and did not decline in any two consecutive periods. This implies that there was no recession during that period. It was therefore decided not to include GDP as a factor in the analysis. 4.7.2 From the data set and information provided the following variables were created as shown in Table 5. The benefit statements were scored using the framework set out in section 3.

4.7.4
The contribution rate was calculated by dividing the gross contribution by the gross monthly salary. The gross monthly salary was calculated by dividing the gross annual salary by 12. This ignores the possibility of a thirteenth cheque, which is typically non-pensionable. 4.7.5 The average contribution rates were calculated for two periods to correspond with the receipt of each of the two benefit statements. The first period was from October 2014 to September 2015, prior to the change in format, and the second period was from October 2015 to February 2016, after the change in format. 4.7.6 A repeated measure ANOVA was carried out to determine if the number of months since last benefit statement was received has a significant effect on contribution rate. Tukey's test was used as a post-hoc test to identify which factors were significantly different from each other. These tests showed that once the data from the members' bonus month were excluded, receipt of a benefit statement, in general, had no effect on the contribution rate.

4.8
Testing for the Effect of Theoretical Effectiveness of Benefit Statements on Contribution Rates 4.8.1 A paired t-test was performed on the sample to determine whether the contribution rates before and after the change in format of the benefit statement were significantly different from each other for each member. 4.8.2 The authors tested the average contribution rate prior to the change in format of the benefit statement against the average contribution rate after the change in benefit statement for each member. Where the averages were significantly different from each other, a fixed effect panel data model was used to identify whether the change in theoretical effectiveness score had a significant effect. Post-hoc tests were carried out to test and correct for multi-collinearity. 4.8.3 A fixed effect model is appropriate for this study as it works well with longitudinal data which is the format of the data used in this study (Allison, 2005;Fitzmaurice et al., 2011). It also allows controlling for variables that have or have not been measured, which is required in this analysis (Allison, 2005). It must also be pointed out that the data set used in this study meets the criteria required by a fixed effect model, namely: the dependent variable for each individual must be recorded on at least two occasions; and the main independent variable, in this case, theoretical effectiveness score, must change in value on those two occasions. Table 6 sets out the variables that were used in the fixed effect model.

Scoring the Benefit Statement
The scores given to each benefit statement for effectiveness at informing members are recorded in Table 7 while the scores for effectiveness at persuasiveness are shown in Table 8. The theoretical effectiveness score for each benefit statement is recorded in Table 9.

Testing for the Effect of Theoretical Effectiveness of Benefit Statements on Contribution Rates
A paired t-test was carried on the whole sample. The test had a p-value of 0.13. The result confirmed that, at a 5% significance level, there was no significant difference between the average contribution rate for the members in the fund before and after the change in format of benefit statement.

DISCUSSION
This section sets out some considerations for areas in which benefit statements can be improved, based on the eight statements reviewed over time. It also explores possible reasons for the results obtained from the case study.

6.1
Theoretical Effectiveness Scores of Benefit Statements 6.1.1 The framework developed in section 3 has been used by different individuals at different points in time. Although easy to use, the scoring can be subjective, which means that where testing changes in member behaviour over time, the same person or group of people would need to rate the benefit statement before and after any changes.
6.1.2 It is notable that the six benefit statements studied from 2012 and 2013 typically had better scores for persuasiveness than for conveying information. This is counterintuitive given that informing members is a more basic aim. In addition, attempting to persuade based on limited factual information can discredit the communication, as discussed in ¶2.2.1. In the case study, there was a far better balance of the scores for informing and persuading. However, there is still scope for administrators and trustees to focus on improving how effectively the benefit statements convey factual information. 6.1.3 The following common shortcomings were noted in the benefit statements: member's investment choice not reflected in the investment return assumptions; no allowance for a cash lump sum at retirement, despite the popularity of cash lump sums in most funds; no sensitivity to assumptions shown; long sentences; very small font size used to convey important information; very limited use of emotion; and very limited use of personal information beyond the use of 'you' and 'your'.

6.2
Effect of Theoretical Effectiveness of Benefit Statements on Contribution Rates 6.2.1 Although in the case study the theoretical effectiveness of the benefit state ment increased, there was no corresponding significant change in the contribution rate.

7.2
Case Study to assess Whether More Theoretically Effective Benefit Statements improve Contribution Rates 7.2.1 The authors developed an analytical approach to test the effect of the theoretical effectiveness on the contribution rate. This methodology could be used on a larger scale to draw general conclusions. 7.2.2 Despite a significant improvement in the theoretical effectiveness of the benefit statement in the one fund studied, there was no corresponding increase in contribution rates. This shows that the improvements made to the benefit statements alone are insufficient to change member behaviour in this case. 7.2.3 There were a number of limitations with the case study. Only one fund from one specific industry was considered. This one fund had only 956 members which is quite small compared to other studies. The membership group was only tracked for 16 months. The lack of variability in the contribution rate made it difficult to fit a model to the data and statistically determine significance of the factors considered.
7.2.4 Another limitation is that it was assumed that all members read their benefit statement and that members actually receive their benefit statement in October. Delays on the part of the employer to distribute the benefit statements may have influenced the results.

Concluding Remarks
From a member's perspective, benefit statements are perceived as important (Van der Waldt & Van Heerden, 2007). The authors have developed a framework to help funds assess the effectiveness of their benefit statements with respect to informing and persuading members. However, it should be noted that in one case study, an improved effectiveness score was not sufficient to change member behaviour.

ACKNOWLEDGMENTS
Thanks are due to the trustees of the fund used in the case study and their fund consultant who supplied the data used. Thanks also go to Mr Dave Crawford who gave input into the framework. Two anonymous referees provided constructive feedback and are thanked for their diligence.