Researching literacy and numeracy costs and benefits : what is possible ?

Assessing the benefits of investing in literacy and numeracy and the costs of poor literacy and numeracy in Australia is hardly an easy or straightforward undertaking. However, there is growing interest in moving forward this area of research. Overseas literature, especially around health literacy, family literacy, financial literacy and the impact of literacy on employees and employers, identifies some useful frameworks and methodologies. The paper discusses findings from a recent Australian Council for Adult Literacy report for the National Commission of Vocational Education Research and suggests ways of moving forward in Australia.

There is then a strong case for a much wider public understanding of the social and economic benefits of improving literacy and numeracy and the costs of poor literacy and numeracy across different areas of life.Ultimately, better community understanding could lead to more resources being made available for adult literacy and numeracy.
In addition, more reliable information about a broad range of social and economic benefits and costs would help to ensure that data from the coming Adult Literacy and Lifeskills Survey (ALLS), to take place in Australia in 2006, is put to the best strategic use.In relation to numeracy, a considerable amount of international effort, including input from Australia, has gone into developing an appropriate framework for directly measuring numeracy.
Here we note that identifying and, where possible, measuring benefits and costs does not necessarily tell us how to go about reducing costs and enhancing benefits, except in the broadest terms.That is another issue.
In the interests of brevity, in this paper we use 'benefits and costs' or 'benefits and costs associated with literacy and numeracy', rather than referring each time to the longer 'benefits of investing in literacy and numeracy and costs of poor literacy and numeracy'.

The ACAL project
The paper is based on findings from a project carried out in 2004-5 by the Australian Council for Adult Literacy (ACAL) with funding from the Commonwealth government through the Adult Literacy National Project (Hartley & to function fully in society (e.g.see the review by Lonsdale & McCurry, 2004).In addition, individuals are increasingly Horne, 2005).Frameworks and methodologies available for determining and measuring adult literacy and numeracy benefits and costs across 'life domains' were identified through a literature review.The main focus is on literacy and social domains, such as health literacy and family literacy, rather than traditional economic areas, although the literature on employment and workplace literacy is briefly reviewed.The second part of the project involved a small number of consultations to explore in a preliminary way the level of interest and the capacity in Australia to move ahead with research into costs and benefits, and to identify the main issues that would need to be addressed to arrive at Australian estimates of benefits and costs.
The ACAL report reflects on the content of the literature, but does not explore in any detail benefits and costs associated with adult numeracy.We acknowledge that while literacy and numeracy are frequently linked in the literature, poor numeracy has a different overall set of consequences to poor literacy, although some may be similar.Nevertheless, the general frameworks identified and some of the representative methodologies used to assess benefits and costs of literacy, together with some of the general issues which emerged from the literature, are likely to be relevant and useful for exploring numeracy costs and benefits.

Measuring costs and benefits
At the outset, it needs to be said that there are undeniable challenges in assessing and measuring benefits and costs associated with adult literacy and numeracy.The impacts (and therefore the costs and benefits) of multiple literacies are complex, cumulative, and interactive.Measurement of costs and benefits is rarely straightforward.Researchers from different backgrounds approach the issue from different perspectives.Economists typically have an ideal standard of measuring everything in monetary terms.This allows them to undertake cost benefit analysis, i.e. to compare the balance of costs and benefits over time, and to calculate a rate of return on an investment in a particular intervention, program or policy.Often, both the economic return (based on output or income) and the social return will be calculated.The social return has a wider base than the economic return and includes monetary valuation of the costs and benefits to individuals, taxpayers and society at large.It goes without saying that it is often not possible to value costs and benefits in monetary terms, especially in the fields of education and health.Units of measurement may be in monetary terms; however, this is dependent on a range of factors such as the data available, the cost of collecting new data and the techniques that can be used to estimate costs and benefits.More often, costs and benefits will be measured in a non-monetary unit, e.g.rates of hospitalisation or smoking rates.
There are, of course, many issues to consider when it comes to reliably measuring costs and benefits and undertaking statistical analysis.A variety of approaches and estimation techniques have been used.In general, it is highly desirable to control for other variables that may affect accurate measurement of benefits and costs, as many socio-economic factors interact with literacy and outcomes such as health.

General frameworks
The most comprehensive available frameworks for investigating benefits and costs are concerned with adult learning, rather than with literacy per se.They are nevertheless relevant and useful because of the concepts used, the types of research they have led to and the issues they raise.Not surprisingly, the frameworks reinforce the importance of multi-disciplinary approaches and underline the need to integrate concepts of 'human capital' and 'social capital'.Literacy and social capital is a growing area of interest; however, currently, most work focuses on social capital outcomes of literacy learning, rather than on measuring the social capital benefits and costs associated with literacy.
The frameworks point to a number of factors that need to be taken into account when estimating costs and benefits.While they may appear obvious to people in the literacy and numeracy fields, they often require sophisticated methodologies and measurement techniques.Such factors include the following: benefits of learning (and literacy) can be 'sustaining' as well as 'transforming' (allowing individuals and communities to continue and improve doing what they, as well as changing them); benefits can be individual and collective; they are cumulative and dynamic in nature (Schuller et al., 2001;2004).Research also shows that benefits and costs may vary according to, for example, gender and stage of life, increasing the complexity of measurement.

Methodologies
Some studies use large datasets, such as the IALS data and various longitudinal cohort studies, to measure economic and social benefits and costs of literacy to individuals and/or the economy and society.

The international adult literacy survey (IALS)
IALS data has been used to examine the link between literacy and labour market outcomes, earnings and the returns to different levels of education, and immigrant earnings.Analysis of IALS data (Williams, 1999, cited in Hagston, 2002) has also shown that receipt of welfare benefits, health, criminal activity, and community participation are linked with literacy skills and educational achievement.Health has been a particularly fruitful area of investigation.Although health outcomes were not directly measured in the IALS, analysis of IALS findings has shown that high levels of literacy are associated with better health outcomes, for example, higher life expectancy and healthier habits and lifestyles (OECD & Statistics Canada, 2000).Similarly, Roberts and Fawcett's (1998) analysis of Canadian IALS data found that people with lower literacy levels were more likely to be at higher health risk, with an increased effect on senior citizens.
In Australia, Chiswick, Lee and Miller (2003) used data from the IALS to examine the effects on labour market outcomes of literacy, numeracy and schooling.It was found that approximately half of the total effect of schooling on labour force participation and on unemployment can be attributed to literacy and numeracy skills.
Findings from a recent study by Coulombe, Tremblay and Marchand (2004) using IALS data are quite compelling.Their conclusion, that a rise of 1% in a country's literacy score relative to the international average is associated with an eventual 2.5% relative rise in labour productivity and a 1.5% increase in GDP per head, has attracted quite a lot of comment.The study is notable because it is the first to identify a clear and significant association between a country's investment in human capital and its economic growth and because it uses a direct measure of skills, i.e. literacy levels as measured by the IALS, to identify this link.Previous studies which focused on educational qualifications had failed to prove a link between human capital and economic growth.

Longitudinal cohort studies
In the UK, two longitudinal cohort studies, the National Child Development Study which began in 1958 and the 1970 British Cohort Study, have been popular data sources for measuring non-economic and economic costs of poor literacy, and benefits to individuals and society of improving literacy, no doubt because of the rich data they include on both economic and social outcomes.
A representative study, which measured both the economic and non-economic impacts of improving adult literacy and numeracy skills, is that of Bynner, McIntosh, Vignoles, Dearden, Reed, and van Reenen (2001).They used data from the two British longitudinal studies and from the UK Family Expenditure Survey and Family Resources Surveys, and employed statistical modelling techniques.As far as non-economic outcomes are concerned, individuals who increase their literacy and numeracy levels were shown to: suffer less from poor physical and mental health; be less likely to have children experiencing difficulty at school; be more likely to be active citizens, as shown by voting behaviour and expressing interest in politics, and are more liberal and less discriminatory in their attitudes.
In Australia, the Longitudinal Survey of Australian Youth (LSAY) dataset has been used to measure some economic outcomes.Marks and Fleming (1998) found that low school achievement in literacy and numeracy was consistently associated with youth unemployment, with effects continuing through to the age of 33.Similarly, Gleeson (forthcoming) has been using the LSAY dataset to examine the economic returns to training for adults with low levels of literacy and numeracy.The use of longitudinal cohort studies in the UK, which include a range of social and economic indicators, points to the possible use of similar longitudinal study data in Australia.This potential was explored in the consultations for the ACAL project.

Health literacy
A useful framework for investigating health literacy is Nutbeam's (1999) schema.It describes functional, interactive and critical health literacy and places health literacy in the broader context of health promotion.Functional health literacy is broadly defined as the ability to read and comprehend medical information and instructions.Interactive health literacy refers to the development of personal skills that improve capacity to act independently and improve motivation and self confidence to act on advice received; critical health literacy refers to personal and community empowerment to act on the social and economic determinants of health.
A considerable amount of work on health literacy benefits and costs has been carried out and some comprehensive summaries and discussions of research exist (e.g.Berkman et al., 2004;Rootman & Ronson, 2003;Shohet, 2004).Most studies to date have investigated either the benefits or the costs of different levels of functional literacy from a medical perspective, although there is growing recognition of the need to go beyond functional literacy.
Higher health literacy has been linked to, for example, knowledge and understanding of such things as mammography, cervical cancer screening, emergency department discharge instructions, smoking, contraception, HIV, asthma, and post-operative care.On the other hand, representative studies have found associations between lower literacy and, for example, higher risk of hospitalisation, not having had a Pap smear or mammogram in the past 2 years, not having a flu injection, and higher rates of depression in various populations.
Studies relevant to benefits and costs of health literacy are varied and indicate the wide range of potential costs and benefits of health literacy.Nevertheless, some of the impacts identified are based on the knowledge and behaviour of individuals in the research sample rather than on actual health outcomes.Most investigate aspects of physical health or specific health issues such as asthma or diabetes, although the literature does include studies of 'mental health literacy'.
We did not identify studies which have valued the benefits of improving health literacy or conducted a cost-benefit analysis of investing in health literacy.Studies are generally not cheap, especially if they use face-to-face interviews to measure individuals' literacy levels and to collect data on outcomes.The extent to which findings can be generalised across different populations and different environments needs further investigation The most well-known tools for assessing health literacy are the Rapid Estimate of Adult Literacy in Medicine (REALM), Test of Functional Health Literacy in Adults (TOHFLA) or S-TOHFLA (shortened version of TOHFLA), and the Wide Range Achievement Test (WRAT).Although quite widely used, they have some limitations and have not yet been validated in different cultural and social contexts.
In Australia, health literacy has not received the same attention as in some other countries.Nevertheless, there is evidence of increasing interest from some medical researchers, e.g.Buchbinder, Hall, Grant, Mylvaganam and Patrick (2001) have investigated the readability of patient information used by Australian rheumatologists; Griffin, McKenna and Tooth (2003) have examined written health education material in the area of occupational therapy.Health literacy is being recognised as a factor which affects decisions to participate in screening programs.In common with the overseas research, most studies focus on functional health literacy, although there is some interest in broader issues such as communication between health professional and patients.

Financial literacy
Definitions of financial literacy generally include elements of skills, knowledge and understanding, for example: (Financial literacy is) personal financial knowledge and skills…Financial literacy involves the ability to understand financial terms and concepts and to translate that knowledge skillfully into behaviour…Literacy implies knowledge of the terms, practices, laws, rights, social norms, and attitudes needed to understand and perform…vital financial tasks.It also includes the fact that being able to read and apply basic math skills is essential to making wise financial choices.(Jacob, Hudson & Bush, 2000).Schagen (1997) defines it as 'the ability to make informed judgements and to take effective decisions regarding the use and management of money'.
However, definitional problems and lack of a well established framework for financial literacy remain.Some argue that it should be seen as an essential life skill.The place of numeracy in relation to financial literacy is not well defined; however is it clear that most definitions incorporate elements of numeracy.
Studies from the US have measured the benefits of improved levels of adult financial literacy gained through workbased financial education programs or through high-school based programs.A few studies have extended this to look at the return on investment to employers of investing in workbased financial literacy programs.
Important measurement issues include: accounting for the interaction between financial literacy and low income, the influence of attitudinal, psychological and 'lifestyle' factors, and the effect of different life stages and associated priorities.The most robust approaches to measuring the benefits of investing in financial education include using pre and post-test designs, and using control groups of people who do not receive financial education.
There is limited research investigating the relationship between the financial literacy levels of employees and general literacy and numeracy skills, and the impact of financial literacy levels on physical and mental health.
Research relating to costs and benefits focuses substantially on those who are employed; few studies explore in detail costs and benefits of financial literacy for unemployed people.
In Australia, research for the ANZ Bank (Roy Morgan Research, 2003) and for the Commonwealth Bank (Commonwealth Bank Foundation, 2005) surveyed literacy levels within the population.Two reports include estimates of costs of poor financial literacy.The Consumer and Financial Literacy Taskforce (Commonwealth of Australia, 2004) modelled the effects of 'bad' financial decision making over the course of a person's life.The Commonwealth Bank Foundation's (2005) study modeled the effects of improving financial literacy levels on individuals and the economy.The increase modeled in the latter was a relatively modest target, i.e. improving the scores of the 10% of the population with the lowest levels of financial literacy on the financial literacy questionnaire, over a period of ten years.It was estimated that this would increase the average annual income of people with the lowest 10% of scores by $3 204, increase Australia's GDP $6 billion and create 16 000 new jobs.It was also estimated that improving the financial literacy of this group would reduce the incidence of persistent sleeping difficulty amongst the Australian population by 2% and the incidence of regular smoking by 2%.

Business, employers and employees
There is a sizeable literature that demonstrates the returns to employers of investing in workforce training generally (i.e.improvements in firm productivity and in some cases higher levels of innovation and/or better financial performance); however studies that focus on costs and benefits of literacy and numeracy are far fewer.
Literacy benefits and costs for small business owners and self-employed people (a growing group in Australia) is a largely unresearched area, although there is some evidence of growing interest.Overall the research indicates, perhaps not surprisingly, that larger rather than smaller companies are making the most investment in workbased training in the literacy area.UK studies suggest that most small business owners are largely unaware of poor literacy and numeracy skills.While there is an obvious link between financial literacy and small business, the importance of broad literacy and numeracy skills is equally important.Some of the methodologies used in exploring literacy and larger enterprises are likely to be useful, although our consultations suggested that the small business sector differs in significant ways from larger enterprises.These differences need to be taken into account in any research in the small business area.Flexible and multiple research approaches are likely to be most useful.
Methodologies used to measure the benefits of improving literacy and numeracy amongst the workforce have typically focused on cost savings and/or productivity gains to the company, although some studies look more widely and include employee focused outcomes such as levels of promotion and rates of absenteeism.The issue of ideal methodologies based on large datasets (panel data is the most ideal) versus achievable methodologies based on availability of data and costs of new data collection is an important consideration.
Relatively few studies include measures of factors that impact directly on employees and that also indirectly influence company profits, i.e. employees' job satisfaction, internal promotion, absenteeism, access and take-up of further training and education.
In Australia, while workplace education and the skills of workers have received considerable attention, the most directly relevant study is that by Pearson (1996).He found that language and literacy training was considered to have had a positive effect on five aspects of the workplace: direct cost savings; access to and acceptability of further training; participation in teams and meetings; promotion and job flexibility; and the value of training (which included issues such as worker morale, confidence to communicate etc.).The study included respondents' quantitative estimate of savings to their companies based on the positive impacts identified.

Family literacy
We found no research studies that have measured the costs of poor family literacy, although there are a fairly large number which have measured the benefits to adults and/or children from participating in family learning programs.Many of the measures used have focused on outcomes such as changes in confidence and attitudes towards literacy and learning by adults and children, or on reading gains.
The few studies that have managed to measure a wider range of benefits have tracked parents and/or children over time, as many of the benefits of family literacy, such as educational and employment gains for adults and educational and developmental gains for children, can only be adequately measured once time has passed.
Longitudinal cohort studies are relatively expensive and require established family literacy programs to be in place so that a sufficient number of participants can be identified and followed-up.These types of programs do not generally exist within Australia.Ideally, a group of family literacy program participants would be compared with a control group who had not participated, further increasing research costs.

Taking measurement of costs and benefits forward in Australia
We concluded that there is sufficient interest and a small research base in Australia for obtaining better information about social and economic costs and benefits in relation to three areas -health literacy, financial literacy and literacy and numeracy and small business.There is quite strong interest in financial literacy from the Commonwealth government and from financial institutions.Health literacy does not at present have any obvious champions but there are interested researchers and a substantial body of overseas research on which to draw.There is a strong case to be made for the importance of literacy and numeracy in relation to small business, and some evidence of interest in the area.
The consultations underlined the importance of a number of factors: convincing government and other research funding bodies that adult literacy and numeracy costs and benefits is an important individual, community and national issue; getting commitment from researchers and others working in various social domains; and discovering what meeting points are possible between the various stakeholders.The following options for the future were suggested: setting up small collaborative cross-discipline working groups to explore common ground and jointly seek funding for further research; further exploration of the possibilities of 'buying in' to existing Australian longitudinal studies in order to measure the range of social and economic costs associated with literacy and numeracy; further targeted consultations with a wider range of people interested in health literacy, financial literacy and literacy and small business; wide dissemination of the report so more people are aware of the substantial body of research in the area, as well as the research issues; promotion of the strategic use of information from the 2006 Adult Literacy and Lifeskills Survey (ALLS).
Collaboration across sectors is crucial to take forward the assessment and measurement of social and economic costs and benefits associated with literacy and numeracy.