Developing a model for identifying students at risk of failure in a Developing a model for identifying students at risk of failure in a first year accounting unit first year accounting unit

This paper reports on the process involved in attempting to build a predictive model capable of indentifying students at risk of failure in a first year accounting unit in an Australian university. Identifying attributes that contribute to students being at risk can lead to the development of appropriate intervention strategies and support services. In this study, regression analysis was used to model the impact of individual factors on grade performance based on a review of the literature and using data extracted from a university’s student information database for all students who completed a first year accounting unit in one semester. The overall findings were that while the explanatory power of the model was poor, a number of variables were found to have a significant impact on performance. These variables included: younger students, males, those enrolled in non-business majors, and those with English as a second language. Further research in this area is warranted with the overall aim of reducing student failure and subsequent student attrition as well as developing appropriate intervention strategies.


Introduction
Studying at university affords students the opportunity to prepare for a career in a designated profession as well as provides them with the opportunity to study in disciplines that are of interest to them.Many more students now attend university than was the case in the past (Dobson and Skuja, 2005).Students commencing university however, generally exhibit a wide range of characteristics and backgrounds.They often vary in age, work experience, educational background, culture, self-esteem, ethnicity, social status, mode of study (full-time v. part-time, on-campus v. external), skill sets (mathematical ability, communication skills, interpersonal skill etc.), command of the English language (English as a first language v. second language), proximity to campus etc.
Many first year students are also faced with dealing with an educational environment which is new to them and brings uncertain expectations concerning both learning and social situations.It is not surprising therefore, that a larger proportion of first-year, on-campus students in Australia are not fully prepared for tertiary education, are uncertain of what is expected of them, and are not motivated to achieve in their studies (McInnis, James, and Hartley, 2000).Compounding the problem is that students entering university often do not have a track record of academic success or have not studied for some years.Many universities cater for these potential students by offering 'alternative' pathways to enter university.This includes, for example, allowing students to enroll at university based on submission of a folio of work, completion of an interview or an audition.
The movement to more flexible entry requirements by universities therefore results in the admission of students with different educational backgrounds, experiences and academic potential, not to mention different social and cultural backgrounds.The challenge for many universities is to not only recognize this diversity of needs, but to cater for this changing and heterogeneous population of students.This includes introducing strategies and implementing interventions which support student success (McKenzie and Schweitzer, 2001).In this sense, the first year experience is critical in terms of retention, satisfaction and graduate outcomes.

Purpose of this Research
Taking into account the importance of the student experience for first year students, the purpose of this research project was to identify students at risk of failing a first year introductory accounting unit as part of a Bachelor of Business degree at an Australian University.The primary research question was 'Can student characteristics and background information as provided in the university student database generate a reliable prediction model which could identify students at risk of failure in a first year accounting unit?'In identifying students at risk at the point of enrolment based on information obtained from the university student record system, our overall objective was to improve the likelihood of success for those students deemed to be at risk of failure.Students deemed 'at risk' of failure and of not continuing with their studies would then be targeted as being in need of assistance.This would be accomplished by the development of appropriate and focused interventions, pathways, policies and procedures.Identifying characteristics of students likely to result in academic failure can act as an 'early warning system' to identify students in need of remedial assistance at an early stage of their academic career.
Identifying students at risk is important in order to improve the targeting of pro-active interventions and support services for them.It is common knowledge that students do not always seek help while trying to adjust to the demands of university life or when they have academic difficulties -some of them only seek assistance when their difficulties are obvious and advanced (Grebennikov and Skaines 2009, p. 67).
The development of academic predictors of academic success is a critical issue for educators (Golding and Donaldson, 2006).This is particularly relevant in the first year where both anecdotal and research evidence suggests that there is a positive relationship between student academic performance and retention (Grebennikov and Skaines, 2009).As noted by Scouller, Bonanno, and Krass (2008), enhancing student performance in the first year and increasing retention rates have (therefore) become important priorities for universities, resulting in a focus on support, especially for students 'at risk'.

Student Attrition
Significant attrition among first year tertiary students is an international phenomenon (Whitehead, 2012).Australian universities are paying increasing attention to reducing student attrition, because it results in considerable costs to the student (e.g., fees, opportunity costs, emotional costs); costs to the institution (e.g., loss of fees, recruitments costs, tuition fees); as well as the fact that in Australia, the Department of Education, Science and Training (DEST 2005) has used the commencing bachelor student attrition rate as one of the performance indicators in allocating its Teaching and Learning Performance Fund (Cao and Gabb, 2006).
Students leaving university represent a waste of university resources especially in an environment of limited financial and general resources (Stillman, 2009).Attrition is also important because it damages the reputation of a university and creates long term implications for attracting new students (Ozga and Sukhnandan, 1998); attrition represents a loss of income for universities and, all things being equal, a failure to accomplish their educational mission (Bean, 1990).Low attrition rates also impact upon the image of a university from the perspective of stakeholders, parents and students.
Due to the linkage between funding and retention rates, many universities address this issue by creating 'Student Retention Officers', by creating committees at both the faculty and university level and by providing support mechanisms and intervention programs for students 'at risk' of failing and leaving degree programs.In this sense, early identification of students at risk of failure should be part of first year monitoring and assessment procedures, in order to provide students with the support they need in a timely fashion (Scouller, Bonanno, Smith and Krass, 2008).
Identifying students 'at risk' not only has implications for the university in terms of retention, but also comes at a cost to students.Students wishing to progress in their degree will be hampered by having failed a unit.The consequences of failure are additional costs, delays in completion of degree, possible change in major, a possible decrease in self-confidence, and inability to progress to second year (accounting units) where introductory accounting for example, is a pre-requisite.In addition, involuntary withdrawal from university because of academic failure or inability to cope with the demands of the educational system lowers self-confidence and self-esteem and potentially represents a negative lifelong economic impact.Students from low socio-economic backgrounds face additional hurdles in achieving academic success.These include: loss of confidence (possibly as a result of inadequate educational preparation), isolation, withdrawal of emotional support from family and peer group, lack of role models and poor study environment and resources (Ramsay, Tranter, Charlton, and Sumner, 1998).

Literature Review
A considerable body of research has addressed the topic of student retention rates as well as the first year experience at university with research suggesting that retention is strongly influenced by academic performance (O'Byrne, Britton, George, Franklin, and Frey 2009).A number of factors associated with student attrition have been well researched, with some findings found to be consistent across institutions and countries however, there has been considerable variation in other findings.Factors that have typically been the subject of investigation have included socio-demographic variables (e.g., gender, age, language, geographic location, socio-economic status, and country of birth); prior experience variables (e.g., prior academic achievement) and institutional variables (e.g., field of study, basis for admission, type of attendance, and employment status) (Cao and Gabb, 2006).
In terms of studies investigating the background characteristics of students (e.g., previous qualifications, gender, race, age, social status) the extant research is not in agreement.As yet, no one core set of significant variables has been identified that predicts attrition.Only entry point score, part-time v. full-time, and gender, are variables consistently recognized as important, but how significant they are is widely debated (Roddan, 2002) In a study of nearly 9 000 undergraduate students commencing at the University of Western Sydney (UWS) in 2004, students leaving UWS without applying to other institutions were found to be significantly higher for those with a low grade point average, part-time students and mature-age students (with work and family commitments), and English speaking background.Low grade point average was found to be associated with such student characteristics as non-English speaking background, male, and low socio-economic status.This research also confirmed what other studies have consistently shown, that non-English speaking background (NESB) students have significantly higher probability of academic differences compared to English speaking background (ESB) students (Grebennikov and Skaines, 2009 p. 67).
In their comprehensive study, Cao and Gabb (2006) examined the characteristics of 12,500 students at domestic bachelor level commencing in 2002, and 2004 at Victoria University with regard to a range of socio-demographic variables associated with student attrition.Their variables and results are summarized in Table One As noted by Grebennikov and Skaines (2009), much of the theory and research exploring the individual, social and organizational variables contributing to student retention consistently reports the following key predictors: previous academic performance and educational qualifications, university entry score, previous course performance as students move through their studies, attendance mode (full-time students have higher retention rates), admission type (proportionally more current school leavers retain in higher education compared to mature-age students); residence (international students have higher retention rates than local students), socio-economic status (SES) with the higher the SES the higher the retention rates), type of housing (on-campus residents have higher retention rates than non-residents), participation in orientation and similar programs (higher retention among participants), student awareness about course or institution before enrolment, and student personal adjustment and involvement in campus life.Less agreement among researchers has been noted regarding the effects of student gender, age, employment, language background, ethnicity, and field of education on student retention and completion of studies.Unfortunately, they suggest that these findings may differ from a specific set of variables contributing to student success in each given university (p.60).
Research which has examined the topic of predictors of student success at university has traditionally examined two main factors: (a) academic factors and (b) non-academic factors (e.g., psychosocial, cognitive and demographic predictors) as suggested by Grebennikov and Skaines (2009).Three factors frequently mentioned in the literature as being related to academic success include previous academic achievement, self-efficacy, and preferred learning styles (Burton and Dowling, 2005).
In their review of the literature concerning factors contributing to academic performance, Grebennikov and Skaines, (2009) noted that there were a number of predictors found to have had a high level of consensus among researchers.These included: previous academic performance and educational qualifications, university entrance score, previous course performance as students move through their studies, gender (women show higher academic achievement than men), age (students in their late 20s and early 30's are more likely to perform better than younger or older students), socioeconomic status (SES) -the higher the better for achievement.On the other hand, lesser agreement has been found amongst researchers concerning possible predictors of student academic performance in respect of such factors as: admission type (school leavers v. mature students), attendance mode (full time v. part time), field of education, employment commitments (full-time, part time or unemployed), level of student employment in campus life (measured by various indicators), language background, and ethnicity.
In summary, studies concerned with factors that influence retention, and to a lesser extent the academic performance of students in higher education have a long history.The purpose of these studies has been to understand what is related to, or predicts, attrition or poor academic performance and to use this information to design appropriate interventions.A wide range of factors has been found to predict or influence retention and performance.However, factors found to be predictive in some studies are not always predictive in others (often due to different methodologies) but interestingly, even studies using the same methodology have shown differing results from different universities so that in general, the results of particular studies cannot be generalized to other environments (see Scott and Smart, 2005 as reported in Rienks and Taylor, 2009).
A particularly good example of the non-generalizability of previous research findings concerns studies which have focused on the relationship between tertiary entrance scores and success at university, which have provided mixed results.Many studies for example, have shown that previous academic performance (as demonstrated by final year high school performance) is a significant predictor of university performance, or at least demonstrates a strong link (see for example, Dobson and Sharma, 1993;Evans and Farley, 1998;McKenzie and Schweitzer, 2001).Levy and McMillan, as reported in Tomazin, (2003) however, found that students with an average university entrance score showed no positive correlation between entrance score and university performance.Dobson and Skuja (2005) found high university entrance scores to be a good predictor, though it was not a good predictor of performance in a number of disciplines, including business.

Intervention Strategies
Identifying those attributes that contribute to student academic performance can assist when developing intervention strategies and support services for students who perform poorly in their studies at an earlier stage (Affendey, Paris, Mustapha, Sulaiman, and Muda, 2010).There are two broad approaches to providing extra academic support to help students succeed during their first year at university.These are either targeting all students who wish to participate in extra learning opportunities or alternatively, targeting only those students deemed to be at risk (O'Byrne, Britton, George, Franklin, and Frey 2009).A program targeting only students deemed to be at risk could potentially be discipline-based (e.g., through additional workshops) and could include different academic support programs (e.g., peer mentoring) with equity of opportunity considered so that all enrolled students could participate if they wished.
Identifying 'students at risk' has important implications for the development of appropriate intervention strategies.McInnis and Krause (2002) emphasize the need to utilize a range of points of intervention to successfully induct students into the tertiary education experience.These include at the point of recruitment, at enrolment stage, during the first year, and at various assessment points.Possible strategies could involve the use of study groups, learning communities and mentoring programs (e.g., with staff or with senior students) or networking strategies Other implementation strategies could include: providing activities which foster student-student and student-staff interaction; linking students deemed at risk with available support services based on referrals (monitoring) by first year unit co-ordinators and other relevant university officials; encouraging collaboration with academic and general support staff to assist in maintaining satisfactory academic progress throughout the semester for students deemed at risk; appropriate and focused counseling; the development of appropriate orientation activities and transition strategies targeting students at risk; instigating a mentor support system by perhaps linking new and experienced students from similar backgrounds (SES, ethnicity, language); and emphasizing the role of key academic staff, counseling staff, learning skills advisors etc.
There is substantial evidence to show that extra support provided by universities does have a significant effect on student performance and retention.Such support significantly improves pass rates, exam grades and levels of retention.The evidence highlights the fact that targeting students for intervention is beneficial and hence, research of this nature is worthwhile (Roddan, 2002).

Research Method
A quantitative approach was adopted, with a number of multivariate models developed to explain two performance outcomes: numerical final mark, and binary pass/fail indicator.The predictive ability of each of these models, with respect to our explanatory variables, was measured.

Data Collection
Personal and course/enrolment details were accessed from the university's student information database for all students who completed the first year accounting unit (ACC1100) in the first semester of 2010.A total of 325 students completed the unit; Table 2 summarizes the information that was recorded for each student.First attempt at ACC1100 First attempt at accounting unit ACC1100?Yes(1) or No(0).
First semester enrolled at University First semester enrolled at ECU? Yes (1) or No (0).Prior literature suggests that these variables are likely to be important but neither was recorded in the student information systems available.Additional data might be gathered via means of student surveys, but these were beyond the scope of this project.

Results
The average profile of the students who passed the introductory accounting unit is compared with those who failed, and with the group as a whole, in Table 3 below.The two groups show similar characteristics with respect to age, gender, attendance type, study mode and parent's education and any differences here are not statistically significant.However, four of the variables differentiate between the groups to varying degrees; these are: Course, First Attempt, First Language and First Semester.
The variable providing the most prominent differentiation between the two groups is First Language.Students with English as their first language make up 81% of the Passed group but a much lower 65% of the Failed group, indicating a significantly higher pass rate for English First Language students (68% compared with 48% for those with another first language: (chi-squared=10.792,p=0.001).The pass rate for those studying ACC1100 for the first time is significantly higher than those repeating the unit.This can be seen in the fact that First Attempt students made up 91% of the Passed group but only 81% of the Failed group (chi-squared=6.844,p<0.01).
Similarly, students taking a Business course have a moderately higher pass rate (chi-squared=4.999,p=0.025), as do students in their first semester at ECU (chi-squared=5.401,p=0.02).The pass rate for those students taking a combined Arts/Business course is significantly lower than those taking other courses (chi-squared=7.415,p<0.01).Final mark is significantly (95% confidence level) positively correlated with Age, Course, First Attempt, First Language and First Semester.Grade (Pass/Fail) is significantly (95% confidence level) positively correlated with Course, First Attempt, First Language, and First Semester.The regression model with 'final mark' as the dependent variable thus suggests an explanatory power of only 10.9%, despite the statistical significance of the constituent independent variables.The intercorrelations between the explanatory variables are so low (see Appendix) as to suggest that multicollinearity is not a problem.The low explanatory power is indicative of the omission of important variables in the regression.However, the equation suggests that candidates most at risk of failure are:  Male  Younger (i.e., < 25)  Enrolled in a non-business major  Studying with English as their second language, and  Deferring completion of ACC1100 beyond their first semester of study.
These outcomes might be used to complement an intervention strategy aimed at locating those students most likely to be at risk of failure.
A second analysis was conducted using a binary pass/fail as the dependent variable, rather than the percentage score.Both linear discriminant analysis (MDA) and logistic regression were used with two groups defined by the Grade (Pass/Fail) variable.The resultant models were very poor, with explanatory power never exceeding 67% for the DA model , or a pseudo R 2 for the logit model of only 8%; the models are not reported here.

Conclusions/Limitations
Based on the statistical analysis carried out in this project, it is unlikely, that the findings would be generalizable for predictive purposes beyond the study site.However, the influential variables (though not necessarily the extent of their influence) have been shown to be common across numerous locations.The findings of this study are largely consistent with those of other studies in this regard, and provide some confirmation of the variables that should be monitored with a view to potential intervention.
The absence of easily accessible 'Entrance Score' data is considered as a severe limitation in this study, since their inclusion would most likely have improved the explanatory power of the derived models.In addition, some of the findings provide a basis for further examination in themselves.For example, we need to know if differences in outcome associated with campus are attributable to class size, quality of teaching or other, currently unmeasured, variables.
As noted by Fraser and Killen (2003), there is ample evidence in the literature on teaching and learning to suggest that teaching strategies, student motivation, approach to studying, interaction student and academic and social systems of the university, cultural expectations, psychosocial factors, and numerous other factors (e.g., interest in the course, motivation, self-discipline, setting appropriate goals, time management, and effort) are likely to influence student success at university.In other words, there are complex processes at hand that influence student success and failure at university.It is clear that further research is warranted in this area.

Table 1 .
. Factors Associated with Attrition Socio-demographic Variables ResultsGender males only a slightly higher attrition rate Age students 20 to 24 consistently demonstrated lowest attrition rates (younger and older similar rates)Language Background similar results for English as first language and second language Socio-economic status (SES) students with low SES has lowest attrition rates in all three years A stepwise regression revealed that progress rate, type of attendance and region were the most powerful indicators of attrition (p.11).

Table 2 .
Description of Variables Recorded for Each Student Notable omissions from this group are Tertiary Entrance Score and any indication of prior Grade Point Average.
* Only 208 students (64%) had their Parent's highest education level recorded

Table 4
below looks at the final mark for ACC1100 and compares the mean marks at each level of each binary or categorical variable.Two of the variables show differences that are statistically significant at the 99% confidence interval: Course and First Semester.More moderately significant differences in final mark (95% confidence interval) are shown with the variables: First Attempt, First Language and Parent's Highest Education Level.Differences in final mark for the variables: Age (25 or over), Attendance Type (Full-time), Gender and Parent's Education Level (Bachelor Degree) are significant only at the 90% confidence interval level.

Table 4 .
Final Mark for ACC1100 -Variation Across Key Variables

Table 5
gives the correlation between each of the variables final mark and grade (pass/fail) and all other variables of interest.Categorical variables with more than two categories have been replaced with dummy (indicator) variables for each category.A full correlation matrix is located in the Appendix.

Table 5 .
Final Mark and Grade (Pass/Fail) -Correlation with Key Variables

Table 6
below gives the result of fitting linear regression models to the Mark variable.The model shown is the 'best fit' model in terms of a trade-off between the highest R2 and the fewest variables.