Theory, evaluation, and practice in widening participation: A framework approach to assessing impact

TheEnglishhighereducation(HE)systemisdeeplystratified,withyoungerstudentsfrommoreprivilegedbackgroundscomprisingthemajorityofthestudentpopulation.Overthe last15yearsconsiderableinvestmenthasbeenmadetowidenparticipationbutattemptsto evaluatetheseinitiativesanddemonstrateimpacthavepresentedamajorchallengefortheHEsector.Thispaperexploresthedevelopmentandapplicationofaframeworkforevaluatingandresearchinguniversity-ledinterventions.DrawinglargelyonthetheoreticalworkofBourdieuitprovidesabasisfordesigningandevaluatingprogrammesandactivitiestodevelopstudentculturalcapitalandhabitus,andfosteragencyandasenseofbelonginginHEsettings.

The framework presented in this paper encapsulates this transformative process, operationalizing capitals, capacities, and practices that students develop into a set of learning outcomes organized under overarching aims.The framework is designed to be accessible to policymakers, academics, practitioners, and non-specialists alike, and this paper focuses on its developmentandapplicationtoaccessandentrytoHE.
UntilrecentlythemainfocusofOFFAandHEFCEmonitoringexerciseshasbeencollecting evidence of activity, requiring HEIs to report expenditure, set and agree output targets, and demonstrate how activity meets those targets. For most HEIs, establishing robust internal reportingsystems,combinedwiththedevelopmentofpolicies,systems,andactivities,represented aconsiderableinstitutionalundertaking.However,thecallforgreaterevidenceofimpactand effectiveness has gathered momentum within government and among HE senior managers, particularly following the introduction of higher university fees in 2012, when expenditure onWP was linked to fee income. OFFA's guidance for producingAccessAgreements (OFFA, 2015a; OFFA, 2015b) and Monitoring Returns (OFFA/HEFCE, 2014) represent something of astepchangeintarget-settingandrequirementsformonitoringandevaluation.Withgreater acknowledgement of institutional differences and the issues facing a wider range of social groups,astrongemphasisondemonstratingimpacthasmovedevaluationfarbeyondloggingthe numberofparticipantsengagedin'outreach'events.ManyHEIsnowemploystafftoevaluateWP activitiesinordertocaptureanddemonstrateimpacttoOFFAandHEFCE,butalsotogenerate informationfortheirownmanagerialpurposes.

Effectiveness of WP activities and interventions
Aswehaveseen,WPresearchandevaluationhavebeencriticizedfortheirlackofrigour.Gorard andSmithsummarizetheissueasfollows: Those advocating specific interventions often claim success for them, but most interventions havehadnorigorousevaluation.Weencounterednorandomisedcontrolledtrialsorsimilar.This makesitdifficulttojudgethesuccess,orotherwise,ofanyattemptstowidenparticipationin theshortterm. (GorardandSmith,2006:116) Demonstrating causal links betweenWP interventions and increased participation of underrepresented groups in HE is challenging.The issue of attribution bedevils many social justice initiatives,andattemptingtoaddressthesethroughquantitativemethods,controlgroups,and longitudinalstudiesarefraughtwithmethodologicalissuesandcontradictions (Copestake,2014). For example, let us consider two typical university outreach activities: a GCSE mathematics 'masterclass' and a 'return to study workshop' for prospective mature students. However engagingandsuccessfulthe'masterclass'mightbe,itcouldonlyeverbeoneelementcontributing toanimprovedgradeatGCSEmaths.Toclaimgreaterimpactwouldnegatetheworkofschools, teachers, and other cultural factors. Similarly, a'return to study workshop' might be deemed successful in meeting its learning objectives, but its participants may not progress to HE for practicalreasons,suchasfinanceorcaringresponsibilities.
Larger data sets and sample sizes can address some of these issues and the HEAccess Tracker(HEAT)database,designedtoassessthelong-termimpactofWPinterventionsthrough trackingparticipants'progressionpatterns,isanexampleofthisinpractice.However,practical andpolicyimperativesinthefluidanddevelopingfieldofWPdemandbothamoreimmediate andamorenuancedresponse.
Tosummarize,whileresearchhaselucidatedsomeofthefactorsbehindlowparticipation and attainment among groups under-represented in HE, its findings are not informing the design,evaluation,andmonitoringofinterventions,orbeingcomprehensivelydisseminatedto practitioners. Practitioner research and evaluation has focused on the successful delivery of activities,withlittledirectengagementwithresearchliterature,althoughday-to-daypractices reflectatacitknowledgethatisevidencedbyitsfindings.
Monitoring processes, linked to expenditure, have been useful tools in the generation and recording of activity, but have had limited engagement with metadata such as HEFCE's analysesoftrendsinparticipation (HEFCE,2010;HEFCE,2013),withtheoreticalresearch,or with practitioner evaluations.The framework presented here seeks to overcome this lack of connectivityandprovideamorecoherentbasisforassessingtheimpactofWPinterventions.
TheNERUPIFrameworkispredicatedonaculturalmodelofwideningparticipation,which locates interventions within a contextual field of engagement where student habitus and the institutionalhabitusesofschoolanduniversityintersect.Inthisrespect,theframework'semphasis onstudents'habitusandcapitalisunderpinnedbyanacknowledgementofHEIs'responsibilities to deliver'enabling' interventions, which facilitate institutional reflexivity as well as personal changeforparticipants.
Theirstudyhighlightedthedifferentialchallengesworking-classparentsfaceinsupportingtheir childrenandhowthiscanimpactonexaminationgradesandopportunitiesforprogressiontoHE. Slow progress in widening access to the most selective HEIs, combined with increased pressuretomeetAccessAgreementtargets,hasledsomeHEIstodevelopinterventionswith the explicit aim of raising attainment. Universities are well placed to contextualize subject knowledge,demonstratehowtheschoolcurriculumrelatestoresearchandcareers,andprovide accesstofacilitiesandopportunitiestoengagewithacademicstaff.
When developing the NERUPI Framework, subject'knowledge' was differentiated from academic 'skills' and incorporated under two distinct aims.This acknowledged that certain groupsofstudentsmayhavehadlimitedopportunitiestodevelopskillssuchasessaywriting and independent research, which are essential for successful university study. In some cases, suchasmaturestudentsreturningtostudy,thedevelopmentofacademicstudyskillsisamajor partofpre-andpost-entryactivity.Thefourthaimintheframeworkisconcernedwithskills developmentandbuildsonBourdieu'staxonomythroughthenotionof'skillscapital',whichis acquiredanddemonstratedthroughacademicpractice.ItincorporatesWPinterventionsthat setoutto: (4) develop students' study skills and capacity for academic attainment and successful graduateprogression.

Practical learning outcomes
The theory-and evidence-based top-down process of generating overarching aims forWP interventions has coincided with a bottom-up analysis and mapping of the programme of interventionsdeliveredbytheUniversityofBath'sWideningParticipationOutreachTeam,which includes campus visits, HE talks, subject taster days, GCSE options days, residential summer schools,andtheuniversity'sbespokeOn Track to Bathintensiveprogrammeforpost-16students. The published aims of the interventions did not fully encapsulate the sophisticated thinking behinddifferenttypesofintervention,orthetacitknowledgeandexpertisethatinformedtheir design,content,anddelivery. Whilesimilarinterventionssharedsomecommonaims,thesewerenotalwaysconsistent andtherewasnostructuretodemonstratehowinterventionswereinterrelated,ortoindicate differentiallevelsofimpactthatmightbehigh-andlow-intensityactivities.Interventionswere 'floating'asboundedentitieswithself-referentialaims,butwithoutaframeworktolocatethem withinthecontextofanoverallprogramme.
The framework's integration of theoretically informed aims with practical learning outcomesenablesbothamacro-analysisoftheimpactandeffectivenessofWPinterventions, and a micro-analysis of its individual components.The stratification of learning outcomes by level and categorization of interventions into low, medium, and high intensity has provided a methodological rationale for more nuanced and meaningful evaluation processes, which are attunedtotheintensityofanintervention,levelofdelivery,andexpectedimpact.
To demonstrate, some end-of-event student data from the 2015Year-12 STEM Summer SchoolwillbepresentedasevidenceinsupportofeachofthekeyLevel-3learningoutcomesfor eachofthefiveaimsintheframework.
For the first aim, to 'develop students' knowledge and awareness of the benefits of higher education and graduate employment', the key Level-3 learning outcome is to'enable students to investigate course and placement options, and social and leisure opportunities at the University of Bath and other universities'.Intheend-of-eventinteractivestudentpoll,98percentagreed(74percent strongly)thatattendingtheeventhadincreasedtheirknowledgeaboutcourseandplacement opportunitiesinHEandopportunitiesforuniversitygraduates.Thisfindingwassupportedby thequalitativedatageneratedfromtheend-of-eventreflectivediscussion,whichdemonstrated thatstudentshadgainedabetterunderstandingofHEandopportunitiesforgraduatesintheir subjectarea,asoneexplained: Itwasgoodtobelearningsomethingnew.Itshowedyouwhereyoucouldgoifyoucontinued doingchemistryanditwasreallyinteresting.
For the second aim, to 'develop students' capacity to navigate higher education and graduate employment sectors and make informed choices', the key Level-3 learning outcome is to'enable students toevaluate course, student finance and graduate employment opportunities and make informed choices that align with personal interests'.Intheend-of-eventinteractivestudentpoll,74percent agreed(34percentstrongly)thatattendingthesummerschoolhadhelpedthemtomakean informeddecisionaboutwhichcoursetostudy.Thisfindingwasagainsupportedbyqualitative data obtained from the end-of-event reflective discussion, where students emphasized how usefulithadbeentoexploresubjectareas,courses,andgraduateopportunitieswithacademics andstudentsworkinginthesefields.Onecommented: The summer school really strengthened my knowledge of what course I want to do. I wasn't sureifIwantedtotakeascienceorengineeringcourse,butnowIdefinitelyknowwhatcourse Iwanttodo.
Forthethirdaim,to'develop students' confidence and resilience to negotiate the challenge of university life and graduate progression',thekeyLevel-3learningoutcomeisto'enable students to anticipate challenges they will face in higher education and make a successful transition to university'. In the end-of-eventstudentpoll,89percentagreed(42percentstrongly)thatattendingthesummer school had made them feel more confident in their ability to succeed at university.This was supported by qualitative data from the end-of-event reflective discussion, which one student encapsulatedbysaying: Ihadneverdoneanythinglikethisbefore,anduniversityjustseemedreallyscary.Doingthishas mademefeelmuchmoreconfidentaboutgoingand,like,Ireallywanttogotouniversity.
Byintegratingcleartheoreticalaimswithpracticallearningoutcomes,theNERUPIevaluation frameworkhasprovidedacomprehensivestructureforlocating WPinterventions,whichenables amorestrategicapproachtoplanning,delivering,andevaluatinginterventionsandprogrammes of activity.The structured approach has enabled rationalization of the design and delivery of interventions and also provided a structure in which to methodologically locate evaluation processes and more effectively assess and demonstrate impact. Nevertheless, it is sufficiently flexibletoencompassarangeofmethodologicalapproachesappropriatetotheinterventionand theagegroup.Throughitsapplicationinthefieldofwideningparticipationtohighereducation, it provides a useful tool for making interventions and evaluation more effective, developing institutional reflexivity, improving monitoring, and contributing to theoretical understandings withinthefield.

Notes on the contributors
Annette Hayton is Head ofWidening Participation at the University of Bath and has many years of experienceinmanagingwideningparticipationactivities.Annetteisinterestedinhoweducationaltheory canbedevelopedandappliedinpracticetopromotepositivechangewithinthesystem.