Benchmarking London in the PISA rankings

TheProgrammeforInternationalStudentAssessment(PISA)isanimportantinternationalstudyof15-year-olds’academicachievement.AlthoughPISAhastraditionallybeenusedtodrawcomparisonsacrosscountries,thereisgrowinginterestintheproductionofregional(i.e.city,state,orprovinciallevel)results.InthispaperwepresentthefirstattempttobenchmarkLondoninthePISArankings.Poolingdataacrossthe2009and2012surveywaves,weestimateda95percentconfidenceintervalforLondon’smathematics,reading,andscience PISAscores.Thesearecomparednotonlytocountry-levelaverages,butalsotothescores ofothermajorworldcitiesandstates.Thepaperconcludesbydiscussinghowtheseresultsshouldbeinterpreted,andpossibledirectionsforfutureresearch.

A further important contribution this paper makes is that it is the first to examine the London effect using data other than the UK national test data. Specifically, PISA attempts to measuredifferentskillstoGCSEexams,withagreaterfocusupontheapplicationofskillsin'real world'situationsratherthanmasteryofcurriculum-basedtasks.FindingaLondoneffectinthis verydifferenttypeoftestisimportanttoconfirmthesuperiorachievementofLondon'syoung peoplecomparedtotherestofEngland.Ontheotherhand,ifnoeffectisapparentinPISA,this opensupquestionsastotheimportanceofnationalhigh-stakestestsforLondon'sperformance.
Within each of these explicit strata, schools are then ranked by a variable (or set of variables)thatarelikelytobestronglyassociatedwithPISAtestscores.Thisisknownasimplicit stratification,withhistoricGCSEperformanceoftheschoolthemostimportantvariableused forthispurposeinEngland(althoughschoolgendercompositionandlocaleducationauthority area also play a role). Schools are then randomly selected, with probability proportional to size,withineachoftheexplicitstrata.Thisprocesshasimportantimplications;thecombination of implicit stratification and random sampling means that a representative sample should be drawnwithineachexplicitstratum.AsGreaterLondonwasanexplicitstratumforEngland,a representativesampleofLondonschoolsshouldhavebeencollectedinPISA2009and2012. Thesameistrueforkeycitiesinothercountries(e.g.Riga,Attica,Lisbon)thatwerealsoused asexplicitstratificationvariables.

Why do we not find the 'London effect' in PISA?
Theresultsintheprevioussub-sectionaresomewhatsurprising,particularlythefactthatwe findLondontoperformnobetterthantherestoftheUK.Thisisperhapsincontrasttowhat onewouldexpect,giventhewidespreadbeliefthatLondonschoolsperformstronglyinnational examinations(e.g.GCSEs).OnepossibleexplanationisthatPISAisameasureofchildren's'real world'skillsandinvolvesapplied/contextualizedratherthancurriculum-basedtasks.Thereare, however, several others including timing of the test (children in England sit PISA six months beforetheirGCSEexaminations)andthefactthatPISAisa'low-stakes'test(i.e.childrenand schoolshavelittleridingupontheresults).Wenowprovidefurtherinsightintothedifference betweenLondonpupils'GCSEandPISAscores,usingPISA2009andPISA2012datathathave been linked to the NPD. 2 It is important to note that the analysis in this sub-section refers to state school pupils only (as NPD data for private school pupils could not be linked) and comparesLondontotherestofEngland(ratherthan,aspreviously,therestoftheUK).

Conclusions
PISA is an important international study of 15-year-olds' educational achievement.Although traditionally used to benchmark educational achievement within individual countries, there is growing international interest in the reporting of PISA results at a more localized level.The contributionofthispaperhasbeentoproducethefirstestimateofPISAtestscoresforLondon, alongside several other major international cities. In doing so, this is the first study to place educational standards in London within an international context. Using PISA 2009 and 2012 data,ouranalysissuggeststheaveragePISAmathematicsscoreinLondonfallsbetween458and 500testpoints;readingbetween461and504points;andsciencebetween477and516points. Overall, we find strong evidence that educational achievement is higher in London than in a numberofdevelopingcities(e.g.SãoPaulo,Port-of-Spain,Dubai)butbehindworldleaderssuch asMassachusetts,NewSouthWales(Sydney),Ontario,andShanghai.

Notes on the contributors
JohnJerrimisaReaderinEducationalandSocialStatisticsattheUCLInstituteofEducation,University College London. His research interests include international comparisons, educational inequalities, and impactevaluationmethods.