A multilevel analysis of the contextual effects in distance education outcomes during COVID-19

The COVID-19 outbreak has forced countries to take extensive measures aimed at minimizing human contact. In this crisis period, distance education has played a crucial role in ensuring continuous learning. However, not all locations have had the same maturity level regarding infrastructure availability, and the city-level heterogeneity in socioeconomic structures might have impeded equal access to distance education. This paper focuses on the contextual dimension of distance education by a comparative approach between in-person and distance education outcomes in Turkey. By a multilevel modelling approach, student outcomes are examined against a set of student-level and city-level determinants of academic success during the COVID-19 period compared to the same academic semester in the previous year. The findings support previous studies, discussing the long-term contextual effects on student outcomes and show that the digital divide between the rural and urban areas and income inequality are the main drivers of city-level variation in students’ success during the pandemic.


Introduction
As one of the major measures against the COVID-19 outbreak in spring 2019, teaching activities moved online almost everywhere in the world. Online education has already become a common practice thanks to the rapid developments in Information and Communication Technologies (Pathak, 2016). Nevertheless, the recent transition from in-person to distance education was unexpected and brought significant uncertainties for higher education institutions, instructors and students (UNESCO, 2020). There is now a growing literature on the observed and potential effects of distance education under the pandemic (Ludvigsson, 2020;Aucejo et al., 2020;Gonzalez et al., 2020). However, no work exists on the relationship between distance education and cities that have suddenly become the hosts of students who access distance education using city infrastructure. This paper aims to focus on this unexplored research area by using administrative data from a Turkish university.
Currently, there are 129 public and 78 private universities and 5 private vocational schools in Turkey. The country has expended its higher education institutions especially in 2014, when the number of universities increased from a total of 53 in 2004 to 180 in 2014. This has led to an overall increase in enrolment rates, which eventually exceeded the OECD average of 70.2 % by 79% in 2014 (Tekneci, 2016). After obtaining the high school diploma, students take a centrally organized test (Higher Education Foundation Examination) to be admitted to universities. Then, Bachelor's degree is granted after the completion of four-year formal education (see Mizikaci, 2003 for an overview of Turkish higher education).
Turkey has a highly centralized higher education system, where the Council of Higher Education (YÖK) is the supreme authority for the regulation of higher education. During the COVID-19 crises, too, the measures have been imposed and implemented by YÖK. Following the World Health Organization declaration that the virus has become a global scale pandemic, the YÖK suspended the education in 202 public and private universities on March 12, 2020, for a week. A "Coronavirus Board" was established immediately to determine universities' online teaching capacities 1 . As of March 30, the majority of the Turkish universities returned to education with digital materials that were delivered by synchronous and asynchronous methods. Perhaps what makes Turkey an interesting study-case is that despite a long tradition of distance education-including correspondence education through letters in 1960s and remote teaching by radio and television centres since 1982 (Ruzgar, 2004;Geray, 2007) -the so-called digital divide between rural and urban areas (Polat, 2012) has the potential of posing threats to the prospect of providing equal access to distance education in the country. In this respect, Turkey is very similar to the rest of the developing world. Many developing countries are subject to this digital gap (Peroni and Bartolo, 2018;Onitsuka and Hidayat, 2018) and, as a consequence, face the same challenge of tackling the recent crises in education, just as Turkey.
Nevertheless, although the transition to distance education was sudden, recent developments in information and communication technologies and overall improved infrastructure in many countries helped for an easier shift from in-person to distance education (Arshad, 2020). Theoretical and applied subjects could be handled online with the aid of existing online platforms and new enterprises. However, despite the general availability of infrastructure and increased internet users (Statista, 2021), not all cities were equally ready to accommodate distance education. During the pandemic, cities worldwide have taken several measures to respond to the virus (OECD, 2020). Extensive lockdowns and general restrictions for human contact have hindered city economies. Meanwhile, cities have also started to host their university students who turned back to their home towns and demanded optimized infrastructure for remote learning. This unreadiness, coupled with the already existing digital gap between rural and urban areas, might have led to unequal access to distance education during the COVID-19 outbreak.
This paper studies the influence of cities on distance education outcomes in Turkey. Previous literature has shown that cities indeed affect several outcomes of individuals, including access to higher education (Türk, 2019), intergenerational social mobility both in terms of occupation (Mueller, 1974;Michelangeli and Türk, 2021) and income (Chetty and Hendren, 2018a;2018b;Michelangeli et al., 2021) and also education at a neighbourhood scale (Owens and Candipan, 2019). However, despite the increasing online education practices worldwide, the literature does not offer any studies on the contextual dimension of student success in distance education.
It is plausible to assume that the internet infrastructure of rural and urban areas, as well as their socioeconomic structure might have created an uneven distribution of academic success in remote learning. In this paper, I exploit the exogenous transition from in-person to distance education posed by the pandemic. I use a full-population university administrative data that follows students for two semesters in 2018-2019 and 2019-2020 academic years, which then also include the COVID-19 period. In addition to the records of educational activities such as the department of study, curriculum and grades, the dataset also includes students' demographic information and their city of parental residence. The panel structure of the dataset allows investigating the city-level effects both for in-person education i.e. when exposure to parental cities is limited and for distance education i.e. when exposure to parental cities is revived.
The empirical exercise is designed in a stepwise fashion. Firstly, a multilevel model approach is used to examine the differences in the determinants of academic success during in-person and distance education. Then the total variance is decomposed into the student and city levels, and the magnitude of change in the latter is discussed. Finally, the best linear unbiased prediction (BLUPs) of random effects are estimated and regressed on a set of city-level covariates. The results show substantial differences between the two delivery methods in terms of student-level determinants of success and city-level variation.
The remainder of the paper is organized as follows: the second section introduces the conceptual framework and hypotheses, the third section presents the dataset and the methods used in the analysis, the fourth section discusses the findings. Finally, the last section concludes with policy suggestions.

Conceptual framework
In this section, following a general discussion on the studies about distance education and recent literature on distance education during COVID-19, the hypotheses and the paper's conceptual framework are presented.
Historically, distance education was designed to respond to the needs of individuals who either lived in rural areas or did not have the time to follow full-time courses in universities while pursuing their professional careers (Lassoued et al., 2020). In recent years, the advances in ICTs and computer access have increased distance learning and teaching practices at all levels of education. The distance system of education conveniently allows students to qualify in their chosen subjects online without the need of physical interaction in classrooms (Sadeghi, 2019). This increasing popularity and the convenience of distance education (flexibility, little or no mobility requirements etc.) have attracted immense scholarly attention. Several papers have studied the effectiveness of distance education, especially in comparison to face-to-face delivery (Coates et al., 2004). Overall, previous literature offers mixed findings regarding the differences in student outcomes, motivation and performance between in-person and distance mode of education delivery. For instance, Weber and Lennon (2007) find no difference in student performance between the two methods. Similarly, in terms of student satisfaction, Driscoll et al. (2012) do not observe any differences among the students who took introductory sociology courses in classrooms or by means of online methods. Meanwhile, Hart et al. (2018) undertake comparative research and find results that favour in-person education in terms of student outcomes in the California Community College system. Additionally, a few works show significant variation in the outcomes of students who belong to different socioeconomic and demographic subgroups. Figlo et al. (2013) compare the learning outcomes of students who took microeconomics course as traditional education with the learning outcomes of students who took the same course online from the same instructor and with the same supplementary materials. Their results not only indicate that the average test score is higher for the students in face-to-face instruction than those in online instruction but also show that there are significant differences in gender, race and ethnicity. Similarly, Xu and Jaggars (2014) find that while face-to-face courses significantly dominate online courses as regards to student performance, the gap in success is higher for males, Black students and younger students. Therefore, both Figlio et al. (2013) and Xu and Jaggars (2014) underline the role of socioeconomic and demographic background in causing disparities in student performance during distance education.
It is worth noting that the recent distance education practices were essentially initiated as a response to the crisis, and the design and methods of the delivery do not directly compare to well-designed online education programs. It is plausible to assume that the aforementioned disparities might become more prominent in a system of distance education which was caught unprepared almost all over the world.
Then, it becomes clear that when constructing the conceptual framework, we need to focus on more recent literature. In this respect, already several papers started to discuss distance education under COVID-19. A few conducted surveys with students to understand their experiences especially at home. For instance, Adnan and Anwar (2020) show that the issues concerning technical availability and internet access are the most striking difficulties students faced in Pakistan. Adarkwah (2020) highlights similar obstacles in distance education in their study for Ghana. In additional to the problems regarding technical and internet infrastructure, Henaku (2020) underlined the new responsibilities students are given at home -such as helping with housework-once they are out of university campuses. Finally, Adedoyin and Soykan (2020) argue that the sudden transition to distance education imposes both challenges similar to the ones identified by other studies and advantages in terms of flexible working conditions, opportunities to integrate global education practices and the role of online academic meetings that facilitate international collaborations. Indeed, the pandemic has revealed the need for internet infrastructure development and technical availability especially in the developing world, and this might be considered as an opportunity for improving services in cities. Cities have played a critical role in distance education during COVID-19 by internet and technical infrastructure but also via other socioeconomic characteristics.
To the best of my knowledge, the literature does not offer studies on contextual analysis of distance education, i.e. the role of the cities where students access higher education. Meanwhile, already well-developed neighbourhood studies document a significant relationship between students' educational outcomes and their current and childhood neighbourhoods (Ellen and Turner, 1997;McDool, 2017). For instance, Brattbakk and Wessel (2013) show that exposure to poor neighbourhoods negatively affects future educational success in Oslo, Norway. In another study, Andersson and Malmberg (2015) test scale sensitivity in contextual effects on educational attainment and find significant effects of neighbourhoods on school success in Sweden. Similarly, Türk and Östh (2019) show that various contextual characteristics such as childhood networks, neighbourhood affluence and human capital accumulations in neighbourhoods explain 36% of inequalities in educational success and that these effects remain influential after the exposure ends in Sweden. At larger scales of geographies, the studies by Chetty and Hendren (2018a;2018b) show several channels in which U.S. counties affect intergenerational social mobility. In their studies, social capital accumulation, poverty, inequality and school quality are illustrated as the factors that create significant variation in status attainment. Türk (2019) find that parental provinces significantly contribute to spatial inequality in access to higher education in Italy at a province level.
Based on the previous studies on contextual influences and neighbourhood studies, and recent and previous works on distance education, the present paper tests the following two hypotheses: Hypothesis 1: There is a significant city-level variation in student outcomes that can be observed even after they migrate to university cities. Hypothesis 2: During distance education, city-level variation has become stronger due to revived exposure to parental cities. Figure 1 displays the conceptual framework on which the hypotheses are based. Universities are institutionally designed to filter socioeconomic and demographic circumstances of students during in-person education. The students are provided with equal access to university facilities such as libraries, internet and other infrastructure, and even nutrition through canteens. Even though background influences can still be in place, higher education institutions aim to minimize these effects. For instance, both Grave (2011) andDanilowicz et al. (2017) found minimal effects of social origin on academic success during university education. Similarly, in Turkey, while female students perform poorly in the university entrance test, once admitted, they outperform male students (Dayıoğlu and Türüt-Aşık, 2007).

Figure 1. Conceptual framework: left-panel represents exogenous factors during in-person education and right panel during distance education.
Source: Author's representation The present paper's conceptual framework suggests that universities also filter socioeconomic background regarding the city of origin. Meanwhile, since students go back to their hometowns during distance education, the ability of higher education institutions to minimize these exogenous factors might lessen and the contextual effects might become more prominent.
The following section presents the dataset and the methods used to test the hypothesis.

Data and methods
This paper uses administrative data from one of the youngest public universities located in Anatolia to examine the city-level determinants of student outcomes in distance education. As in other higher education institutions in Turkey, the university started distance education activities on March 30, 2020. All lectures and related activities were conducted online until the final exams in June 2020. As the teaching activities, exams were also conducted online. The dataset includes information for all registered students of undergraduate-level both during the COVID-19 period and also for the same term in the previous year. This means that we can design a comparative framework, where students' final exam grades are compared between the two terms.
As shown in Table 1, final grades during distance education have slightly increased on average. This is in line with the recent literature arguing the increase in student success during COVID-19 (Gonzalez et al., 2020). Moreover, the total number of female students and international students have slightly increased in Spring 2020. The remaining statistics are very similar in the two periods.  Table 1 also shows the city-level variables used in the model for distance education. All variables except fibre optic infrastructure is provided by TURKSTAT 2 . The variable rurality indicates the rural population's share, and Gini is the income inequality index for each city. Fibre internet per capita indicates the length of the optic fibre network in cities divided by the total population to measure the quality of internet infrastructure in cities. The variables highly educated population, literacy rates, GDP per capita and unemployment rates are included in the final model as predictors of socioeconomic structure of cities 3 . Turn-out rates indicate the share of voting population among the eligible population in the municipal election in 2019 and are used as a proxy of social capital (Helliwell and Putnam, 1995).
The regression framework of the present paper is based on a multilevel modelling approach. Multilevel models are extensively used for education research, where students are often assumed to nest in classes, schools or under other theoretical groups (Goddard, 2003;Anderson and García, 2020). In a parallel fashion, in this paper, the students are defined in the first level, and they are considered to be clustered under cities. A major advantage of using multilevel models is that the fixed and the random part of the equation can be estimated simultaneously, and the variance can be decomposed into student and city-level factors. Below I present the stepwise methodology used in the analysis.
First, a so-called null model is defined by fitting the grand mean with only random specification: where the intercept term is equal to the sum of average intercept 0 and city-level deviation terms as follows: substituting (2) in (1) yields: where is log of the final exam grades of student whose parental city is , 0 is intercept term, is city specific effects and is first level residuals. Eq.3 summaries the relative variation in two levels (Hox, 2002), which can be used to test the hypothesis that city-level variation indeed has significant effects on student success. Eq.3 is run for both terms as separate regressions. We can then follow the city-level effects in the two semesters by comparing the outputs from multilevel models to test the second hypothesis.
The city-level effects derived from the Null model might also reflect other influences that are not essentially spatial but might result from, for instance, the sorting of similar individuals in the same locations. In order to control for individual effects, a set of first-level covariates are added to the model. The full model of the student's success then can be defined as: In difference to Eq.3, Eq.4 includes as a set of individual level covariates such as gender, GPA, year of studies, and a dummy for soft vs hard sciences. A straightforward method to examine the dependence of student success on city-level influences is to compute the Intra-class Correlation Coefficient (ICC). ICC measures the relative importance of the group level in the distribution of the dependent variable. The ICC of student success can be written as follows: In Eq.5 while 2 denotes second level variance, 2 denotes the residual variance at the first level. The ICC takes a value 0 in the absence of a link between level 1 and 2 variances and takes 1 if the group level explains all the variance. The ICCs are computed both for the null and full models, and the magnitude of the change from in-person to distance education periods is discussed.
Best Linear Unbiased Predictions of city-level (random effects) are computed for the distance education period in the final step. The BLUP is a method for predicting random effects, and the predicted values represent group-level differences in the response variable (Snijders and Bosker, 1999). By estimating BLUPs, we can examine both the change in the influence of cities on student performance (by ICCs) and the underlying mechanisms that lead to the observed changes. In the present paper, after constructing the full model, the city-level effects are predicted for the distance education model. The predicted values are then regressed against a set of city-level covariates, including internet access, rurality, house conditions and quality of life indicators.
The following section presents the findings from the step-wise methodology.

Findings
This section presents the findings from a comparative perspective between inperson and distance education periods. As introduced in the previous section, the stepwise method starts with an empty multilevel model to examine the city level variation of grades in the two periods. The empty model outputs then can be used to decompose the total variance into individual and group level components. The ICCs are used to compare the two periods, which are then interpreted as the city level effects in explaining the student success. In the second step, a full multilevel model is defined, and the outputs from the models are discussed for the two academic years in a comparative setting. Finally, to study the contextual dimension of distance education in detail, city effects are predicted as BLUPs, and a linear regression model is fitted with several city-level characteristics to examine the relevance between random effects and city attributes.
In Table 2, regression outputs and ICCs are presented for the null and full models for the Spring 2019 and Spring 2020 semesters. Across all four models, the city level variance and Log Likelihood tests confirm our first hypotheses that there is indeed a city-level variation in the student success both during face-to-face and distance education periods. This also confirms the findings of previous studies on the relationship between contextual influences and educational outcomes both during exposure and after the exposure ends (Türk and Östh, 2019). The ICCs reported for the null models also suggest that the city level variation has considerably increased during distance education, hence, when exposure to the city characteristics was returned. Specifically, the total explained variance in final grades has increased from 5% to 24% during online education. This points out a rather quick emergence of contextual effects, already in one semester of distance education experience, and confirms the paper's second hypothesis that the city-level effects become more potent with exposure.
The heterogeneity in city effects is likely to be dependent on varying internet infrastructure and differing local contexts in cities. Before exploring these factors, I return to the full model, where student-level covariates are added to the baseline specification. As mentioned above, a part of city-level variation might be due to the factors that are not spatial. By including student-level information, we can control some of these aspatial factors. Note that the dataset does not have family background information, which would be a better proxy of similar families' sorting pattern (in terms of socioeconomics) into the same locations.
The second and fourth columns of Table 2 present the regression outputs for in-person education and distance education terms. The findings reveal several interesting comparisons between in-person and distance education periods regarding student-level determinants of success. According to the results, while female students have been more successful than male students in both terms, the coeffects suggest that the differences have decreased during distance education. This result might be related to female students' new duties in their homes, such as helping house works (see Teke Lloyd, 2019 for a discussion on gendered norms in Turkey).
Similarly, international students performed better in both periods, but differences decreased during distance education. Following the sudden restrictions on international flights, most international students had to remain in Turkey, especially in the spring semester. This might have negatively affected international students' performance due to the psychological stress of being away from home during the pandemic. Table 3 also reveals that students are more successful in elective courses than compulsory courses and more during distance education. This is in line with previous studies that show that students are more motivated for elective courses (Darby, 2006), and elective courses are evaluated more positively than compulsory courses during online education (Smart and Cappel, 2006). Additionally, GPA (lagged one term) suggests that successful students continue to perform better and that GPA becomes a stronger predictor of grades during distance education.
Interestingly, the dummies for the year of studies and also the dummy indicating social sciences show the opposite impact between the two academic terms. While first-year students received higher grades in the in-person education period, the opposite is valid during the distance education period. That the fourth-year students are significantly less successful than the first-year students might be related to the extra time they spend looking for employment and internships and preparing for graduate school applications while attending university education. Furthermore, the coefficients point to increasing positive affect as the year in the university increases in distance education. These results can be interpreted as the advantage of the university experience. First-year students had only half of a year in the university, and they had not had enough experience and time to develop friend networks with their classmates, especially for peer learning. This is an important finding underlining the significance of the university environment on the successful development of students. The results also suggest that while the performance of social and hard science students did not differ during in-person education, the students in social science programs have been more successful in distance education than those in hard sciences. The significant differences between soft and hard sciences might be explained by the difficulty of teaching and learning applied courses online as also indicated by the study of Lassoued et al. (2020). Nevertheless, the findings regarding university experience and performance by social vs hard science are important and deserve further study. Moving on to city-level variation, we can see that a part of group-level variance decreases as we include student-level information. As mentioned above, this is because the fixed part of the multilevel model explains a degree of variation in the second level (Snijders and Bosker, 1994). Precisely, the city-level variation in in-person education is calculated as 0.017 and in distance education, 0.124. This is again in line with the second hypothesis of the present paper.
The analysis shown in Table 2 is conducted with the full population of students in the two periods. Since the university is young and growing over time, the total number of students change between Pre and Post-COVID periods. Therefore, in order to test the robustness of the results, I excluded the new students and those who graduated in the 2018-2019 academic year from the sample and ran the Null and full multilevel models only for the students who are present in both academic terms. The multilevel outputs are presented in Appendix Table A1. The output indicates that the city-level results are robust also when following the same students throughout two periods.
In the next step, BLUPs from Eq. 4 for distance education are predicted, and the variation in city-effects during distance education is mapped in Figure 2. Consistent with previous studies on the regional divide in Turkey (Gezici and Hewings, 2007), the students who accessed distance education in the far east and east-west cities have been substantially disadvantaged. The maps also show a similar pattern in the Northern cities with a high rural population. Table 3 presents regression outputs that better communicate the association between the city-random effects and contextual variables. The variables rurality and Gini coefficient of income inequality show a significant and negative correlation with city-effects. This means that the economic divide and digital divide between the rural and urban areas inhibited academic success during distance education. The digital divide between rural and urban areas is now a well-documented issue, especially in developing countries (Fong, 2009;Furuholt and Kristiansen, 2007;Lembani et al, 2020). In Turkey too, even though one-fourth of the population lives in rural areas, access to the internet and computers is considerably lower in these regions than in urban areas (Yıldız, 2010;Polat, 2012).
Similarly, fibre optic internet infrastructure per capita shows a strong and positive association with city-level effects. With this finding, it becomes clear that city infrastructure regarding internet access has been a significant determinant of student success in distance education and that the uneven distribution of internet access has generated a degree of inequality in student outcomes. Furthermore, Table 3 suggest that residing in a city with a high population of university graduates and literacy rates was associated with better academic success. This is in line with the extensive literature on the relationship between student achievement and human capital accumulation (Checchi, 2006). However, it is worth noting that these results may be driven partly by the lack of parental background information in the dataset. That is, the share of a highly educated population in cities might be predicting the likelihood of students to come from a highly educated household.  Source: Author's representation Even though GDP per capita and unemployment rates do not result in statistically significant coefficients, the variables Gini index of income inequality and the share of highly educated individuals might be picking up most of the variation concerning the socioeconomic structure of the cities. Finally, the model also includes the variable turn-out rates as a predictor of social capital. It has a positive but insignificant coefficient. This is -to some extent-an expected result since the interaction of students with cities has been limited due to extensive restrictions on the mobility of youngsters in Turkey 4 . However, especially the analysis regarding socioeconomic structure of cities and social capital accumulation must be repeated after a year of experience with distance education during COVID-19.

Conclusions
This paper has tested the hypothesis that the cities -through several channelsaffect students' academic success in higher education both during in-person and distance delivery. The main argument proposed in the conceptual framework was that during the COVID-19 outbreak, cities have had even further influence on students due to the revival of exposure to parental cities. The multilevel models confirmed the hypotheses and indicated that the heterogeneity of city effects in distance education mainly originate from differing levels of internet infrastructure availability but also other socioeconomic characteristics. The digital divide between the rural and urban areas and income inequality resulted in the main drivers of citylevel variation in students' success during the pandemic.
The results presented in this paper suggest that to ensure equal access to distance education, the digital divide must be prioritized in the policy agenda. Of course, the fundamental issues such as regional inequalities must also be addressed to ensure quality education as one of the sustainable development goals of the U.N. and its local implementation. Both the recent crises and the findings of this study highlight the importance of cities' role against global shocks. This means that cities must become resilient to a range of shocks both in terms of fair socio-economic structure and good infrastructure extending to rural areas.
In addition to city effects, this paper has also offered several interesting insights into distance education during COVID-19. Overall results point to substantial differences between in-person and distance education outcomes, which underline universities' essential aspect in equalizing the opportunities for their students. Perhaps a secondary implication of this paper then relates to the role of traditional university education on campuses. Despite the shifting trends towards distance education practices that started in the pre-COVID period as a consequence of the advancements in ICTs, there is a need of acknowledging the deep-rooted achievements of universities and maybe advocating hybrid systems where students benefit both from the experiences acquired in campuses but also from the ease of accessing information in distance education.