The Relationship Between Participation in Out-of-Class Activities and Cognitive and Social Outcomes of Korean College Students

In the era of the 4 Industrial Revolution, higher education institutions should change practices of educational programs and services, which are mainly based on traditional classroom-based instructions, to allow students to have more diverse experiences. Since college students spend relatively more time engaged in out-of-class activities than attending regular courses, it is necessary to examine how participating in out-of-class programs is related to cultivation of the competencies that the future demands. This study explores the relationship between out-of-class activity participation and perceived change in cognitive and social outcomes of Korean college students. Five out-of-class activities were examined: learning community, undergraduate research, service learning, internship, and residential college programs. K-NSSE (Korea-National Survey of Student Engagement) data were analyzed using hierarchical linear model analysis. The study findings are consistent with the results of previous research that demonstrated a positive association between participating in out-of-class activities and students’ cognitive and social outcomes.


Introduction
The massification of higher education is now widespread across many countries, as reflected in the OECD's annual report on education issues (OECD, 2016). The average first-time tertiary entry rates in OECD member countries lies at 59%, meaning that more than half of high school graduates proceed to higher education institutions. In some member countries like Korea, more than 70% of high school graduates pursue at least bachelor's or equivalent degrees. According to Shin and Teichler (2014), these countries have entered the postmassification of higher education, which is characterized by the influx of and thereby fierce competition among higher education institutions. The increase in demand for higher education has now led to the mushrooming of various types of higher education institutions, i.e., colleges and universities. With the tertiary education market now saturated, higher education institutions must strive to meet student needs to prevent dropouts. Therefore, they should shift the focus of their organizational operations from the university organization itself to the educational needs of their customers -namely, the students.
In general, a high school student's daily school life can be divided into regular curricular and after-school activities, with relatively more time being spent on the former than the latter. In contrast, a college student's campus life includes regular courses and out-of-class activities, with relatively more time spent on the latter. Thus, depending on the content and extent of their out-of-class activity participation, a student's college experience and thus their outcomes can vary widely. Therefore, colleges and universities can create and offer intentionally-designed out-of-class activities to enact certain outcomes, such as higher retention rates, student engagement, and job-readiness. Thus, in the post-massification era of higher education, colleges and universities need to more elaborately and purposefully design out-of-class activities in addition to regular courses in order to survive amidst the competition.
Previous studies (Astin, 1984;Brint & Cantwell, 2010;Everson & Millsap, 2005;Ishler & Upcraft, 2005;Kuh, Kinzie, Buckley, Bridges, & Hayek, 2006;Pascarella & Terenzini, 2005;Tinto, 1993;Wilson et al., 2014;Zacherman & Foubert, 2014) have demonstrated that college students' participation in different out-of-class activities is related to their cognitive and social development, persistence, student involvement and engagement, career decisions, and satisfaction with college. However, these studies have focused on the influence of specific out-of-class activities on certain outcomes. For example, participating in a study abroad program has been shown to be associated with students' social development and career decisions (Finley & McNair, 2013;Knouse, Tanner, & Harris, 1999). In other studies, researchers investigated if participation in any dormitory-based program is related to student engagement (Kuh et al., 2006) or persistence (Astin, 1984), both studies (Astin, 1984;Kuh et al., 2006) concluded that participation in residential college programs are positively associated with both student outcomes, i.e., student engagement and persistence.
Definitions of out-of-class activities for college students vary. Some researchers define them as extracurricular activities such as music, art, and volunteering, while others define them as co-curricular activities linked to students' learning. In this study, with reference to the definition of Bartkus, Nemelka, Nemelka, and Gardner (2012), we demarcate out-ofclass activities as those that do not comprise regular courses, but are possibly related to or part of such courses, that occur outside of the classroom, and that are conducted under the auspices of the institution. They can be either academic or non-academic. Bearing this definition in mind, this study focuses on the relationship between out-of-class activity participation and the cognitive and social outcomes of college students in Korea. The three research questions are as follows: 1. What kinds of out-of-class activities do Korean college students commonly engage in? 2. Is there any association between participating in these activities and perceived change in cognitive outcomes? 3. Is there any association between participating in these activities and perceived change in social outcomes?

Literature Review
Out-of-Class Activities and College Effects According to Pascarella and Terenzini (2005), what undergraduate students experience either on or off campus during their college years is related to college outcomes such as grade-point average (GPA), procuring a job (Smart, 1986), and persistence. In his model of Input-Environment-Outcomes, Astin (1991) argued that students' individual backgrounds, such as socio-economic status (SES), pre-college experiences, and demographic characteristics, are associated with student outcomes by providing environments in which colleges and universities can actively intervene. College environments include the people within the institutions, regular courses and extra-curricular programs, policies, the climate or culture of the institution, and other experiences that students encounter during their college years. Furthermore, Tinto (1993) argued that students enter an institution with pre-entry experiences and individual characteristics that may influence their goals and institutional commitment. Based on their goals and the extent of their institutional commitment, each student experiences both academic and extra-curricular activities differently. During those activities, they experience various interactions with the faculty and peers, both of which are associated with academic and social integration. Tinto claims that these integrations may ultimately affect students' decisions to leave or remain in their institutions. Although the fundamental concept of college effects is similar to those of previous researchers, Weidman (1989) examined college students' socialization process. Upon entering higher education institutions with pre-college experiences and individual characteristics, students have different campus experiences that could be either academic or social. Importantly, their experiences can be influenced by their interactions with their parents as well as non-college reference groups. According to Weidman, through these interactions, students can be inspired, select their careers, come to prefer certain lifestyles, and establish their value systems. Terenzini and Reason's (2005) Comprehensive Model of Influences on Student Learning and Persistence may most closely indicate the relationship between college effects and participation in out-of-class activities. According to them, high school graduates enter college with different individual characteristics and pre-college experiences. However, it is also true that what students experience on campus is influenced by both the institutional context and peer environments. The institutional context includes a) the internal structures, policies, and practices of the institution, b) academic and co-curricular programs, and c) faculty culture. These factors interplay with one another and are then associated with students' peer environment, which includes a) classroom experiences, b) curricular experiences, and c) out-of-class experiences. These peer factors also interact with one another and are related to student outcomes, e.g., cognitive and socio-emotional development, changes in attitudes and values, and the decision to leave or remain in their institutions. Terenzini and Reason's research interests largely lie in students' peer environment, which can be influenced by institutional context. Students' college experiences with their peers both within and outside of the classroom can be shaped by the program, policies, and practices of the institution, and the attitudes and commitment of the faculty. Given that out-of-class activities form a major part of the college experience, the study attempted to uncover what kinds of out-of-class activities are associated with students' cognitive and social outcomes and in what ways.

Out-of-Class Activities and Student Outcomes
Many studies (Everson & Millsap, 2005;Kim, Shin, Seo, & Hwang, 2001;Kuh et al., 2006;Pascarella & Terenzini, 2005;Wilson et al., 2014;Zacherman & Foubert, 2014) have demonstrated that participating in out-of-class activities is associated with students' overall college experiences. According to these studies, as students participate in more activities, the quality and scope of their college outcomes can be improved (see Astin, 1984;Brint & Cantwell, 2010;Everson & Millsap, 2005;Ishler & Upcraft, 2005;Kuh et al., 2006;Wilson et al., 2014;Zacherman & Foubert, 2014) Scholars who consider academic outcomes to be one of the college effects argue that participation in out-of-class activities directly or indirectly influences academic achievement (Brint & Cantwell, 2010;Zacherman & Foubert, 2014). Another group of scholars (Huang & Chang, 2004;Hurtado, 2003;Mauk, 2006) suggest that out-of-class activity participation is positively related to the development of cognitive abilities such as writing and thinking. Other researchers (Weidman, 1989;Mauk, 2006) have indicated that out-of-class programs may facilitate interactions with faculty and/or peers on and off campus and thus promote social development. Participation in various activities outside of regular classes is also regarded as an important determinant of student involvement, leading to students' decisions to persist or depart (Ishler & Upcraft, 2005;Kuh et al, 2006;Tinto, 1993).

Out-of-Class Activities as High Impact Practices (HIPs)
Since college students spend most of their time between and after regular classes, and outof-class activities vary in kind, both the types of activities in which students engage and the extent to which they participate are linked to student outcomes. In fact, many scholars have studied the influence of participating in specific out-of-class activities on varied student outcomes such as academic achievement, cognitive and social development, persistence, and career preparedness. In this context, researchers call these influential out-of-class activities or programs high-impact practices (HIPs). HIPs comprise educational programs offered by the institution, which are highly associated with students' engagement, persistence, and learning and career outcomes (Brownell & Swaner, 2010;Kinzie, 2011;Kuh, 2008; National Survey of Student Engagement [NSSE], 2013; Pascarella, Seifert, & Blaich, 2010). Although scholars have suggested a variety of HIPs, those commonly cited include learning community, undergraduate research, service learning or community-based learning, internship, co-op, field experiences, residential college programs, etc. Table 1 suggests types of out-of-class activities as HIPs and related literature.

Methodology Data and Sample
This study involved secondary data analysis with data obtained from the database of the Korean-National Survey of Student Engagement (K-NSSE), which is the Korean validated version of NSSE in the US. The NSSE has been widely employed to measure student engagement in US higher education institutions. K-NSSE includes questions that investigate students' experiences from both in and out-of-class activities and educational outcomes of Korean four-year college students (see Bae, Kang, & Hong, 2015a). This cross-sectional study employed the data from the 2015 K-NSSE. A sample of 42,228 students from 99 universities was analyzed.

Variables and Measurement
Dependent variables. To examine the relationships between participation in out-of-class activities and student outcomes, two dependent variables were analyzed: a) Cognitive outcomes: How much has your experience at this institution contributed to changes in your knowledge, skills, and personal behaviors in the following areas? 1) thinking critically and analytically, 2) analyzing numerical and statistical information, and 3) solving complex real-world problems. b) Social outcomes: How much has your experience at this institution contributed to changes in your knowledge, skills, and personal behaviors in the following areas? 1) working effectively with others, 2) understanding people of other backgrounds (economic, political, etc.), and 3) being an informed and active citizen.
Each item was measured on a 4-point Likert scale, coded from 1 to 4 (very little=0, some=20, quite a bit=40, very much=60). The cognitive outcomes and social outcomes variables were created by averaging the scores of the three questions in each category. The reliability of both variables is significantly high -Cronbach's alpha = .816 for Cognitive outcomes and .809 for social outcomes.
Independent variables. The independent variables comprise participation in out-of-class activities and include questions regarding the extent to which students had participated in the following activities: 1) participating in a learning community, 2) working with faculty on undergraduate research, 3) conducting service learning, 4) participating in an internship, coop, field experiences, etc., 5) participating in residential college programs. All variables were measured on a 4-point Likert scale (very little=0, some=20, quite a bit=40, very much=60).

Data Analysis
To examine the relationships between college students' out-of-class program participation and cognitive and social outcomes, this study conducted both descriptive and inferential statistical analyses. To investigate the general characteristics of the sample, the means and standard deviations of all control, independent, and dependent variables were calculated and correlation coefficients among the variables were calculated and suggested.
Second, to estimate the influence of participation in five types of out-of-class activities on cognitive and social outcomes, this study employed Hierarchical Linear Modeling (HLM) analysis. Namely, the hierarchically nested data structure of this study was taken into account -i.e., students as the unit of analysis are nested within the universities.
Statistical assumptions were tested to conduct inferential statistics. SPSS 18.0 and HLM 7 were used for the analyses.

Statistical Models: Hierarchical Regression Equations
The equation of HLM analysis is as follows. This regression equation involves the direct effects of the independent variables on the outcome variables both at the student and university levels.
Null model. The null model, with no predictor variables, was used to investigate the extent to which the predictive ability of the conditional model was improved by including the student-and university-level independent variables. The student-and university-level residual variances, σ r 2 and σ u0 2 of the null model, were compared to those of the fitted conditional model. These residual student effects are assumed to be normally distributed, with a mean of 0 and a variance σ 2 ; Where Y ij = outcomes for student i within university j, β 0j = the intercept (student-level), r ij = the residual error term indicating a unique effect associated with student ij, γ 00 = the intercept (university-level) u 0j = the residual error terms indicating a unique effect associated with university j.
All variations among the universities are captured by these residual error terms.
Conditional model. To explain the relationship between independent variables and outcome variables, the conditional model was created, with students as the units of analysis. The outcome variable, Y ij , is predicted by 15 student-level independent variables as below. The intercept (β 0j ) and slope coefficients (β 1j to β 15j ) in this regression equation are assumed to vary randomly across the universities. The variation of the regression coefficients that indicate the effects of the 15 predictor variables on the outcome variables differ across the universities.
Y ij = β 0j + β 1j (gender)+ β 2j (grade)+ β 3j (social science)+ β 4j (education)+ β 5j (engineering)+ β 6j (natural science)+ β 7j (medical)+ )+ β 8j (art & music)+ β 9j (high school performance)+ β 10j (household income)+ β 11j (learning community)+ )+ β 12j (undergraduate research)+ β 13j (service learning)+ β 14j (internship)+β 15j (residential college program)+r ij Where: β 1j ... β 15j = regression coefficients of the student-level equation The university-level model was created to explain the variation in the student-level regression coefficients as follows. Given the purpose of the study to examine the impact of the five out-of-class activities on outcome variables at the student level, the conditional model included no predictor variable at the university level model. In other words, a random intercept model was employed in which intercepts are assumed to vary across universities while sloped are the same across universities.
Finally, the mixed model is as below: Y ij = γ 00 + γ 10 (gender)+ γ 20 (grade)+ γ 30 (social science)+ γ 40 (education)+ γ 50 (engineering)+ γ 60 (natural science)+ γ 70 (medical)+ γ 80 (art & music) + γ 90 (high school performance)+ γ 100 (household income) + γ 110 (learning community)+ γ 120 (undergraduate research)+ γ 130 (service learning)+ γ 140 (internship)+ γ 150 (residential college program)+r ij + u 0j To make the value of the intercept meaningful and interpretable, the independent variables are all grand mean centered, with the exception of two dummy variables -gender and major. In the HLM analysis, centering means "the process of linear transforming a variable X by subtracting a meaningful constant, often some type of mean X" (Luke, 2004, p. 48). Accordingly, the intercept in this study comprises the expected outcome for student i within university j whose values on independent variables are equal to the grand mean.
The HLM analysis offers statistical parameters, including: a) the fixed effects regression parameters (the gammas), which suggest information about the direction and strength of the relationship between independent and dependent variables and b) the random effects variance components, which present the residual variance at the student-and universitylevels. Random effects variance components were analyzed to examine the predictive ability of the conditional model. This can also be understood as effect size in multilevel modeling. To estimate the effect size, the proportional reduction of the prediction error (PRE) (Luke, 2004) was calculated by comparing the residual variances between the conditional and null models. Table 3 presents the results of the descriptive statistics. In total, 44% of students were male students. There is a large distribution of students majoring in social sciences. High school performance grade is on average three to four among nine ranks -the academic evaluation system based on students' relative standing among all others. The average monthly household income is 3-4 million Korean won.

Findings Descriptive Statistics and Correlation Analysis
Regarding participation in out-of-class activities, students were found to be most active in learning community (M=27.92), followed by internships (M=23.04), residential college programs (M=21.86), service learning (M=20.86), and undergraduate research (M=18.98). Regarding the dependent variables, the students had a high level of cognitive and social outcomes (M=30.66 for cognitive outcomes and 32.77 for social outcomes).  Table 4 shows the results of the correlation analysis. Most variables were identified as being statistically correlated. Since some of the correlation coefficients exceeded .50 and, particularly, the coefficient between cognitive and social outcomes was .780, multicollinearity was assessed for all variables. All variance inflation factors (VIFs) were smaller than 10 with a tolerance of more than .1. Therefore, no multicollinearity existed in this analysis.  Table 5 presents the results of the HLM analyses. It demonstrates the relationship between participation in each out-of-class activity and perceived changes in the cognitive and social outcomes of Korean college students.

The relationship between participation in out-of-class activities and cognitive outcomes.
This research employed the HLM statistical technique with the assumption that hierarchically structured data would violate the independence assumption of ordinary least squares (OLS) regression. Thus, to examine what proportion of variance in dependent variables is explained by the university level variables, we calculated the intraclass correlation coefficient (ICC= σ u0 2 /(σ u0 2 + σ r 2 ). The result demonstrates that approximately 4% of the total variation in students' cognitive outcomes may be determined by their experiences associated with the different environments of the universities they attend (ICC=0.04=8.81/8.81+213.81).
The fixed effects of independent variables on dependent variables are shown in Table  5. The fixed effects regression parameter (the gammas) in the upper panels demonstrates that male students had greater cognitive outcomes than female students (γ 10 =2.35, p<.001). Students' grades are not statistically related to changes in the cognitive outcomes perceived by college students. Social science and education majors have higher levels of cognitive outcomes than humanities majors (γ 30 =0.88, p<.01 for social sciences; γ 40 =2.16, p<.001 for education). Medical science majors demonstrate lower levels of cognitive outcome than humanities majors (γ 70 =-2.08, p<.001). However, in terms of cognitive outcomes, engineering and natural sciences majors were not found to have differences compared with humanities majors at a significant level of 0.05. Students who performed better in high school were found to have a higher level of cognitive outcome (γ 90 =0.48, p<.001). Students' socioeconomic status (SES), as measured by monthly household income, is statistically and positively associated with cognitive outcomes (γ 100 =0.16, p<.001). Regarding participation in out-of-class activities, all predictor variables were found to positively influence students' perceived cognitive outcomes (p<.001). In other words, students who participate in each out-of-class activity are more likely to have higher cognitive abilities or competences -i.e., thinking critically and analytically, analyzing numerical and statistical information, and solving complex real-world problems. In particular, the effects of participation in the learning community were found to be greatest on students' cognitive outcomes (γ 110 = 0.20, p<.001). Meanwhile, participation in undergraduate research, internships, and residential college programs were found to be less effective in promoting students' cognitive outcomes (γ 120 = 0.07, γ 140 = 0.07, γ 150 = 0.07, p<.001).
For random effects analysis to calculate an effect size -i.e., the predictive ability of the fitted conditional model, the proportional reduction of prediction error was estimated by comparing the residual variances, σ r 2 and σ u0 2 , between the null and conditional models. Since this study employed the intercept model of HLM analysis, meaning there are no predictor variables at the university level, the effect size was estimated only at the student level. The result demonstrates that 31% of the variance in cognitive outcomes was additionally explained by student level predictors (0.314=213. 81-146.77/213.81). In addition, to find the effect size of the model, Cohen's f 2 was calculated (Cohen's f 2 = .69). According to Cohen's guideline (1988), the model has a large effect size.

The relationship between participation in out-of-class activities and social outcomes.
ICC was first calculated to ascertain what proportion of variance in social outcomes is determined by university level variables. Approximately 4% of total variation in students' social outcomes is explained by their experiences related to the different environments of the university they attend (0. 040=8.86/8.86+212.94).
The fixed effects of predictor variables on social outcomes are shown in Table 5. The fixed effects regression parameter demonstrates that male students have greater social outcomes than female students (γ 10 =0.89, p<.001). Interestingly, however, when controlling for other variables, students' grade has a negative relationship with social outcomes (γ 20 =-0.30, p<.001). The results suggest that social sciences, engineering, natural sciences, medical science, and art & music majors demonstrate a lower level of social outcome than humanities majors. Of note, education majors had a higher level of social achievement than humanities majors. Students who performed better in high school have a higher level of social outcome (γ 90 =0.46, p<.001). Students' SES has a positive impact on social outcomes (γ 100 =0.13, p<.001).
Participation in all types of out-of-class activity has positive effects on students' social outcomes. As shown in the estimation of the effects on cognitive outcomes, participation in a learning community has the greatest impact on students' social outcomes (γ 110 = 0.22, p<.001). The effects of participation in undergraduate research on social outcomes was minimal (γ 120 = 0.02, p<.001). The relationship between participation in residential programs and social outcomes was positive but not large (γ 150 = 0.08, p<.001).
For random effects analysis to find an effect size, the proportional reduction of prediction error was calculated by comparing the residual variances, σ r 2 and σ u0 2 , between the null and conditional models. The result demonstrates that 27% of student level variance in the dependent variable, social outcomes of students, was additionally explained by student level predictors (0.273=212. 94-154.85/212.94). To find the effect size of the model, Cohen's f 2 was calculated (Cohen's f 2 = .73). According to Cohen's guideline (1988), the model has a large effect size.

Discussion and Conclusion
College students spend a considerable amount of time participating in out-of-class activities either on or off campus. Although college experiences may be associated with regular classes to some extent, it can be speculated that participation in out-of-class activities may have equivalently significant or more relation to their overall college experiences and, consequently, their college outcomes. This study began with this assumption and examined whether participation in certain out-of-class programs is related to different college outcomes and, if so, to what extent. To explore these research questions, we analyzed the K-NSSE data with hierarchical linear model analysis. In order to estimate the relationships between participating in different out-of-class activities and changes in cognitive and social outcomes, included were various individual background variables such as gender, major, school year, high school academic performance, family SES as control variables.
The results can be summarized as follows. Male students tend to report more improvement in both cognitive and social outcomes. As college students advance through school, they are likely to do better in terms of cognitive outcomes, while no relationship was found between grade and social outcomes. Additionally, high school performance was positively associated with both cognitive and social outcomes. Student SES, measured by family monthly household income, has a positive association with both cognitive and social outcomes. This suggests that with little burden of livelihood, students from high SES families may have more opportunities to participate in out-of-class programs and may therefore have greater improvement in cognitive and social outcomes. No consistent patterns were found in the relation of participation in out-of-class activities to dependent variables, except for education and medical science majors. Compared with humanities majors, education majors reported more changes in social/cognitive outcomes. Medical science majors showed fewer changes in social/cognitive outcomes than the reference group.
These findings are consistent with previous research, indicating that participating in various out-of-class programs is influential on student outcomes such as social and cognitive behaviors. For instance, previous studies commonly reported positive influences of participating in learning communities on student engagement, involvement, retention, academic performance, and social development (Gordon, Young, & Kalianov, 2001;Hoffman, Richmond, Morrow, & Salomone, 2002;Johnson, 2000;Zhao & Kuh, 2004). Likewise, in this study, joining a learning community showed the most positive influence on both cognitive and social outcomes. Given the nature of the learning community where students may have diverse peer interactions, it seems natural that students who are involved in learning communities have a higher level of change in social behaviors, i.e., working effectively with others, understanding people from other backgrounds, and being active citizens. Similarly, participating in residential college programs also demonstrates a positive relation with students' cognitive and social outcomes. Considering that one major goal of residential college programs is to promote students' social interactions, it is interesting that the study found a comparatively lower association of residential college program participation with changes in students' social behaviors. Besides learning community and residential college programs, many researchers examined the association between participating in each activity and changes in cognitive outcomes (Batchelder & Root, 1994;Jones, 2002;Lim, Seo, & Kwon, 2008), job-readiness (Finley & McNair, 2013;Knouse et al., 1999), and interpersonal skills (Fernald & Goldstein, 2013;Jones, 2002;Miller, Rycek, & Fritson, 2011). These results coincide with the findings of previous studies, suggesting that participating in service learning or internship is also positively related with both cognitive and social outcomes. However, undergraduate research shows relatively less positive association with social outcomes while presenting similarly positive connection with cognitive outcomes as for other out-of-class activities. Most previous studies on undergraduate research examined its impact on persistence, development of research and laboratory skills, pursuit of higher degrees in the same field and interpersonal relationships with faculty mentors. Because the survey items of social outcomes in this study were not targeted to measure those outcomes, the results are not comparable. Of note, however, this study found that conducting under-graduate research is positively associated with perceived changes in cognitive outcomes that may be helpful to the pursuit of higher degrees. However, it may be that within Confucian culture, Korean college students undergo hierarchical laboratory culture where faculty mentors are difficult to access and thus the survey participants might perceive their social experiences or competences to have barely improved compared to cognitive outcomes.
Differently from secondary school students, college students have relatively plentiful time and can thus engage in various out-of-class activities. Accordingly, the kinds of outof-class activities in which they participate and the extent to which they participate can affect the outcomes of their college experiences (Bae, Jeon, & Han, 2015b). In this study, we aimed to investigate the connection between participation in diverse out-of-class activities including learning communities, undergraduate research, service learning, internship, and residential college programs and perceived changes in their cognitive and social outcomes. Previous studies (Brownell & Swaner, 2010;Kuh & Schneider, 2008;Laird, BrckaLorenz, Zilvinskis, & Lambert, 2014;O'Neill, 2010) have demonstrated the significant and positive relation between participating in out-of-class programs and a variety of student outcomes. The results indicated that participation in all five activities was associated with both cognitive and social outcomes, with learning community the most influential on both. This suggests that students participating in learning communities have more opportunities not only to acquire knowledge and skills but also to discuss their knowledge with peers, to think critically and synthetically, and to apply knowledge to real-world problems. This may also contribute to promoting skills of working effectively with others and understanding people from different backgrounds. Conversely, conducting undergraduate research has the least association with both cognitive and social outcomes among the five activities, although the coefficient is statistically significant. This may be because undergraduate research is less popular and developed in Korea's universities and in many cases, is conducted personally not team-based.
In the 4th industrial revolution, which requires critical and creative thinking and socioemotional skills, higher education institutions should provide students with the competencies to meet the demands of the times. While these competencies can be learned in regular courses, they can also be cultivated by highly influential out-of-class programs. In this sense, it is worthwhile that the study found a positive relationship between participation in five out-of-class activities and perceived changes in both cognitive and social outcomes. Furthermore, the effect size was identical across the activities and outcomes, implying that at the individual level, the out-of-class activities above can be improved by higher education institutions to fit students' needs. Each institution must design and develop similar outof-class activities while considering the situation of each institution and students' diverse needs.
Finally, it is important to understand the limitations of the current study. First, considering the large sample size, the results should be interpreted with caution. Since the K-NSSE data consist of quite a large sample of 42,228 students, the statistical significance may have been affected. Second, the K-NSSE data are based on students' self-reported or perceived outcomes of their college experiences. As found in many higher education studies that have generally using self-reported data for analysis (see Herzog & Bowman, 2011), the results may have validity-related issues. In future studies, a research design is recom-mended that employs strict psychological or educational measurements, allowing examination of the more de facto relationship between participation in out-of-class activities and college students' cognitive and social outcomes.