Skip to main content
Published Online:https://doi.org/10.1026/0012-1924/a000140

Zur Überprüfung der postulierten Messmodelle für Fragebögen zur Lehrveranstaltungsevaluation (LVE) werden konfirmatorische Faktorenanalysen (CFA) entweder auf Studenten- oder Veranstaltungsebene durchgeführt. Der resultierende Modellfit ist oft inakzeptabel (Marsh et al., 2009). Die vorliegende Studie vergleicht die konventionellen CFA-Verfahren in der LVE mit einer Multilevel-CFA anhand eines empirischen Beispiels mit 183 334 Studentenurteilen. Die Ergebnisse verdeutlichen die Überlegenheit der Multilevel-CFA für die Analyse des Messmodells eines Fragebogens zur LVE. Es werden die Vorteile der Multilevel-CFA für die Erfassung der multidimensionalen Veranstaltungsqualität aufgezeigt und Anwendungsmöglichkeiten für die Hochschulforschung diskutiert.


Multilevel Factor Analysis for Students’ Evaluations of Teaching

The measurement model of questionnaires for students’ evaluations of teaching (SET) is typically evaluated with confirmatory factor analysis (CFA) using either student ratings or class means. However, the modelfit is often inacceptable (Marsh et al., 2009). We compare the traditional CFA approaches of SETs with the multilevel confirmatory factor analysis (ML-CFA) based on an empirical example of 183 334 student ratings. The application of a ML-CFA to SETs is strongly recommended due to the results. We emphasize the advantages of a ML-CFA for measuring course quality and discuss applications for research in higher education.

Literatur

  • Asparouhov, T. & Muthén, B. O. (2006). Comparison of estimation methods for complex survey data analysis. Mplus Web Notes. Zugriff am 11. 12. 2013. Verfügbar unter www.statmodel.com/download/SurveyComp21.pdf First citation in articleGoogle Scholar

  • Born, S., Loßnitzer, T. & Schmidt, B. (2006). Lehrveranstaltungsevaluation an der Friedrich-Schiller-Universität Jena–Eine Analyse der Dimensionalität der eingesetzten Fragebögen. In B. Krause & P. MetzlerHrsg., Empirische Evaluationsmethoden (Bd. 10, S. 99 – 116). Berlin: ZeE Verlag. First citation in articleGoogle Scholar

  • Browne, M. W. & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. LongEds., Testing structural equation models (pp. 136 – 162). Thousand Oaks, CA: Sage. First citation in articleGoogle Scholar

  • Clayson, D. E. (2007). Conceptual and statistical problems of using between-class data in educational research. Journal of Marketing Education, 29, 34 – 38. First citation in articleCrossrefGoogle Scholar

  • d’Apollonia, S. & Abrami, P. C. (1997). Navigating student ratings of instruction. American psychologist, 52, 1198 – 1208. First citation in articleCrossrefGoogle Scholar

  • Hu, L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1 – 55. First citation in articleCrossrefGoogle Scholar

  • Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8, 325 – 352. First citation in articleCrossrefGoogle Scholar

  • Kaplan, D. & Elliot, P. R. (1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation Modeling, 4, 1 – 24. First citation in articleCrossrefGoogle Scholar

  • Loßnitzer, T., Schmidt, B. & Born, S. (2007). Zentrale Lehrveranstaltungsevaluation an der Friedrich-Schiller-Universität Jena–Qualitätsmodell und Messinstrument. In M. Krämer, S. Preiser & K. Brusdeylins (Hrsg.), Psychologiedidaktik und Evaluation VI. (S. 327 – 335). Göttingen: V&R. First citation in articleGoogle Scholar

  • Marsh, H. W. (1983). Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/ course/ instructor characteristics. Journal of Educational Psychology, 75, 150 – 166. First citation in articleCrossrefGoogle Scholar

  • Marsh, H. W., Muthén, B. O., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S. et al. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439 – 476. First citation in articleCrossrefGoogle Scholar

  • Marsh, H. W. & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 53, 1187 – 1197. First citation in articleCrossrefGoogle Scholar

  • Muthén, B. O. (1984). A general structural equation model with dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115 – 132. First citation in articleCrossrefGoogle Scholar

  • Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376 – 398. First citation in articleCrossrefGoogle Scholar

  • Muthén, B. O. & Asparouhov, T. (2015). Item response modeling in Mplus: A multi-dimensional, multi-level, and multi-timepoint example. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of item response theory: Models, statistical tools, and applications. Boca Raton, FL: Chapman & Hall/CRC Press. Zugriff am 11. 12. 2013. Verfügbar unter www.statmodel.com/download/IRT1Version2.pdf First citation in articleGoogle Scholar

  • Muthén, B., du Toit, S. H. C. & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Conditionally accepted for publication in Psychometrika. First citation in articleGoogle Scholar

  • Muthén, B. O. & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267 – 316. First citation in articleCrossrefGoogle Scholar

  • Muthén, L. K. & Muthén, B. O. (1998 – 2012). Mplus user’s guide (7 th ed.). Los Angeles, CA. First citation in articleGoogle Scholar

  • Rindermann, H. (2009). Lehrevaluation: Einführung und Überblick zu Forschung und Praxis der Lehrveranstaltungsevaluation an Hochschulen mit einem Beitrag zur Evaluation computerbasierten Unterrichts (2. Aufl.). Landau: Empirische Pädagogik e. V. First citation in articleGoogle Scholar

  • Ryu, E. & West, S. G. (2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling, 16, 583 – 601. First citation in articleCrossrefGoogle Scholar

  • Schermelleh-Engel, K., Moosbrugger, H. & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23 – 74. First citation in articleGoogle Scholar

  • Schmidt, B. & Loßnitzer, T. (2010). Lehrveranstaltungsevaluation: State of the Art, ein Definitionsvorschlag und Entwicklungslinien. Zeitschrift für Evaluation, 9, 49 – 72. First citation in articleGoogle Scholar

  • Schreiber, J. B., Stage, F. K., King, J., Nora, A. & Barlow, E. A. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99, 323 – 337. First citation in articleCrossrefGoogle Scholar

  • Skinner, C. J., Holt, D. & Smith, T. M. F. (Eds.). (1989). Analysis of complex surveys. West Sussex, England: Wiley. First citation in articleGoogle Scholar

  • Spooren, P., Brockx, B. & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83, 598 – 642. First citation in articleCrossrefGoogle Scholar

  • Toland, M. D. & de Ayala, R. J. (2005). A multilevel factor analysis of students’ evaluations of teaching. Educational and Psychological Measurement, 65, 272 – 296. First citation in articleCrossrefGoogle Scholar

  • Wu, J. & Kwok, O. (2012). Using SEM to analyze complex survey data: A comparison between design-based single-level and model-based multilevel approaches. Structural Equation Modeling, 19, 16 – 35. First citation in articleCrossrefGoogle Scholar