Abstract
Zur Überprüfung der postulierten Messmodelle für Fragebögen zur Lehrveranstaltungsevaluation (LVE) werden konfirmatorische Faktorenanalysen (CFA) entweder auf Studenten- oder Veranstaltungsebene durchgeführt. Der resultierende Modellfit ist oft inakzeptabel (Marsh et al., 2009). Die vorliegende Studie vergleicht die konventionellen CFA-Verfahren in der LVE mit einer Multilevel-CFA anhand eines empirischen Beispiels mit 183 334 Studentenurteilen. Die Ergebnisse verdeutlichen die Überlegenheit der Multilevel-CFA für die Analyse des Messmodells eines Fragebogens zur LVE. Es werden die Vorteile der Multilevel-CFA für die Erfassung der multidimensionalen Veranstaltungsqualität aufgezeigt und Anwendungsmöglichkeiten für die Hochschulforschung diskutiert.
The measurement model of questionnaires for students evaluations of teaching (SET) is typically evaluated with confirmatory factor analysis (CFA) using either student ratings or class means. However, the modelfit is often inacceptable (Marsh et al., 2009). We compare the traditional CFA approaches of SETs with the multilevel confirmatory factor analysis (ML-CFA) based on an empirical example of 183 334 student ratings. The application of a ML-CFA to SETs is strongly recommended due to the results. We emphasize the advantages of a ML-CFA for measuring course quality and discuss applications for research in higher education.
Literatur
2006). Comparison of estimation methods for complex survey data analysis. Mplus Web Notes. Zugriff am 11. 12. 2013. Verfügbar unter www.statmodel.com/download/SurveyComp21.pdf
(2006). Lehrveranstaltungsevaluation an der Friedrich-Schiller-Universität Jena–Eine Analyse der Dimensionalität der eingesetzten Fragebögen. In B. Krause & P. MetzlerHrsg., Empirische Evaluationsmethoden (Bd. 10, S. 99 – 116). Berlin: ZeE Verlag.
(1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. LongEds., Testing structural equation models (pp. 136 – 162). Thousand Oaks, CA: Sage.
(2007). Conceptual and statistical problems of using between-class data in educational research. Journal of Marketing Education, 29, 34 – 38.
(1997). Navigating student ratings of instruction. American psychologist, 52, 1198 – 1208.
(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1 – 55.
(2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8, 325 – 352.
(1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation Modeling, 4, 1 – 24.
(2007). Zentrale Lehrveranstaltungsevaluation an der Friedrich-Schiller-Universität Jena–Qualitätsmodell und Messinstrument. In M. Krämer, S. Preiser & K. Brusdeylins (Hrsg.), Psychologiedidaktik und Evaluation VI. (S. 327 – 335). Göttingen: V&R.
(1983). Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/ course/ instructor characteristics. Journal of Educational Psychology, 75, 150 – 166.
(2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439 – 476.
(1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 53, 1187 – 1197.
(1984). A general structural equation model with dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115 – 132.
(1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376 – 398.
(2015). Item response modeling in Mplus: A multi-dimensional, multi-level, and multi-timepoint example. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of item response theory: Models, statistical tools, and applications. Boca Raton, FL: Chapman & Hall/CRC Press. Zugriff am 11. 12. 2013. Verfügbar unter www.statmodel.com/download/IRT1Version2.pdf
(1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Conditionally accepted for publication in Psychometrika.
(1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267 – 316.
(1998 – 2012). Mplus user’s guide (7 th ed.). Los Angeles, CA.
(2009). Lehrevaluation: Einführung und Überblick zu Forschung und Praxis der Lehrveranstaltungsevaluation an Hochschulen mit einem Beitrag zur Evaluation computerbasierten Unterrichts (2. Aufl.). Landau: Empirische Pädagogik e. V.
(2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling, 16, 583 – 601.
(2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23 – 74.
(2010). Lehrveranstaltungsevaluation: State of the Art, ein Definitionsvorschlag und Entwicklungslinien. Zeitschrift für Evaluation, 9, 49 – 72.
(2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99, 323 – 337.
(1989). Analysis of complex surveys. West Sussex, England: Wiley.
(2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83, 598 – 642.
(2005). A multilevel factor analysis of students’ evaluations of teaching. Educational and Psychological Measurement, 65, 272 – 296.
(2012). Using SEM to analyze complex survey data: A comparison between design-based single-level and model-based multilevel approaches. Structural Equation Modeling, 19, 16 – 35.
(