Elsevier

Computers & Education

Volume 53, Issue 1, August 2009, Pages 132-141
Computers & Education

Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills

https://doi.org/10.1016/j.compedu.2009.01.008Get rights and content

Abstract

This study investigated how the online problem based learning (PBL) approach employed in an online learning environment influenced undergraduate students’ critical thinking skills (CTS) and content knowledge acquisition. The pretest–posttest control group design was used in the study. The subjects included the students who were enrolled at the Department of Primary School Mathematics Teaching in Anadolu University Education Faculty. Subjects attended to Computer II course in 2008 spring. Experiment group attended the online PBL course whereas the control group attended the online instructor-led course. Each group consisted of 20 students. Data collection tools consisted of a multiple choice content knowledge acquisition scale and the Watson–Glaser critical thinking skills test. The results of two-way mixed design ANOVA indicated that learning in the online PBL group did not have a significant effect on the content knowledge acquisition scores. It was also revealed that learning in the online PBL group had a significant effect on increasing the critical thinking skills.

Introduction

Today’s working conditions have required fundamental changes in the profiles of work power, which basically stemmed from the rapid change and transformation in the nature of information. For societies to survive in this competitive world, they need to equip individuals with skills to conduct research, use and transform information, think critically and reflectively, and make higher order decisions. In addition, technological changes along with the changes in the workplace have made critical thinking abilities more important than ever before. In this regard, the instruction designed to assist students’ critical thinking focuses on skills that are applicable across different domains of knowledge and the disposition to use these skills (Halpern, 1999). The report of the National Commission on Excellence in Education, A Nation at Risk (1983) sounded an alarm regarding insufficient and ineffective attempts to foster higher order thinking skills in schools including critical thinking and problem solving. The Secretary’s Commission on Achieving Necessary Skills (1991) regarded higher order thinking competencies as complementary for productive workplaces including critical thinking, decision making, problem solving and reasoning. In short, the ability to understand and use information is emphasized rather than merely possessing it (Richardson, 2003). In this respect, it is crucial for teacher candidates to have these higher order thinking and problem solving skills along with the ability to cooperate and work effectively within a team. Thus, integration of constructivist approaches such as problem based learning (PBL) into the instructional endeavors carries utmost importance (Yaşar, 1998), as constructivism represents how people deal with real-life problems in society by working with peers to make effective thoughtful decisions, take the initiative and solve problems (Jonassen, 1997).

Different definitions of PBL in the literature address three basic principles to differentiate PBL settings from those that are not implementing the PBL. First, it is necessary to have a problem to trigger learning. Second, PBL is not an instructional technique in isolation; rather, it is a holistic approach involving interaction of several learning approaches and methods. Third, PBL is almost always student-centered. These principles offer the opportunities to actively process information, trigger prior knowledge, have a meaningful content, and research and organize information (Kocaman, Okumuş, & Bahar, 2003).

PBL is based on several learning approaches. For instance, Schmidt (1993) maintains that PBL is based on information processing theory, in which learning requires learners to participate actively in the process of retrieving, constructing, and using information; and relate new information with their prior knowledge. Norman and Schmidt (2000) explain PBL through a constructivist viewpoint where information is acquired and general principles are learnt during problem solving practices to be used for similar future problems. Hoffman and Ritchie (1997) consider PBL as ‘a student-centered strategy that has significant contextualized, real-world, ill-structured situations while providing resources, guidance, instruction and opportunities for reflection to learners as they develop content knowledge and problem solving skills’ (p. 97). As indicated in the definition, students have an active role in solving a real-life problem. Learning occurs when learners reflect on their learning, while instructors serve as guides and facilitators. In this respect, instructors do never offer the ready response for a problem (Barrows, 1996, Gallagher, 1997, Stepien et al., 1993). They have a direct influence on the success of the process. More specifically, they exert the direct influence to attract students’ attention whereas they have an indirect influence in increasing achievement (Dolmans, Wolfhagen, & Scmidt, 1994). Proficiency in providing guidance carries utmost importance in this regard. While providing guidance and facilitating the process, they are supposed to sustain the functionality of the group and lead the group to learning objectives (Abacıoğlu, 1998).

PBL activities are designed in a way that is possible for instructors to offer guidance for ill-structured and complicated problems (Gallagher, Stepien, & Rosenthal, 1992). Such problems require learners to adopt higher order thinking skills such as critical thinking. Barrows (1985) summarizes the features of ill-structured problems. First of all, sufficient information to understand the problem is more than the information provided in the problem situation. Second, new information about the problem situation can change the definition of the problem. Third, different perspectives are necessary to interpret information. Finally, there is no definite and single solution for the problem.

Ill-structured problems are similar to those that learners are supposed to solve in their real lives in which the definition of the problem is not clear and some features of the problem are not provided. Particularly the information directing learners to the solution is not provided within the problem situation. Since they are not limited to common classroom situations, the solutions are not predictable. Moreover, resorting to different disciplines can be necessary to solve the problem. As they are similar to real-life problems, student motivation is generally high (Jonassen, 1997).

It has been maintained in the literature that PBL positively influence learning outcomes along with learners’ higher order thinking skills such as creative thinking, problem solving, logical thinking and decision making. For instance, Elshafei (1999) compared the Calculus achievement levels of second graders in five different high schools and 15 different classes where PBL and traditional instructional methods were implemented. Findings indicated that students in PBL settings did prefer this method and had higher levels of achievement. In addition, it was revealed that students in PBL settings had better solutions for given problems in comparison to those who were in traditional classroom settings. Kaptan and Korkmaz (2000) investigated the influence of PBL on problem solving skills and self-efficacy levels of in-service teachers. Findings revealed that the experimental group which was exposed to fundamental science activities through PBL had higher self-efficacy and logical thinking scores than the control group. Deveci (2002) explored the influences of PBL in social science classes on students’ attitudes, achievement levels and retention grades all of which were found to be higher in PBL settings. Yaman and Yalçın (2005) investigated the effects of PBL on creative thinking skills of teacher training students through a pretest–posttest control group design, which revealed that the PBL group had higher scores in creative thinking measures in comparison to the control group. Similar to above studies, several researchers claimed that PBL had a positive influence on problem solving and critical thinking skills (Gallagher, 1997, Hmelo, 1998).

Critical thinking has been the focus of several current implementations in educational settings. Thinking is based on relating and drawing conclusions on notions and events, and involves a variety of different cognitive processes such as implicating, problem solving, examining, reflecting and criticizing. Thinking begins with a physical or psychological inconvenience stemming from lacking the solution for a problem whose solution becomes the objective for an individual. Higher order thinking skills like critical thinking, creative thinking and problem solving are considered necessary skills for 21st-century individuals. Thus, it is necessary to examine these notions objectively, scrutinize on the contents of these skills, and elaborate on the ways to equip individuals with such skills. Problem solving involves cognitive, sensory and psychomotor domains which helps instructors to resort to a large variety of contexts and materials. However, it was maintained that the most valid and reliable way to equip individuals with problem solving skills is to integrate it with creative thinking and decision making (Kalaycı, 2001).

As a non-controversial claim, critical thinking entails awareness of one’s own thinking and reflection on the thinking of the self and others. In this regard, metacognition is defined as thinking about thinking, a skill originating early in life when children first become aware of the mind. However, metacognition does not always develop to the extent that we would like (Kuhn & Dean, 2004). The construct of metacognition can be examined within a developmental framework during which the metacognitive skills become more powerful and effective gradually as they operate increasingly under individuals’ conscious control. Thus, improving metacognitive awareness of what to believe and how to know; and employing metacognitive control on information processing strategies are important developmental and educational goals.

Critical thinking involves critical implication and discussion, which has a crucial role in activating problem solving and decision making processes (Chaffee, 1994). Critical thinking is a constructivist analysis process to examine what is going on in our environments. This analysis system can be used to define problems, take actions towards an aim, make decisions and conduct retrospective evaluations. In order to define, describe, measure and evaluate the critical thinking process, it is necessary to understand indicators of CTS. Several researchers provided the literature with comprehensive classifications in this regard. For instance, the Watson–Glaser classification of CTS involves defining a problem, determining possible solutions and strong assumptions, drawing valid conclusions regarding the solution and evaluation these conclusions (Demirtaşlı-Çıkrıkçı et al., 1996, Kaya, 1997). Such skills are

  • Inference: Defining a problem involves selecting the most appropriate information piece leading to the solution. Making decisions regarding the credibility of assumptions based on the information provided within a text is an inference process.

  • Recognition of assumptions: Recognizing structured and ill-structured assumptions, and deciding whether an assumption based on a specific chunk of information is really based on that information.

  • Deduction: Drawing valid conclusions and making decisions about the relationships among the premises on a specific situation.

  • Interpretation: Evaluating evidences regarding a situation, drawing valid conclusions based on evidences, deciding whether conclusions are credible, and differentiating credible and unreliable generalizations based on a specific data.

  • Evaluation of arguments: Determining the strong and weak aspects of inferences.

Critical thinking is a comprehensive thinking way involving analysis, synthesis, interpretation, evaluation and noticing assumptions. Thus, making use of CTS in instructional practices in teacher training helps instructors to equip learners with significant 21st-century skills.

As a result of constructivism, a transition from teaching to learning and from teacher-centered approaches to learner-centered approaches has been experienced. Online learning practices usually follow constructivist perspectives and make use of the Internet to facilitate personalized learning regardless of time and space boundaries. Moreover, as online learning environments are flexible, attractive and interactive, it is more convenient to implement constructivist and PBL practices through online learning tools.

Studies on the use of PBL in online learning environments generally described the implementation process. For instance, Cho and Jonassen (2002) examined students’ use of online discussion supports to back up their discussions on problem situations. It was revealed that the influence of the discussion support varied according to the type of the problem. Students solving ill-structured problems delved into more intense discussions. A significant relationship between discussion and problem solving was found during ill-structured problems. Finally, it was revealed that students meaningfully transferred what they had learnt from online discussion facilities to their actual discussions during problem solving.

Donnelly (2006) conducted a case study where online PBL techniques were integrated into a face-to-face setting. Instructors, librarians and technical support staff from different universities in Ireland participated in a graduate course for ten weeks. Social interaction in PBL was sustained through both face-to-face and online discussion groups. It was revealed that group activities played the key role in the success of the process. Participants appreciated the fact that the instructor’s role was explained clearly at the beginning of the process as facilitator, guide and discussion moderator. It was also indicated that sustaining the student–content and student–student interaction carried utmost importance.

McLinden, McCall, Hinton, Weston, and Douglas (2006) conducted a two-phase research project to develop online problem-based resources for the professional development of teachers of children with visual impairment. First, they designed, developed, implemented and evaluated an online flexible instructional resource based on PBL principles. Of 90 professionals attending an online professional development course, ten voluntarily participated in the study. In the second phase, they explored how the pilot online PBL resources develop in the first phase can effectively be embedded into the restructured programme of the study. Findings indicated that participants accessed and used online resources quite comfortably. It was implicated that online PBL applications should support access to information and collaborative learning, and sustain interaction among learners. It was also indicated that online PBL environments can be an effective tool to provide special education professionals with in-service training.

An (2006) worked out several instructional design principles for instructors, practitioners and instructional designers regarding the design and development of online collaborative PBL environments. It was suggested that only online courses with appropriate collaboration characteristics should make use of the PBL. More specifically, if the scheduling of the course makes it hard to implement collaborative activities, the PBL need not be used. It was suggested that problems should be relevant to students’ current or future life experiences. In addition, the nature of the problem situations, number of solutions and problem contexts should be taken into account along with technological infrastructure while arranging the working groups.

Some studies compared online PBL environments with other methods. Özdemir (2005) maintained that students in a collaborative PBL environment outperformed those who were in an individual PBL environment in terms of critical thinking scores. It has been observed that, in online PBL studies, individual differences like cognitive flexibility has also been investigated in addition to inquiries on different methods. Batting (1979) defines cognitive flexibility as the skill to use the most effective learning strategy for a specific content or the skill to determine problem solving steps for a certain problem. More specifically, cognitive flexibility can be defined as (a) alternative learning strategies and problem solving skills existent in learners’ repertoire, (b) the ability to choose from these alternative learning strategies and problem solving skills effectively. In this regard, Alper (2003) investigated the effect of cognitive flexibility in online PBL on student achievement levels and attitudes. Thirty high school students participated in the study. Students were divided into three categories in terms of their cognitive flexibility as: low-, medium- and high-level; and their achievement levels were measured through the Ren Test. Findings indicated that online PBL application increased students’ achievement levels and retention scores. In addition, it was revealed that the level of cognitive flexibility did not have an effect on student achievement, attitude and retention scores. Dennis (2003) conducted an experimental study with 34 medical science students to compare face-to-face and online PBL classrooms. Findings indicated that the groups did not differ in terms of achievement, but the online group spent significantly more time for activities.

A review of literature indicated that PBL had positive effects on increasing content knowledge (Candela, 1999, Korucu, 2007), while some studies suggested that it did not have a significant effect on content knowledge acquisition (Alper, 2003, Deveci et al., 2002, Elshafei, 1999). Such controversial findings could stem from the followings:

  • Different constructs and skills related to achievement has been examined in different studies.

  • Online learning tools have a unique motivating effect.

  • PBL has different effects on learners from different age-levels.

  • Individual differences like cognitive flexibility effects the findings.

In addition, some studies revealed that PBL was effective in increasing critical thinking skills (Albanese & Mitchell, 1993; Vernon & Blake, 1993) which could stem from the followings:

  • Structured problem situations in online PBL environments urge learners to think deeper, question, discuss and conduct research.

  • Students reason about the things they learn and the things they need to learn.

  • The organization of information resources in online PBL environments helps learners to individualize the content.

  • The new information organization helps learners assume a new role as initiative takers and guides.

The studies examined reveal that implementing PBL in online environments has gained increasing interest particularly in higher education institutions. Training teachers who have critical thinking, problem solving, collaboration and networking skills is a must in the current century. In this respect, implementing methods like PBL in teacher training settings can serve as an example of using alternative methods in online environments. In addition, implementing online collaborative PBL helps increasing technology use and higher order thinking skills. However, implementation of online instruction should not be realized as the transfer of face-to-face activities to online environments. It seems that the potential of online environments to encourage participation and facilitate student–student, student–instructor, student–technology and student–content interaction is not benefited sufficiently. This situation constituted the problem statement of the current study. More specifically, the current study aimed to compare online PBL and online instructor-led learning in terms of their influence on students’ content knowledge acquisition and critical thinking skills. Following research questions were addressed:

  • 1.

    Do the content knowledge test scores of the online PBL group increase more significantly than the scores of the online instructor-led group?

  • 2.

    Do the CTS scores of the online PBL group increase more significantly than the scores of the online instructor-led group?

Section snippets

Methods and procedures

In the current study, the pretest–posttest control group design was implemented to investigate the effects of the independent variable (i.e. online PBL vs. instructor-led) on dependent variables (i.e. content knowledge and CTS scores). Groups were randomly assigned to the levels of the independent variable to realize a true experiment (Karasar, 2000).

Content knowledge acquisition

In order to see whether the content knowledge test scores of the online PBL group increased more significantly than the scores of the online instructor-led group, prior content knowledge and content posttest scores of both groups were calculated first. Descriptive statistics are given in Table 1.

As shown in Table 1, the experiment group’s pretest mean (18.20) increased to 29.95 in the posttest whereas the control group’s pretest mean (18.30) increased to 27.40. That is, both groups had higher

Conclusions and discussion

The current study conducted in online PBL environment revealed that the PBL group and the instructor-led group did not differ in terms of knowledge-, comprehension- and application-level skills. Findings were somewhat in line with the Candela (1999) study maintaining that undergraduate students exposed to PBL environments did not outperform students in traditional settings. Similarly, Korucu (2007) compared the content knowledge acquisition scores of elementary school students assigned to PBL

References (44)

  • A.C.C. Chiang et al.

    Redesigning chat forum for critical thinking in a problem-based learning environment

    Internet and Higher Education

    (2004)
  • H. Abacıoğlu

    Değerlendirme ve geribildirim, d.e.ü aktif eğitim çalışmaları eğitim yönlendiricisi kurs kitapçığı [Evaluation and feedback, DEU educational studies educational guidance course booklet]

    (1998)
  • M.A. Albanese et al.

    Problem-based learning: A review of literature on its outcomes and implementation issues

    Academic Medicine

    (1993)
  • Alper, Y. A. (2003). The Effect of cognitive flexibility on students’ achievement and attitudes in web mediated problem...
  • An, Y. J. (2006). Collaborative problem-based learning in online environments. Unpublished Phd dissertation. Indiana,...
  • H.S. Barrows

    How to design a problem-based curriculum for pre-clinical years

    (1985)
  • H.S. Barrows

    Problem-based learning in medicine and beyond: A brief overview

  • R.L. Blake et al.

    Student performances on step 1 and step 2 of the United States medical licensing examination following implementation of a problem based learning curriculum

    Academic Medicine

    (2000)
  • Candela, L. L. (1999). Problem based learning versus lecture: Effects on multiple choice test scores in associate...
  • J. Chaffee

    Thinking critically

    (1994)
  • K.L. Cho et al.

    The effects of argumentation scaffolds on argumentation and problem solving

    ETR&D

    (2002)
  • N. Çıkrıkçı

    Watson–Glaser eleştirel akıl yürütme gücü ölçeğinin (form YM) lise öğrencileri üzerindeki ön deneme uygulaması [Pilot implementation of Watson–Glaser scale on high school students]

    Ankara Üniversitesi Eğitim Fakültesi Dergisi

    (1993)
  • Demirtaşlı-Çıkrıkçı, N. (1996). Eleştirel düşünme: bir ölçme aracı ve bir araştırma [Critical thinking: A scale and a...
  • J.K. Dennis

    Problem-based learning in online vs. face-to-face

    Education for Health

    (2003)
  • Deveci, H. (2002). The Effect of problem based learning to attitudes, success and retention in social studies....
  • D.H.J.M. Dolmans et al.

    A rating scale for tutor evaluation in a problem-based curriculum: Validity and reliability

    Medical Education

    (1994)
  • R. Donnelly

    Blended problem-based learning for teacher education: Lessons learnt learning

    Media and Technology

    (2006)
  • Elshafei, D. L. (1999). A comparison of problem based and traditional learning in algebra II. Dissertation Abstract...
  • S.A. Gallagher

    Problem-based learning: Where did it come from, what does it do, and where is it going?

    Journal for the Education of the Gifted

    (1997)
  • S.A. Gallagher et al.

    The effects of problem-based learning on problem solving

    Gifted Child Quarterly

    (1992)
  • D.F. Halpern

    Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker

    New Directions for Teaching and Learning

    (1999)
  • C.E. Hmelo

    Problem-based learning: Effects on the early acquisition of cognitive skill in medicine

    The Journal of the Learning Sciences

    (1998)
  • Cited by (0)

    This study has been derived from the first author’s PhD dissertation completed at Anadolu University Graduate Institute of Educational Sciences.

    1

    Tel.: +90 (222) 3350580x3519; fax: +90 (222) 335 0579.

    View full text