Internet-of-Things Curriculum, Pedagogy, and Assessment for STEM Education: A Review of Literature

The exponentially growing Internet of Things (IoT) market presents compelling opportunities for innovators to develop and create meaningful applications affecting everyday life. This shifting IoT paradigm is placing new demands on educational institutions to train students with IoT-relevant skills. Numerous education researchers continue to document the implementation of IoT-based curriculum, pedagogy, and assessment techniques for STEM education. In this paper, we systematically reviewed this literature and identified 60 journal articles and conference papers that have reported implementing IoT curriculum and associated instructional approaches, educational technologies, and assessment strategies for K-12 and university students. The curriculum, pedagogy, and assessment strategies presented in these studies were analyzed in the context of the sensing, networking, services, and interface layers of the IoT technology paradigm. The review identifies best educational practices and synthesizes actionable strategies for educators to implement effective IoT learning experiences for students. These strategies leverage low-cost IoT hardware, open-source IoT software, active learning-based instructional approaches, and direct/indirect assessment methods. As IoT education becomes increasingly pervasive, the review and strategies provided in this paper may serve as a guide for future educational efforts.

. The IoT 4-layer architecture proposed in [1] was used to formulate the eligibility criteria for the systematic review reported in this paper. The original 4-layer architecture did not include the cybersecurity block. However, given the critical role cybersecurity plays in IoT systems, we have updated the 4-layer architecture with cybersecurity applied to all layers.
teaching Science, Technology, Engineering, and Mathematics (STEM) topics in an experiential format.
The multi-dimensional nature of IoT technologies have been succinctly captured in multi-layered conceptual architectures by several authors [1], [2], [9]- [12]. A popular model is the IoT World Forum Reference Model that has been widely adopted in the literature [13]. While these conceptual architectures feature different numbers and names for IoT layers, they have strong overlaps in terms of the core IoT paradigms. In its most general form, IoT can be conceptually expressed as a system formed by the interaction of 4 key dimensions: real-world sensing, communication networks, software services, and user interfaces. These dimensions result in a 4-layer model that was proposed by Da et al. in [1] and is depicted in Figure 1. When viewed through the lens of IoT/STEM education, this 4-layer model allows for educators to design curriculum, pedagogy, and assessment in a multi-disciplinary manner while immersing students in the technical and societal challenges associated with IoT.
Given the increasing demand for IoT-trained professionals across all industries, it is critical to synthesize literature focusing on IoT education. Future educational efforts can draw on the best practices noted by existing work. While there exists some surveys and reviews in IoT education such as [14]- [18], they primarily focus on the use of IoT technologies on educational campuses [14], [15], [17], or feature a limited educational scope [16]. A survey that focused only on undergraduate IoT curricula across several universities in China was published recently [18] where in the authors note that most IoT engineering curricula that they reviewed were ''an unsystematic patchwork and was deficient in practical platforms'' and that a Technical Knowledge Map of IoT Engineering can assist educators in cultivating IoT talent. By contrast, the goal of this paper is to provide the IoT and STEM community with a review of state of the art in IoT education in universities and K-12 settings with an emphasis on three critical areas of instructional design: curriculum, assessment, and pedagogy [19].

II. METHODOLOGY
A literature search was conducted to collect publications that report on IoT-based course curricula for K-12 and universities. The guidelines used for this review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach [20]. Figure 2 depicts the flowchart of the PRISMA approach for identifying, screening, assessing the eligibility of the articles, and finally including the articles meaningful for this study. 1) Identification: Our initial search with the search terms ''Internet-of-Things education'' generated over 15,000 search results. The preliminary screening included reviewing the abstract of the resulting papers to formulate search words better. Based on these formulations, different combinations of terms such as ''Internet-of-Things coursework'', ''Internet-of-Things and STEM education'', ''Internet-of-Things and Engineering education'', ''Internet-of-Things pedagogy'', ''Internet-of-Things assessment'', ''Internet-of-Things review'', and ''Internet-of-Things survey'' were used to refine the search further. The screening process was conducted on 4766 articles. of STEM Education. Additionally, proceedings from marquee educational conferences such as IEEE Frontiers in Education, ASEE Annual Conference, International Society for Technology in Education (ISTE) among others were also collected. The refined search yielded approximately 2340 research publications. 3) Eligibility: The eligibility criteria of the articles to be included in the review was the following: a) Research papers that reported on coursework, projects, and learning modules covering the 4-layers of IoT technology ( Figure 1) for K-12 and university students were included. b) Research papers that reported on quantitative or qualitative methods to assess IoT-based learning outcomes, measure students' growth in IoT skills and IoT knowledge and capture student attitudes towards IoT/STEM were included. c) Research papers that reported on the pedagogical approach used as part of their IoT-based coursework were included. d) Research papers published between 2010 and 2022 were included. We found that prior to 2010, IoT-based educational efforts were in their conceptual phase. e) To scope our work, we only focused on peer-reviewed research published in the English language.

A. ANALYSIS
The relevant articles were gathered and carefully analyzed. Essential details including article title, author names and academic department, grade/college-year level, course name, project types, pedagogical approach, assessment types, main result, and publication year were tabulated in an excel sheet.
The following synthesis was performed: 1) K-12 and University -The papers were classified based on whether the reported educational effort was focused on K-12 or universities. 2) Curriculum -The papers were classified based on which of the 4 IoT layers depicted in Figure 1 they covered through their educational efforts. 3) Pedagogical Approach -The papers were classified based on the pedagogical approaches that were reported. These included project-based learning (PjBL), problem-based learning (PBL), and collaborative learning (CL), among others. 4) Assessment Types -The papers were classified based on two broad assessment categories (1) direct assessment, which included assignments, quizzes, and projects to understand the student's knowledge of the proposed coursework, and (2) indirect assessments that included surveys, interviews, and open-ended questionnaires.

B. RESULTS
After a careful screening based on the synthesis criteria, a total of 60 papers were selected for detailed analysis. The distribution of these papers is as follows: 1) Publication years - Figure 3 (left) depicts the distribution of publication dates for the 60 research papers. 2) K-12 and University -15 papers reported on educational efforts with K-12 students. Forty-two papers reported on educational efforts with university students. Figure 4 depicts this distribution in further detail in terms of K-12 grade levels and graduate/ undergraduate university programs. 3) Curriculum -As shown in Figure 3 (right), 22 papers reported covering the Sensing layer, 14 papers reported covering the Network layer, 9 papers reported covering the Service layer, and 14 papers reported covering the Interface layer in their educational efforts. 4) Pedagogical Approach -15 papers reported implementing project-based learning (PjBL), 5 papers reported implementing problem-based learning (PBL), and 5 papers reported implementing collaborative learning (CL). 5) Assessment Types -17 papers reported implementing direct assessment and 14 papers reported implementing indirect assessments. The rest of the manuscript is organized as follows: Section III reviews in detail IoT curriculum content in the context of the 4 IoT layers. Section IV provides details about the active-learning based instructional approaches reported in the literature reviewed here. Section V reviews the relatively unexplored area of IoT-coursework assessment. Section VI categorizes the real-world applications that have been reported in the IoT education literature reviewed here. Section VII presents information about challenges that IoT educators face, strategies to address these challenges effectively, and potential areas for future IoT education research.  Section VIII concludes this review and is followed by a formal acknowledgment of the US National Science Foundation (NSF) grant that supported this study [21].

III. REVIEW RESULTS: INTERNET-OF-THINGS CURRICULUM CONTENT
In this section, we provide information about the curriculum presented in the papers reviewed here. We note that there exists a body of literature on IoT curriculum that has proposed flexible curricular models to integrate IoT topics to existing Computer Science (C.S.) and engineering classes and courses across multiple disciplines [22]- [26]. For this review, the curriculum is divided into four categories according to the layer(s) of the IoT architecture.

A. SENSING LAYER CURRICULUM
The sensing layer is the primary layer in any IoT-based system and includes sensors or transducers embedded in realworld environments. These sensors or transducers capture environmental data and provide a real-time understanding of the physical phenomenon occurring in the environment. From an IoT content point of view, the sensing layer is the most widely employed layer to introduce IoT concepts in K-12, undergraduate, and graduate studies. This widespread coverage of the sensing layer can be attributed to the availability of low-cost sensor devices and kits that can easily interface with low-cost single-board computers such as the Arduino [27], Raspberry Pi [28], and Micro:bit [29]. These low-cost sensors and computers, depicted in Figure 5, allow educators and students to rapidly develop an understanding of the sensing layer through sensor data acquisition and visualization. Table 1 presents a summary of different sensor categories reported in the IoT educational literature and the specific sensor types.

1) KEY OBSERVATIONS ABOUT IoT SENSING LAYER CURRICULUM
• Learning Objectives: The curriculum focuses on students learning to interface with analog and digital sensors, process sensor data using foundational computer programming constructs, and making sensor-data-based decisions to affect lights, sounds, and motion.
• Dominance of Environmental and Proximity Sensors in IoT Curriculum: Temperature, humidity, light, sound, motion, and proximity sensors are the most 38354 VOLUME 10, 2022  widely reported sensors used in the IoT curriculum. These sensors are widely used for environmental monitoring and home automation projects.
• RFID integration in IoT Curriculum: RFID sensor systems consist of active and passive tags and readers. The tags are available in different form factors and memory storage size and have been integrated into IoT educational projects [41], [48], [49].
• Biomedical Sensors in IoT Curriculum: An IoT application area that has strong potential to engage students in STEM education is biomedical IoT devices [37], [45]- [47]. Biomedical sensors are used to detect specific biological or physical processes, transmitting or reporting the monitored data. Some of the commonly used biomedical sensors include: heart sound, blood flow, respiration, heart rate, blood pressure, and pulse oximeter, muscle sensors for Electromyography (EMG), and Electrodermal activity [50], [51]. Only a handful of authors have reported on the use of biomedical sensors for educational content development. A key aspect to keep in mind while using biomedical sensors in class is student safety since these sensors interface with the human body.
• Gas/chemical monitoring-based educational projects is another relatively unexplored area. A key barrier to using such sensors is the sensor cost and the sensor calibration required to use them reliably. In K-12 IoT-based curriculum, the sensing layer is the most covered topic since it provides the opportunity to introduce foundational concepts of programming hardware to younger students. Figure 6   educators to integrate such open-source resources into their curriculum with relative ease. The relatively low cost of these Arduino and Raspberry Pi single-board computers makes them an easy choice for educators when developing their IoT curriculum. Higher cost and a lack of thriving open-source community associated with professional-grade computing boards and other IoT hardware has resulted in low adoption of such devices in classroom settings.

B. NETWORK LAYER CURRICULUM
The network layer's role is to connect IoT system elements and share information with other connected devices. This layer is significant as IoT-based systems are connected through wired or wireless networks. Wi-Fi, Bluetooth, Zigbee, LoRa are the most popular wireless technologies for communication used by IoT devices and systems. These technologies work on 900 MHz, 2.4 GHz, and 5 GHz bands and offer anywhere between a few hundred kilobits per sec (Kbps) to a few hundred megabits per second (Mbps) data transmission rates. The choice of a network layer technology in an IoT system requires considerations of cost, data rates, ease of implementation, performance, battery life, reliability, and any governmental restrictions.

1) KEY OBSERVATIONS ABOUT IoT NETWORK LAYER CURRICULUM
In general, the curricular content for the network layer overlaps with computer networking courses. Several topics relevant to IoT networking can be readily drawn from a computer networking syllabus.
• Learning Objectives: The curriculum for the IoT Network Layer focuses on students exploring computer networking architectural layers, wireless networking standards and protocols such as Wi-Fi, Bluetooth, and Zigbee, wired networking using Ethernet, cell-phone data network technologies, the details of IPV4 and IPV6, setting up and configuring virtual local area networks (VLANs), Sockets API Programming, and incorporating the Publish/Subscribe pattern of data messaging transport using the Message Queue Telemetry Transport (MQTT) protocol.
• Dominance of Wi-Fi or Ethernet in IoT Curriculum: Most IoT-based curricula report on the use of Wi-Fi or Ethernet to connect sensors with computers and transfer the collected data [33], [45], [47], [52].
Wi-Fi and Ethernet are integrated using the inbuilt capabilities of low-cost computers such as the Raspberry Pi. Additionally, advanced Wi-Fi modules/microcontroller boards such as the ESP modules, Arduino Nano IoT, and Arduino MKR can also be used to integrate Wi-Fi capabilities in IoT projects [53]- [56]. These modules have been depicted in Figure 5. The use of Wi-Fi and Ethernet also enable educators and students to explore IoT-specific messaging protocols such as MQTT [45], [57], [58].
• Computer Networks and Embedded Systems: Graduate level IoT curriculum that focuses on the interdisciplinarity of computer networks and embedded systems has been reported by authors in [59]. Such an integrated curriculum focuses on the fundamentals behind IoT network layer technologies, design principles, device infrastructure, architecture, and protocol frameworks of IoT devices.
• Bluetooth Usage in IoT Curriculum: The use of Bluetooth was reported in only a handful of works [32], [41]. Bluetooth has shorter communication ranges when compared to other wireless technologies such as Wi-Fi, LoRa, and Zigbee [60]. However, given that the cost of integrating Bluetooth technology in IoT projects continues to decrease rapidly, it is expected that an increasing number of IoT educational efforts will use Bluetooth in the future.
• Zigbee Usage in IoT Curriculum: Zigbee uses IEEE 802.15.4-based specification to create a low-power wireless ad-hoc network with small, digital radios [61]. Similar to the scant use of Bluetooth, the use of Zigbee radios in educational settings has been reported only in a handful of publications [39], [48], [62] • LoRa (Long Range) is a modulation technique used to create Low Power Wide Area Networks [63]. It is based on spread spectrum modulation techniques derived from chirp spread spectrum technology [64]. Based on our review, the use of LoRa technology as part of an IoT curriculum remains to be seen. Figure 7 depicts the distribution of the most used communication technologies observed in the reviewed IoT studies. WiFi or Ethernet is the popular choice in most IoT-based curricula and was reported in 90.0% of the reported studies. The use of cloud services is crucial in developing modern IoT systems. These cloud services are most easily accessible via WiFi or Ethernet. Bluetooth is the second popular choice with 6.0% studies reporting its use. The availability of app development resources makes mobile app development accessible to educators and provides an engaging experience for students.

C. SERVICE LAYER CURRICULUM
This layer encompasses the cloud-based web and IoT services such as information exchange and storage, data management, notifications, search engines, trustworthiness management, and communication. A well-designed service layer will architect IoT application requirements and provide thoughtful software abstractions, APIs, standards, and protocols to support the needs of services, applications, and users in the IoT systems.

1) KEY OBSERVATIONS ABOUT IoT SERVICE LAYER CURRICULUM
Due to the significant proliferation of cloud providers such as Amazon, Microsoft, Google, Heroku, IBM, Alibaba, Huawei among others, the cost of cloud based services has drastically reduced in the past decade [65]- [71] Concurrently, the accessibility of cloud platforms continues to improve.
• Learning Objectives: The curriculum for the IoT Service Layer immerses students in the technical challenges of designing and implementing a cloud-architecture for data storage and retrieval using modern database technologies, incorporating cloud data messaging transport using the MQTT protocol [72], configuring cloud-based services for notifications, emails, and text messages, setting up data analytics dashboards for visualizing relevant data, configuring user-device association and access levels for secure software deployment, ensuring secure remote device updates through the cloud, and user modeling for server/MQTT load testing and scaling.

• Dominance of Amazon Web Services (AWS) in IoT
Curriculum: AWS remains the most widely reported cloud service used in the IoT curricula [37], [45], [62]. In addition to the use of general compute and storage resources of AWS EC2 in IoT classes, authors have also reported on the use of specialized services by students such as AWS Simple Notification Service (SNS) and AWS Simple Email Service (SES) [73]- [75].
• Data Analytics Services for IoT: Modern cloud providers have integrated data analytics services seamlessly with IoT platforms, thus providing educators the ability to cover data analytics literacy in IoT curricula. An example of such a curricular implementation is available in [46]. The authors report the use of Microsoft Azure's Stream Analytics service by students to analyze data from their IoT hub [76]. Students stored the analyzed data in a database service provided by Microsoft Azure [77].
• Industrial IoT Services: In contrast to general cloud service providers, industrial IoT-specific cloud providers such as GAIA, Particle, and Blynk have also been used in IoT curricula to prototype final/capstone projects [78]- [80]. These services are effective in covering real-time services required for industrial-grade IoT systems, including fault tolerance and security.
• Storage and Visualization Services: Storage and visualization are the most commonly used cloud services reported in IoT curricula. Student use of general-purpose clouding computing services such as IBM cloud and Apache Cloudstack in IoT curricula to store energy consumption data and temperature logs has been reported by authors in [36] and [81].
• Other services: The use of several other cloud service providers such as IFTTT for device-to-cloud interfacing, Twilio for text messaging, Locust.IO for load testing, Amazon Alexa for voice integration, and Chef for software management automation have been reported in the literature [82]- [87]. Due to the rapid growth and maturity of cloud service platforms, we anticipate using such platforms in IoT classes to only increase in the future.

D. INTERFACE LAYER CURRICULUM
The interface layer enables users to visualize sensor-based information. The interface layer also allows users to interact with IoT devices to control them or modify their settings. This layer includes hardware displays such as LEDs (monocolored or multi-colored), Liquid Crystal Displays (LCDs), and touch screens. Software interfaces such as dashboards, alerts, prompts, and notifications also form a part of this layer.
More modern interfaces such Amazon Alexa, Apple Siri, and Google Assistant provide a voice interface to IoT system users [86], [88], [89].

1) KEY OBSERVATIONS ABOUT IoT INTERFACE LAYER CURRICULUM
A key curricular pillar of the IoT interface layer is creating intuitive user interfaces (U.I.) and relevant user experiences (UX). The focus on user interface and experience has yet to be reported extensively in the context of IoT curricula -this presents an opportunity to incorporate the study of human factors and experiences in IoT classes.
• Learning Objectives: The curriculum for the interface layer introduces students to the journey of UI/UX design and involves planning, user persona research and human factors considerations, UI/UX design methods, and testing for verification and validation. While there is a significant overlap between the curricular topics covering the IoT interface layer and those that cover general VOLUME 10, 2022 UI/UX design courses, the focus in an IoT-centric course is primarily on contextualizing the UI/UX in terms of embedded IoT devices associated actions.
• Dominance of LED Interfaces in IoT Curriculum: LEDs are the most commonly used interface devices in IoT curricula due to their low cost and easy operation with computing devices such as the Arduino or Raspberry Pi. Mono-color LEDs (single or 7-segment models) have been used to prototype user interfaces that display the state of 3-way traffic signals, the energy consumption in a 3-phase electrical system, and real-time sensor data [33], [48], [79], [90].
• Liquid Crystal Displays: LCDs provide more human-readable information relative to LEDs but are also more expensive than LEDs. The use of LCDs was reported in [52] where students displayed heading angles in degrees for robot motion.
• Mobile Interfaces: Touch screens are an essential interface while designing and developing mobile apps on smart devices for IoT systems. In [47] and [49], the authors reported on students working with mobile applications that were used to control and access an IoT system. Despite the ubiquitous nature of smart devices, touch screens for software app interfaces in the context of IoT curricula have yet to be widely reported.
A reason behind this is the expansion of curricular scope and sequence to accommodate the non-trivial amount of student time required to cover mobile application programming topics.
• Voice Interfaces: Voice assistants provide a vital interface for home automation and biomedical IoT systems. These voice assistants can be incorporated in IoT curricula using hardware modules and associated software APIs [43]. Due to the increased accessibility of voice interfaces, their use in IoT-based curricula is expected to grow.

E. PROGRAMMING LANGUAGES USED IN IoT CURRICULUM
Implementing IoT-based curricula in the classroom requires the use of programming languages such as C/C++, JavaScript, Lua, and Python [96]- [99]. Due to the extensive use of Arduino boards in IoT education, C/C++ features prominently across the majority of educational efforts reported in the literature as they are used to program the Arduino boards. The use of JavaScript and its variants such as Node-Red for web-based and cloud services development has been reported in the literature [45], [100]. IoT system user interface programming has been reported in the literature, with students using languages such as Scala and Blockly to create the interfaces [59], [101], [102]. Figure 8 depicts the languages used for immersing students across the 4 IoT layers in the literature reviewed.

F. CYBERSECURITY IN IoT EDUCATION
The scale of the interconnectedness of IoT device networks, and the plethora of user-facing, real-world IoT applications lend themselves to several cyber vulnerabilities and associated attack vectors. We note that despite significant literature published about cybersecurity educational efforts in K-12 and universities, the literature on incorporating cybersecurity topics for IoT education is only recently starting to gain momentum [103]- [106]. Broadly speaking, IoT systems can be considered cyber-physical systems (CPS); thus any topical coverage of CPS cybersecurity is relevant to IoT education.

IV. REVIEW RESULTS: INSTRUCTIONAL APPROACHES FOR IoT EDUCATION
A majority of IoT educational efforts have reported on the use of active learning based teaching approaches [107]. The recurring theme across these approaches is that students attempt to solve a real-world problem using IoT technologies and frameworks [6], [31], [33], [34], [38], [41], [48], [59], [62], [78], [87], [108]- [117]. Commonly reported active learning approaches for IoT education include problem-based learning (PBL), project-based learning (PjBL), activities based learning, inquiry based learning (IBL), and collaborative learning. PjBL is a teaching method in which students gain knowledge and skills by working for an extended period on a project to investigate and respond to an authentic, engaging, and complex question, problem, or challenge [118]. Problem-based approach (PBL) is a student-centered approach in which students learn about a subject by working in groups to solve open-ended problems [119]. For positive learning and effective outcomes, it is important to thoughtfully design active learning approaches (e.g. PBL and PjBL) to instill technical knowledge and foster student engagement [120]- [122]. The following sections illustrates different approaches used in university and K-12 settings.

A. INSTRUCTIONAL APPROACHES FOR UNIVERSITIES
IoT topics are integrated using several approaches in university course curricula. These courses are offered at both undergraduate and graduate levels.

1) KEY OBSERVATIONS ABOUT IoT INSTRUCTIONAL APPROACHES AT UNIVERSITIES
• IoT using PjBL for 1 st year undergraduate courses: The use of PjBL allows educators to introduce fundamental IoT topics and technologies at early stages of undergraduate curricula in first-year courses such as ''Introduction to Engineering'' [108]. These introductory courses mainly cover IoT architecture and enabling technologies by immersing students in the configuration of hardware platforms and building applications for existing IoT devices.
• IoT using PjBL for final year undergraduate courses: For students in the final year of their undergraduate engineering programs, the use of PjBL for IoT education has been widely reported [33], [34]. These final year IoT courses feature laboratory-based hardware that students have access to for the duration of the entire academic term (semester). The hardware allows students to conduct well-defined experiments that build towards their IoT projects.
• IoT in C.S. Curriculum: Due to the introduction of computer programming courses early on in undergraduate C.S. curricula, C.S. students can study advanced IoT topics in their sophomore (2 nd ) and junior (3 rd ) years as reported by the authors in [41] and [48]. IoT curriculum and technologies can be effectively integrated into C.S. classes such as systems programming and embedded systems using PjBL [41]. Integrating IoT education in C.S. curricula also allows for the integration of cybersecurity topics [48].
• PjBL for IoT Research Projects in the classroom: PjBL allows educators to integrate ongoing IoT research projects into special topics classes that can be attended by both graduate and undergraduate students [6], [62]. Such integration and cross-listing allow educators to train both graduate and undergraduate students with industry-relevant practical IoT skills.
• Convergence in IoT Classrooms: An IoT-based multidisciplinary curriculum has the potential to serve as a conduit to convergence in STEM education using PBL and PjBL approaches [123]. Such convergence was reported by authors in [31] and [109] where undergraduate and graduate students worked on addressing societally relevant environmental problems such as urban rooftop greenhouse design while working in collaboration with relevant stakeholders (gardeners and greenhouse operators).
• Educational Technologies for IoT Curricula: The use of educational tools and technologies is widespread in universities. Hardware technologies such as Arduino boards and associated sensor kits are popular among educators and students alike [38]. Additional tools such as Littlebits, Pocket Labs, and Tiles Toolkit provide a unique way for students to immerse themselves in IoT project design [34], [109], [117], [124], [125].

B. INSTRUCTIONAL APPROACHES FOR K-12
In contrast to universities where IoT education is tightly integrated with course curricula, most IoT educational experiences for K-12 have been reported in the literature as outreach or after-school activities.

1) KEY OBSERVATIONS ABOUT IoT INSTRUCTIONAL APPROACHES IN K-12
• PjBL and IBL are widely used in K-12: PjBL and IBL for IoT education are popular approaches in K-12. Similar to the use of PjBL in universities, authors have reported on K-12 students using IoT sensors for creating projects that span several disciplines [112], [116].
• IoT for Play-based Learning in K-12: IoT technologies can create playful learning experiences for K-12 students [113]. Physical computing devices are used to immerse students in exploring the interdependence between sensors and actuators, and concepts underlying systems thinking and IoT. For example, in [113], the authors conducted K-12 workshops that featured sensor-based physical cubes for play exercises. The direction in which the cube was tilted altered a LED matrix display. Covering the cube would be detected by a light sensor to produce different LED effects.
• Professional Development for K-12 Teachers: IoT training workshops for the professional development of K-12 teachers is a widely used practice. Teachers that experience IoT hardware, software, and curriculum firsthand can deliver meaningful educational experiences to their students. For example, the authors VOLUME 10, 2022 in [115] conducted summer training workshops for high school and middle school teachers with activities centered around building automation. According to the authors, the workshops would expose teachers to IoT and empower them to motivate their students to choose engineering/technology career pathways. Similarly, the authors in [129] scaled their IoT curriculum to primary and secondary schools' by organizing and evaluating a continuing education course for teachers in preuniversity education.
• IoT for Introducing C.S. to K-12 Students: IoT can be used to introduce C.S. principles in K-12 due to its heavy focus on physical computing, and the availability of easy-to-use programming languages such as Python and NodeMCU/Lua [130]. These modern programming languages reduce the barrier to entry in C.S. for K-12 students. For example, the authors in [129] report on the use of IoT hardware and software for 6-12 summer programs. Students focused on creating IoT prototypes using pre-made code templates. Similarly, recent projects from the U.S. National Science Foundation (NSF) are also creating an IoT pedagogical ecosystem for integrated C.S. and Software Engineering education for 9-12 students [21].

C. REMOTE LABORATORY FOR IoT
The advancements of internet technologies have enabled the development of remote laboratories or remote workbenches for IoT education. Students can remotely conduct experiments through these laboratories while not being present at the same physical location as the laboratory setup. The recent COVID pandemic underscores the need for the development of such remote laboratories so that students can continue to receive hands-on practical training without compromising their educational experience [131]. A small number of IoT educational practitioners have identified this need and have developed remote laboratories for IoT education [132]- [135]. The architecture for building a remote laboratory is built around a programmable single-board computer such as the Arduino, Raspberry Pi, Lego Mindstorm builds, and Nucleo-64 [134]. The software system for such laboratories is composed of several modules: (1) a server written in PHP or Node.js, (2) a web interface written in HTML5 and JavaScript, (3) a database to store data, and (4) a driver software that interfaces with the single-board computers. Drivers are written in C++, Java, or Python, based on the programming language of choice and compatibility with the target single-board computer. In [132], the authors suggest that remote monitoring along with the remote laboratory improves the learning performance of the students since the teacher can monitor student work and code in real-time via commercially available remote access tools.

V. REVIEW RESULTS: ASSESSMENT METHODS FOR IoT EDUCATION
Based on our review, perhaps the most understudied area in IoT-based educational efforts is rigorous assessment. Because of the convergent nature of IoT applications, assessment in IoT-based education requires multi-disciplinary and multimethods approaches. As depicted in Table 2, the assessment methods adopted by practitioners in IoT pedagogy can be broadly classified into direct and indirect assessments. Given that each assessment method has its limitations, for a meaningful evaluation of IoT learning objectives and course experience, it is necessary to implement a multi-method strategy involving direct and indirect approaches [109].

A. KEY OBSERVATIONS ABOUT ASSESSMENT STRATEGIES FOR IoT EDUCATION
Direct assessments provide quantifiable evidence about how well students have mastered learning outcomes based on evaluating actual student work samples [136]. Indirect assessments ascertain the perceived extent or value of learning experiences in students by assessing their opinions or thoughts about course knowledge and skills [136]. The representative strategies under each category are presented here as follows: 38360 VOLUME 10, 2022 1) Direct Assessment: These assessment strategies are further classified as formative and summative approaches [136].
• Formative assessment in IoT education are often synchronized with regularly conducted lectures, laboratory exercises, or experiments around specific IoT topics covered in the curriculum. These lectures, exercises, and experiments are organized such that the students are introduced to the building blocks of an IoT system through progressively difficult challenges [34], [37], [38], [45], [46], [48], [78]- [80], [117], [126]. Students submit video demonstrations, code files, or reports for evaluation at the end of each lecture, laboratory exercise, and experiment. These submissions are graded using topic-specific rubrics that assess students' skills and knowledge. Additionally, quizzes at regular intervals that evaluate the progress in students' learning of the theoretical aspects of IoT systems have been reported [44]. Written essays that encourage students to explore IoT technologies that are yet to be fully adopted have also been used by authors as a formative assessment of student's work student work [32].
• Summative assessment methods target activities conducted towards the end of the course to evaluate the cumulative IoT knowledge and skills of students. In IoT education, the predominant form of cumulative activity is a final/capstone project. Several studies report on classes with a final project, where students solve a real-world problem by applying IoT skills learned during the course [34], [37], [38], [43], [45], [46], [59], [78]- [80], [87], [117], [126], [137]. Summative assessment is then performed using student artifacts such as project reports, project demonstrations, video presentations, design reports, and project pitches. Rubrics used to grade these artifacts feature dimensions of PjBL assessment with IoT specific customizations [138]. Other summative assessment techniques such as quizzes conducted towards the end of the course to assess students' theoretical knowledge of IoT technologies have been reported [44], [127]. For example, the authors in [44] conducted a pre-test and post-test as a summative assessment of the learning gains made by the students during the course.
2) Indirect Assessment: This includes the assessment of student attitudes/perspectives, learning gains, and teamwork skills. Indirect assessment is helpful in that it can measure certain affective aspects of student learning, such as values, perceptions, and attitudes, from a variety of data sources. Based on the approaches used, indirect assessments are classified into quantitative and qualitative methods.
• Quantitative assessment methods reported in the literature include Likert Scale surveys to understand students' learning gains, perceptions of IoT activities incorporated in the instruction, expectations from the IoT course, and for collecting student feedback on engagement, impressions, and attitudes [32], [33], [37], [43], [44], [46], [48], [79], [117], [126]- [128]. Data collection using quantitative assessments is often conducted at the end of the course. The data is statistically summarized to glean insights about student response and can be further analyzed using parametric and non-parametric statistical methods to discern any significant variables affecting student responses. As with all survey-based data collection, an appropriate number of samples must be collected along with taking into account the statistical power and effect size [139], [140].
• Qualitative assessment methods reported in the literature include focus group interviews, analyzing feedback comments collected at the end of feedback surveys, analyzing comments on online support forums, and one-on-one interactions with students [6], [34], [37], [43], [109], [117], [128]. In [6], the course was hosted online, so the feedback comments left by students on public forums were analyzed qualitatively to understand student perceptions of the course. Additionally, authors in [109] reported students maintaining weekly diary entries that recorded the students' progress, their weekly achievements, obstacles encountered, and the means students either used or had been recommended to use for overcoming them.

VI. REVIEW RESULTS: REAL-WORLD APPLICATIONS
To showcase the true potential of IoT-based coursework, we review and broadly classify the plethora of real-world applications covered inside or outside of the classrooms. The general theme across all of these real-world applications is the use of projects that involve sensor data acquisition, data transmission to a web service or a web-based dashboard, and user notifications based on data values. Fig. 9 depicts the expansive list of student projects based on application category and types listed across the papers reviewed. Table 3 enlists the corresponding papers. The IoT-based projects observed in the classrooms provided here showcase the breadth of real-world projects which are feasible for the classroom environment.

A. ENVIRONMENTAL MONITORING
Environmental monitoring is a recurring theme across the IoT-coursework literature [31], [34], [38], [45], [78], [109]. Such coursework focuses on an environmentally-themed project-based curriculum that immerses students in the technical challenges of creating dashboards that log temperature, humidity, and moisture levels in the environment. These projects prototype applications for greenhouse and plant monitoring, and weather and air quality monitoring:

1) GREENHOUSE AND PLANT MONITORING
In [31], and [109], the authors reported details of a course in which 13 Computer Science (C.S.) graduate students designed a cloud-based greenhouse monitoring system that collected temperature and humidity data. This project-based exposition was conducted over a duration of seven-weeks. The collected data was used to design subsystems that worked in conjunction with the greenhouse to check for the humidity of soil and fertilizer, and airflow. In [45], the authors designed a hands-on laboratory-based course for final-year Electrical and Computer engineering (ECE) undergraduate students. Teams of 3 to 4 students developed their own IoT device for their final projects. A student team delivered a project that monitored plant moisture content in real-time and used Twitter to notify users when the plant needed watering.

2) WEATHER AND AIR QUALITY MONITORING
In [78], the authors introduced IoT as an open elective course in the eighth semester for C.S. undergraduates. As a part of this course, students designed IoT-based devices to measure environmental problems such as air and waste pollution. The IoT devices that students designed sampled light, humidity, and temperature sensors in real-time. This data was transmitted to web services such as Twitter and Xively [141], [142]. The authors of [34] delivered an elective course for fourth-year undergraduate students studying Information Technology (I.T.). The students designed an outdoor weather station that provided automatic air temperature and relative humidity measurements. The recorded atmospheric data was displayed on a web page. In [38], the authors designed a practice-oriented approach to introduce concepts of IoT to second-year undergraduate students. As a part of the course curriculum, students developed a project concept model that could detect and measure the temperature and carbon dioxide content in a room and the crowdedness or the noise of the room. The designed model also provided advisory alerts through SMS messages to move away or open a window. Environmental sensors can also be used for inferring health impacts as demonstrated in [32] where the authors delivered a course for junior/senior undergraduate ECE students. As a part of the course curriculum, students developed a project, ''Smart Wall,'' using moisture level sensors to infer mold growth.

1) SMART COFFEE MACHINES
In [45], and [36], students designed a coffee pot that communicated detailed information on the amount of coffee consumption and brewing time with a user. The authors of [36] reported on graduate Electrical Engineering (EE) and MBA students developing Cafe IoT -an IoT system that acquired and displayed data from a coffee maker in the break room of the university. The system tracked the break room usage using Infrared motion sensors, the power consumption of the coffee maker, and the amount of brewed coffee.

2) INTEGRATED SMART HOUSE
In [33], the authors reported on undergraduate students working on a house temperature and humidity monitoring system. In [91], the authors extended the curriculum described in [33] by engaging undergraduate students in designing a scaled-down (miniaturized) smart house model that not only monitored but also controlled temperature, lighting, air purification, irrigation, and gas leak alarms. The data captured from the smart house sensors was transferred to a cloud server and visualized via a web application.

3) SMART HOME SECURITY
The students in [41], [78], and [80] designed a motion detection system for Smart Homes using a motion sensor to monitor any forceful entry and provide email and buzzer notifications to the user. Similar to several commercial products available in the market, the system in [41] also featured a camera that captured images when motion was detected and shared it with the user via a cloud-based server [146], [147]. In [41], the students developed a system that unlocked a door whenever a known user was in the vicinity of the door. Of note is the use of finite state machines in the laboratory curriculum reported by [80] for undergraduate and graduate E.E. students.

4) SMART FAUCET
In [143], the authors reported on undergraduate Computer Science (C.S.) students designing a faucet that measured water usage and notified a user of her/his water consumption using an LED. The water consumption data was also displayed on a website to create a leader board that ranked users from least water usage to most water usage.

5) SMART REFRIGERATOR
In [37], the authors note on Industrial Engineering (I.E.) and Technology Management undergraduate students designing a computer vision (CV) system that monitored the type and quantity of food products in a refrigerator. The computer vision system data was sent to an Amazon Web Services (AWS) server to infer to notify users about any shortage of food products and track any abnormal dietary habits.

6) SMART PET ACCESS CONTROL SYSTEM
In [44], the authors report on a course for first-year undergraduate students that was centered around a design project involving a pet access control system. Students integrated infrared sensors to identify whether a pet animal was in the vicinity of the system.

7) OTHER SMART SYSTEMS PROJECTS
Other examples in this category of projects include smart mats [143], smart fire and shock detection system [42], smart water leak system [144], and smart water bottle [45].

C. CONNECTED INFRASTRUCTURE
Urban and suburban environments provide a plethora of opportunities to create IoT applications. Such connected infrastructure projects have been integrated in IoT-coursework effectively [37], [41], [78], [127]. These classroom projects focus on instrumenting the built environment with sensors and connecting them with real-time dashboards and notification systems. Student activities range from presenting hardware and software designs to prototyping IoT systems as described below:

1) SMART PARKING SYSTEM
In [41] and [37], the authors highlight the use of motion sensors and RFID technology employed by students to design a smart parking system. The system in [41] unlocked the doors of a parking garage based on RFID information, and in [37] used a motion sensor to capture and collect vehicle data throughout the city. In [78], students used an ultrasonic sensor to determine whether a vehicle was parked at a designated spot. This data was logged using a Blynk cloud server [148].

2) SMART TRANSPORTATION-WHEELCHAIRS, BUS STOP, AND BUS NETWORKS
The authors in [127] conducted workshops with high school student teams to come up with designs for smart transportation systems. Students presented wide ranging design ideas, including a system that enhanced the driving experience for bus drivers by integrating data from traffic monitoring systems, maps, public web cameras, GPS, and weather forecasts. Students also presented an IoT-based smart wheelchair design and a smart bus stop concept that focused on enhancing the mobility of senior citizens and visually impaired individuals living in a smart city. The smart wheelchair design had built-in sensors and services (like GPS and maps) to calculate VOLUME 10, 2022 the route based on real-time traffic data. The smart bus stop concept featured digital media such as sports news, relevant warnings, and reminders to take medications to visually impaired individuals when they visited the bus stop.

3) SMART WASTE COLLECTION SYSTEM
The authors in [37], and [127] reported on students designing a smart waste collection system with the help of weight sensors and cellular communications to monitor the weight of garbage cans and notify the local municipality. Similarly, students in [78] designed a system using Ultrasonic sensors to identify the level of the waste collected in the trashcans.

D. CONNECTED BIOMEDICAL DEVICES
Only a handful of authors have reported on the use of biomedical sensors for IoT-curriculum development [37], [45]- [47], [143]. Real-world biomedical applications prototyped in class include baby monitoring and smart wearable devices as described in the following. A key aspect to keep in mind while using biomedical sensors and associated electronics in class is student safety since these electromechanical devices can interface with the human body. As such, educators may be required to procure appropriate institutional review board clearances for biomedical sensor-based IoT curricula.

1) BABY MONITORING
In [37], the authors designed a project-based course for Information Systems (I.S.) undergraduate students that focused on smart baby monitoring. Students designed and implemented an IoT system that captured a baby's motion and temperature inside the room. The system notified parents of any abnormalities in the baby's temperature using Amazon's Simple Notification Service (SNS) [74]. The system was triggered by the baby's motion to record a short video of the baby and transmit it to the parents. Similarly, in [47], the authors reported that undergraduate Biomedical Engineering students designed a smart baby incubator that recorded a baby's pulse, temperature, and humidity inside the incubator. This data was transferred using Wi-Fi to a cloud-based server and was visualized using an Android application.

2) SMART WEARABLE
The authors in [45], and [46] report on final year undergraduate ECE students and graduate I.S. students, respectively, developing wearable health monitoring devices that transmitted data to a web application. In both these studies, the students monitored human heart rate. In [143], the authors designed an undergraduate C.S. course where students developed a smart workout glove with accelerometers that tracked the number of weightlifting repetitions completed during a workout session. The authors also report on a dashboard that displayed this information along with user pulse rate.

E. EDUCATIONAL ROBOTS
A handful of authors report on the use of Bluetooth or Wi-Ficontrolled ground robots for students to explore sensor-driven behaviors within the context of IoT layers. In [52], the authors designed a robot that could teach core mathematical concepts such as measurement, linear and nonlinear functions, degrees, and radians using the robot's sensors and motor control. In [126], the authors designed a course for first-year C.S. undergraduate students wherein students designed a ground robot that navigated through a maze using Fuzzy Logic and a Proportional-Integral-Derivative (PID) controller.

VII. DISCUSSIONS: IoT EDUCATION CHALLENGES, STRATEGIES, AND FUTURE RESEARCH AREAS
Based on our literature review, we found that as educators try to deliver IoT educational experiences for their students, several challenges and research areas emerge. In this section, we note these challenges and research areas. Where possible, we also note practical strategies that educators can use to mitigate these challenges to varying extents.

1) Physical Computing and Electronics Devices:
IoT classes are based on physical computing devices such as Arduino and Raspberry Pi, and associated sensors/electronics as depicted in Fig. 5. Comprehensive IoT education should ensure that students get hands-on experience using an exhaustive collection of sensors. Even though electronics hardware costs continue to come down every year, procuring IoT educational kits for students places a financial burden on the educational institution and a logistical burden on the instructional staff to manage hardware parts. Several hardware components get phased out every few years, so keeping up with the latest hardware upgrades can be challenging.
• An effective strategy that educators have employed to mitigate this challenge to some extent is to start with a focus on the sensing layer of IoT with low-cost components such as temperature, light, and sound sensors, and an Arduino/Raspberry Pi 0/Raspberry Pi Pico board. The total cost of such a hardware ensemble is less than $10, and a group of 2 students can share the kit. While not exhaustive, such a kit provides a starting point to IoT education by primarily focusing on the key learning objectives of the sensing layer and providing a gentle introduction to the network layer.
• A future research area that we see is the design of a low-cost hardware kit (ideally <$5 per student) that can provide a comprehensive IoT educational experience for students across the world.

2) Incorporating Software Updates and Paradigms:
Given the rapid evolution of cloud systems, any IoT service/interface layer software needs continuous upkeep by the instructional staff. Regular updates to software used in the classroom for IoT education require keeping current with software configurations and dependencies and using the latest software APIs. Such regular software updates, in turn, leads to continuous management of technical documentation that is to be used by students as classroom resources. Additionally, the authors in [87] rightfully point out that to keep the curriculum relevant to industry standards, the software development paradigms used in the classroom must mirror their evolution found in the industry.
• An effective strategy that educators have employed to mitigate this challenge to some extent involves the use of open-source software and Linux systems.
The use of open-source software keeps the cost of procuring the software set up minimal while allowing instructional staff to create stable ''containers or images'' of the software set up for the academic year [149]. A benefit of such an approach is that the software setup is uniform across all student computers. Before beginning the new academic year, any updates or patches can be applied to the software set up.
• A future research area in this regard is the development of an ''IoT educational software-ina-box system'' or an ''IoT educational web-based platform'' that can be easily installed and run (or accessed through the browser) on Windows, Mac, Linux, and Chromebook computers found in schools and universities. We envision that such a system would require minimal installation/upgrade effort on the part of the instructional staff.

3) Training Educators and Instructional Staff:
Given the variety of technical topics and technologies that are part of an IoT course, any IoT instructional staff needs to be well-versed in the core concepts. Educators need to have versatile hardware and software diagnostic skills to support students through an active learning-based educational experience. Instructors also need to keep up with the rapid developments in the IoT technology landscape, thus emphasizing continuous instructor education.
• An effective strategy that educators have employed to address this challenge is by teaming up with industry professionals or other Subject Matter Experts (SMEs) to create modern curricular resources, videos, and lectures [103].
• A future research area in this regard is developing effective K-12 professional development and university instructional staff training that focuses on base competency, modern pedagogy, and selfsufficiency. 4) Effective Educational Assessment: Most of the papers reviewed note the use of active learning-based instructional approaches. At the same time, only a handful of the papers report on rigorous assessment techniques applied for IoT education. Given the multi-disciplinary nature of IoT, assessing student knowledge and skills gains is non-trivial.
• An effective strategy that educators have employed to address this challenge to some extent is the use of existing PjBL rubrics and C.S. rubrics for direct assessment along with indirect assessment tools such as surveys and interviews [138], [150]- [152].
• A future educational research area here is the development of rigorous and practical tools for assessment of skills and knowledge, and students' affective response.

5)
State-of-the-art Content: From a content perspective, any change in the physical hardware or IoT software requires timely updates of the lesson guides, lecture slides and videos, assignments, project statements, and solutions. This need places a non-trivial burden on the instructional staff. Also, as mentioned earlier, IoT cybersecurity is a relatively new focus area for educational researchers. However, it is a critical topic for IoT-trained students and professionals. Given the ever-changing cybersecurity landscape, keeping up with associated curricular updates requires sustained efforts on the part of the instructional staff.
• A future educational research area here is the development of active learning-based curricular content that covers the core concepts of all 4 IoT layers and can accommodate technology/topical updates with minimal burden on the instructional staff. Another future research area is the effective incorporation of cybersecurity education in IoT classes.

VIII. CONCLUSION
As the IoT industry grows and evolves rapidly, so will the need to provide compelling educational experiences to cultivate the next generation of IoT-trained professionals. This paper presents a systematic review of the literature that covers IoT education from a K-12 or university educator's perspective. Guided by the PRISMA methodology, we systematically reviewed 60 journal and conference papers covering IoT-based educational efforts. The studies were analyzed through the lens of the well-established 4-layer IoT model. Critical insights about IoT curricular content, pedagogical approaches, assessment methods, and real-world classroom projects were articulated. The sensing layer featured prominently across K-12 and universities in terms of IoT curricular content, followed by the interface layer. Instructional approaches based on active learning were most widely used in K-12 and universities, with real-world projects as anchoring activities for classes. The review shed light on the fact that rigorous assessment strategies that leverage a mixed-method approach for IoT-based curricula are needed. The review also led to an exhaustive list of IoT-based real-world classroom projects that may serve as a guide for interested educators. VOLUME 10, 2022 Finally, we discussed the challenges and future research trends associated with IoT education. Care was taken to perform as exhaustive of a review as possible while focusing on the relevant scope. However, as with every systematic review, the number of keywords and the size of the database may be a limiting factor for our review. Future reviews may also use the IoT World Forum Reference Model instead of the 4-layer model. In contrast to other IoT education survey papers, a significant contribution of this review paper is that it focuses on the content, pedagogical approaches, and assessment methodologies adopted in IoT education and highlights the key trends and opportunities for future IoT educators. . His research interests include optimal, multi-dimensional, data-driven decision-making, through the use of techniques from mathematical programming, linear and nonlinear systems theory, statistics, and machine learning. On the education front, he works on bringing innovation to the classroom by introducing novel course content, pedagogical methodologies, and assessment techniques.