Abstract
What data on learning are needed depends on what learning outcomes are expected. In this chapter, two recently influential perspectives on learning outcomes are considered: research on the brain (neuroscience); and research on the demands of the information age. Desirable learning for the information age can be categorised in terms of domains of knowledge as well as various 21st century competencies: ICT-related competencies, conceptual knowledge competencies, cognitive competencies, metacognitive competencies, personal competencies, and interpersonal competencies. Implications for the collection of assessment data on student learning are considered and point to the need for assessment reform.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The classic treatment of this was the 1939 fable by J. Anton Peddiwell (pseudonym for Harold R.W. Benjamin) in the book The Saber-tooth Curriculum (republished by McGraw-Hill in 2004). In this moral tale, the young are taught three things necessary at the time for the success of the tribe: fish-grabbing-with-bare hands; horse-clubbing; and sabre-tooth tiger-scaring-with-fire. But, even when climate change makes these practically useless, and the necessary skills become using-nets-to-catch-fish, catching-antelopes-with-snares and digging-bear-pits, the three ‘traditional’ skills are not replaced without a struggle and are justified by their usefulness in training the mind. The tale is still pertinent; Dumont and Istance (2010) note that many schools and classrooms are still preparing students for an industrial economy and that ‘too many schools do not exemplify [the conclusions of learning scientists]’ and that there is a ‘great disconnect’ between learning research and educational practice (p. 21).
- 2.
‘What this means is that when one neuron sends signals to another neuron, and that second neuron becomes activated, the connection between the two neurons is strengthened. The more one neuron activates another neuron, the stronger the connection between them grows. Some neuroscientists remember this by the phrase, “What fires together, wires together!”’ (Blakemore & Firth, 2005, p. 133)
- 3.
‘There is an interactive relationship between experience and brain structure: experience causes changes in brain structure, which in turn, influence the effects that subsequent experience has on the brain. Hence, it continually undergoes a process of experience-dependent reorganisation throughout life. So, individual learning differences arise as a result of a continual and cumulative interaction between genetic factors and their environmental contexts. The environment influences the expression of genes relevant to learning throughout the lifespan, which expression results in structural changes in the brain. These modifications then affect subsequent experience-elicited genetic expression. In this way, each individual’s brain accumulates structural idiosyncrasies which mediate learning processes. This means that it is difficult to prescribe one ideal learning environment for everyone—while the sensible nurturing of the brain will benefit all, it will not necessarily be equally effective.’ (OECD, 2007, p. 60)
- 4.
‘Overall, neuroscience research confirms the important role that experience plays in building the structure of the mind by modifying the structures of the brain: development is not solely the unfolding of preprogrammed patterns. Moreover, there is a convergence of many kinds of research on some of the rules that govern learning. One of the simplest rules is that practice increases learning; in the brain, there is a similar relationship between the amount of experience in a complex environment and the amount of structural change.’ (Bransford et al., 2000, p. 125)
- 5.
‘…studies of the brain show that individual characteristics are far from fixed – there is constant interaction between genetic function and experience and plasticity, such that the notion of what constitutes an individual’s capacities should be treated with considerable caution.’ (OECD, 2007, p. 156)
- 6.
‘Emotional experiences are also built into the architecture of the developing brain. In fact, emotion and cognition operate seamlessly [are inextricably linked] in the brain. The brain is organized into assemblies of neurons with specialised properties and functions. A stimulus elicits a network response of various assemblies to produce a learning experience. Particular components of this experience can usefully be labeled cognitive or emotional, but the distinction between the two is theoretical since they are integrated and inseparable in the brain.’ (Hinton & Fischer, 2010, p. 119).
- 7.
‘Far from the focus on the brain reinforcing an exclusively cognitive, performance-driven bias, it suggests the need for holistic approaches which recognise the close inter-dependence of physical and intellectual well-being, and the close interplay of the emotional and cognitive, the analytical and the creative arts.’ (OECD, 2007, p. 18)
- 8.
Various terminologies exist, with essentially the same meanings. Here, the chosen terminology is ‘21st century competencies’. An alternative with the same meaning is ‘21st century competences’. There is also widespread reference to ‘21st century skills’, though skill would seem to be a narrower concept than competency, as well as being less applicable to some competencies. ‘Key competencies’ and ‘core competencies’ are other alternatives.
- 9.
A quite different categorisation that has gained some acceptance sees ten competencies organised into four categories: ways of thinking; ways of working; tools for working; skills for living in the world (Binkley et al., 2012).
- 10.
Another prominent framework (Partnership for 21st Century Skills [P21], 2019) identifies key subjects as: English, Reading, Language Arts; World Languages; Arts; Mathematics; Economics; Science; Geography; History; and Government and Civics. These should be infused with five ‘interdisciplinary’ themes: Global Awareness; Financial, Economic, Business and Entrepreneurial Literacy; Civic Literacy; Health Literacy; and Environmental Literacy. These are to supplemented by: Learning and Innovation Skills (critical thinking; communication; collaboration; and creativity); Information, Media and Technology Literacies; and Life and Career Skills.
- 11.
Saavedra and Opfer (2012) adopt nine lessons from learning science for teaching ‘21st century skills’, consistent with other messages in this book: teach through the disciplines; develop lower order and higher order thinking skills simultaneously; encourage transfer of learning; teach students to learn to learn; address misunderstandings directly; recognise teamwork as both a learning outcome and a learning facilitator; exploit technology to support learning; and foster students’ creativity.
- 12.
ACARA (n.d.) promotes ICT Literacy as one of several general capabilities in the national curriculum, with relevant content descriptions and elaborations in each learning area of the curriculum, and publishes rubrics for the several elements and sub-elements across a learning continuum of six levels. https://www.australiancurriculum.edu.au/media/1074/general-capabilities-information-and-communication-ict-capability-learning-continuum.pdf
- 13.
Hattie and Donoghue (2016) explore teaching strategies for the three stages of learning that they identify as surface, deep and transfer.
- 14.
Many such examples now exist. See, for example, Jin, Shin, Hokayem, Qureshi, and Jenkins (2019).
- 15.
Delandshere (2001, 2002) extended that critique by pointing out that there are many different and nuanced theories of learning that imply radically different approaches to assessment, though typically these differences are unexplicated and therefore elusive. It appears that the focus of many learning theories is on learning as a process and that outcomes are undefined. Critiquing the critique is beyond the scope of this review. Here, a socially conservative stance is accepted, that considers many theories of learning as an expression of different social values, and therefore contestable. It is accepted that knowledge and expertise are defined within a socio-political context, which is the basis for their legitimacy, but also that knowledge and expertise have assessable objective characteristics (such as required for pilots, surgeons, accountants—or any ‘expert’ or perhaps any ‘educated person’). The current argument, however, is that standardised testing is incapable of validly assessing the range of desirable learning outcomes that have been defined through analysis and consensus, as well as empirical research, as desirable for the wellbeing of individual students and society.
- 16.
- 17.
- 18.
Details on Curriculum for Excellence can be accessed through https://education.gov.scot
- 19.
The SNSAs are multiple-choice, on-line, adaptive, un-timed, flexible-delivery tests (Leng, 2019).
- 20.
These networks are sometimes referred to as schemas, described by Levy and Murnane (2004) as ‘mental structures that represent concepts and relationships that connect them’ (p. 63). It is noted that ‘expertise requires more than facts. It requires understanding how the facts are linked together, and how things actually work. These relationships allow a person to generalize from a specific case to a class of problems’ (p. 63).
- 21.
The focus here is on observed learning outcomes , though these may have been defined by considering what is desirable, for example, positive versus negative emotions.
- 22.
Further exploration of emotion in learning is beyond the scope of this book. Some of the complexities are discussed in Rowe (2017).
References
Ainley, J., Fraillon, J., Schulz, W., & Gebhardt, E. (2016). Conceptualizing and measuring computer and information literacy in cross-national contexts. Applied Measurement in Education, 29(4), 291–309. https://doi.org/10.1080/08957347.2016.1209205
Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries (OECD Education Working Paper No. 41). OECD. https://www.oecd-ilibrary.org/education/21st-century-skills-and-competences-for-new-millennium-learners-in-oecd-countries_218525261154
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2000). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Pearson, Allyn & Bacon.
Australian Council for Educational Research. (2018). Scottish national standardised assessment: National report for academic year 2017 to 2018. Scottish Government. https://www.acer.org/files/SNSA-National-report-2017-18.pdf
Australian Curriculum, Assessment and Reporting Authority. (2018). National assessment program sample assessment—ICT literacy: Years 6 and 10. https://www.nap.edu.au/docs/default-source/default-document-library/2017napictlreport_final.pdf?sfvrsn=2
Australian Curriculum, Assessment and Reporting Authority. (2019). Program of research: Key findings from four international comparative studies. https://www.australiancurriculum.edu.au/media/4394/ac-bcc-fncc-sc-nzc-key-findings.pdf
Australian Curriculum, Assessment and Reporting Authority. (n.d.). About the Australian Curriculum. https://www.australiancurriculum.edu.au/about-the-australian-curriculum/
Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change: An empirical exploration. The Quarterly Journal of Economics, 118, 1279–1333. https://doi.org/10.1162/003355303322552801
Autor, D. H., & Price, B. (2013). The changing task composition of the US labor market. MIT Economics. http://economics.mit.edu/files/9758
Baas, D., Castelijns, J., Vermeulen, M., Martens, R., & Segers, M. (2015). The relation between assessment for learning and elementary students’ cognitive and metacognitive strategy use. British Journal of Educational Psychology, 85(1), 33–46. https://doi.org/10.1111/bjep.12058
Bannert, M., & Mengelkamp, C. (2007). Assessment of metacognitive skills by means of instruction to think aloud and reflect when prompted: Does the verbalisation method affect learning? Metacognition and Learning, 3(1), 39–58. https://doi.org/10.1007/s11409-007-9009-6
Bereiter, C., & Scardamalia, M. (2012). What will it mean to be an educated person in the 21st century? The Gordon Commission on the Future of Assessment in Education. https://www.ets.org/research/policy_research_reports/gordon_commission?WT.ac=research_policyresearchreports_gordoncommission_180223
Berg, J., Osher, D., Same, M. R., Nolan, E., Benson, D., & Jacobs, N. (2017). Identifying, defining, and measuring social and emotional competencies: Final report. American Institutes for Research. https://www.air.org/sites/default/files/downloads/report/Identifying-Defining-and-Measuring-Social-and-Emotional-Competencies-December-2017-rev.pdf
Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., et al. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). Springer. https://doi.org/10.1007/978-94-007-2324-5_2
Birenbaum, M. (1996). Assessment 2000: Towards a pluralistic approach to assessment. In M. Birenbaum & F. J. R. C. Dochy (Eds.), Alternatives in assessment of achievements, learning processes and prior knowledge (pp. 3–29). Springer. https://doi.org/10.1007/978-94-011-0657-3_1
Blakemore, S.-J., & Firth, U. (2005). The learning brain: Lessons for education. Blackwell.
Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Book I: Cognitive domain. Longman, Green.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience and school. National Academy Press.
Broudy, H. S. (1982). What knowledge is of most worth? Educational Leadership, 39(8), 574–578.
Caena, F. (2019). Developing a European framework for the personal, social and learning to learn competence (LifEComp): Literature review and analysis of frameworks. Publications Office of the European Union.
Campbell, M. (2019). Standard setting in Queensland: The Queensland Certificate of Education. In J.-A. Baird, T. Isaacs, D. Opposs, & L. Gray (Eds.), Examination standards: How measures and meanings differ around the world (pp. 182–205). University College, Institute of Education Press.
Care, E. (2018). Twenty-first century skills: From theory to action. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 3–17). Springer. https://doi.org/10.1007/978-3-319-65368-6_1
Care, E., Anderson, K., & Kim, H. (2016). Visualizing the breadth of skills movement across education systems: Skills for a changing world. The Brookings Institution.
Care, E., & Kim, H. (2018). Assessment of 21st century skills: The issue of authenticity. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 21–39). Springer. https://doi.org/10.1007/978-3-319-65368-6
Care, E., Kim, H., Vista, A., & Anderson, K. (2018). Education system alignment for 21st century skills: Focus on assessment. The Brookings Institution.
Care, E., & Luo, R. (2016). Assessment of transversal competencies: Policy and practice in the Asia-Pacific region. UNESCO.
Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29(4), 250–264. https://doi.org/10.1080/08957347.2016.1209204
Care, E., Vista, A., & Kim, H. (2019). Assessment of transversal competencies: Current tools in the Asian region. UNESCO.
Care, E., Vista, A., & Kim, H. (2020). Optimizing assessment for all: Focus on Asia. The Brookings Institution.
Chalkiadaki, A. (2018). A systematic literature review of 21st century skills and competencies in primary education. International Journal of Instruction, 11(3), 1–16. https://doi.org/10.12973/iji.2018.1131a
Cheng, K.-M. (2017). Advancing 21st century competencies in East Asian education systems. Asia Society, Center for Global Education. https://asiasociety.org/files/21st-century-competencies-east-asian-education-systems.pdf
Child, S., & Shaw, S. (2016). Collaboration in the 21st century: Implications for assessment. Research Matters, 22(Summer 2016), 17–22.
Clayton, B., Blom, K., Meyers, D., & Bateman, A. (2003). Assessing and certifying generic skills: What is happening in vocational education and training? National Centre for Vocational Education Research.
Coghlan, D., & Brydon-Miller, M. (2014). Cognitive mapping. In The SAGE encyclopedia of action research. Sage. https://doi.org/10.4135/9781446294406
Conley, D. T. (2003). Understanding university success. University of Oregon, Center for Educational Policy Research.
Conley, D. T. (2005). College knowledge: What it really takes for students to succeed and what we can do to get them ready. Jossey Bass.
Conley, D. T. (2010). College and career readiness: Helping all students succeed beyond high school. Jossey Bass. https://doi.org/10.1002/9781118269411
Conley, D. T. (2011). Redefining college readiness: Volume 5. Educational Policy Improvement Center. https://www.ct.edu/files/pdfs/p20/RedefiningCollegeReadiness.pdf
Connelly, F. M., & Xu, S. J. (2010). An overview of research in curriculum inquiry. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (pp. 324–334). Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.00067-1
Cook, R., & Weaving, H. (2012). Key competence development in school education Europe: KeyCoNet’s review of the literature: A summary. European Schoolnet. http://keyconet.eun.org/c/document_library/get_file?uuid=3a7a093c-4c8f-473c-8702-f38ed86bb730&groupId=11028
Cumming, J. J. (2020). Senior secondary school assessment and standard-setting in Queensland, Australia: Social context and paradigmatic change. Assessment in Education: Principles, Policy & Practice, 27(2), 160–177. https://doi.org/10.1080/0969594X.2019.1684877
Darling-Hammond, L. (2012). Policy frameworks for new assessments. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 301–339). Springer. https://doi.org/10.1007/978-94-007-2324-5_6
Darling-Hammond (Ed.). (2014). Next generation assessment: Moving beyond the bubble test to support 21st century learning. Jossey-Bass.
de Corte, E. (2010). Historical developments in the understanding of learning. In H. Dumont, D. Istance, & F. Beavides (Eds.), The nature of learning: Using research to inspire practice (pp. 35–67). OECD. https://doi.org/10.1787/9789264086487-en
Dede, C. (2010). Comparing frameworks for 21st century skills. In J. A. Bellanca & R. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. 51–76). Solution Tree Press.
Delandshere, G. (2001). Implicit theories, unexamined assumptions and status quo of educational assessment. Assessment in Education: Principles, Policy & Practice, 8(2), 113–133. https://doi.org/10.1080/09695940123828
Delandshere, G. (2002). Assessment as inquiry. Teachers College Record, 104(7), 1461–1484.
Deming, D. J. (2015). The growing importance of social skills in the labor market (Working Paper No. 21473). Bureau of Economic Research. https://doi.org/10.3386/w21473
Desoete, A. (2008). Multi-method assessment of cognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3, 189–206. https://doi.org/10.1007/s11409-008-9026-0
Detterman, D. K. (1993). The case for prosecution: Transfer as an epiphenomenon. In D. K. Detterman & R. J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition, and instruction (pp. 39–67). Ablex Publishing Corp.
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327
Dumont, H., & Istance, D. (2010). Analysing and designing learning environments for the 21st century. In H. Damont, D. Istance, & F. Beavides (Eds.), The nature of learning: Using research to inspire practice (pp. 19–34). OECD. https://doi.org/10.1787/9789264086487-en
Ercikan, K., & Oliveri, M. E. (2016). In search of validity evidence in support of the interpretation and use of assessments of complex constructs: Discussion of research on assessing 21st century skills. Applied Measurement in Education, 29(4), 310–318. https://doi.org/10.1080/08957347.2016.1209210
European Commission. (2020). Supporting key competence development: Learning approaches and environments in school education: Conference report. Publications Office of the European Union. https://op.europa.eu/en/publication-detail/-/publication/37cc844f-4248-11ea-9099-01aa75ed71a1/language-en/format-PDF/source-search
European Council. (2018). Council recommendation of 22 May 2018 on key competences for lifelong learning. Official Journal of the European Union, 61(C 189) https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32018H0604(01)&rid=7
European Union. (2019). Key competencies and lifelong learning. Office for Official Publications of the European Communities.
Eurydice. (2009). National testing of pupils in Europe: Objectives, organisation and use of results. Education, Audiovisual and Culture Agency. http://eurydice.org
Ferrari, A. (2013). DIGCOMP: A framework for developing and understanding digital competence in Europe. Publications Office of the European Union.
Fischer, K. W. (2009). Mind, brain, and education: Building a scientific groundwork for learning and teaching. Mind, Brain, and Education, 3(1), 3–16. https://doi.org/10.1111/j.1751-228X.2008.01048.x
Fischer, K. W., & Bidell, T. R. (2006). Dynamic development of action, thought and emotion. In W. Damon & R. M. Lerner (Eds.), Theoretical models of human development: Handbook of child psychology (Vol. 1, pp. 331–339). Wiley. https://doi.org/10.1002/9780470147658.chpsy0107
Fischer, K. W., Goawani, U., Geake, J., & The Task Force on the Future of Educational Neuroscience. (2010). The future of educational neuroscience. Mind, Brain, and Education, 4(2), 69–80. https://doi.org/10.1111/j.1751-228X.2010.01086.x
Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., & Friedman, T. (2019). IEA international computer and information literacy study 2018 assessment framework. Springer. https://doi.org/10.1007/978-3-030-19389-8
Funke, J., Fischer, A., & Holt, D. V. (2018). Competencies for complexity: Problem solving in the 21st century. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 41–53). Springer. https://doi.org/10.1007/978-3-319-65368-6_3
Gallés, N. S. (2004). A primer on learning: A brief introduction from the neurosciences. OECD.
Geisinger, K. F. (2016). 21st century skills: What are they and how do we assess them? Applied Measurement in Education, 29(4), 245–249. https://doi.org/10.1080/08957347.2016.1209207
Goetz, T., Keller, M. M., Lüdtke, O., Nett, U. E., & Lipnevich, A. A. (2019). The dynamics of real-time classroom emotions: Appraisals mediate the relation between students’ perceptions of teaching and their emotions. Journal of Educational Psychology, 1–18. https://doi.org/10.1037/edu0000415
Gordon, E. W., & Campbell, E. B. (2012). Context and perspective: Implications for assessment in education. The Gordon Commission on the Future of Assessment in Education. https://www.ets.org/research/policy_research_reports/gordon_commission?WT.ac=research_policyresearchreports_gordoncommission_180223
Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., et al. (2009). Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education. Center for Social and Economic Research. https://doi.org/10.2139/ssrn.1517804
Gordon, J., Rey, O., Siewiorek, A., Vivitsou, M., & von Reis Saari, J. (2012). KeyCoNet 2012 literature review: Key competence development in school education in Europe. European Schoolnet. https://hal-ens-lyon.archives-ouvertes.fr/ensl-01576387/document
Hattie, J., & Donoghue, G. M. (2016). Learning strategies: A synthesis and conceptual model. Science of Learning, 1, 1–13. https://doi.org/10.1038/npjscilearn.2016.13
Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. https://doi.org/10.4324/9780203887332
Hayward, E. L. (2007). Curriculum, pedagogies and assessment in Scotland: The quest for social justice. Assessment in Education: Principles, Policy & Practice, 14(2), 251–268. https://doi.org/10.1080/09695940701480178
Hayward, L. (2015). Assessment is learning: The preposition vanishes. Assessment in Education: Principles, Policy & Practice, 22(1), 27–43. https://doi.org/10.1080/0969594X.2014.984656
Herde, C. N., Wustenberg, S., & Greiff, S. (2016). Assessment of complex problem solving: What we know and what we don’t know. Applied Measurement in Education, 29(4), 265–277. https://doi.org/10.1080/08957347.2016.1209208
Herman, J., & Hilton, M. (2017). Supporting students’ college success: The role of assessment of intrapersonal and interpersonal competencies. The National Academies Press. https://doi.org/10.17226/24697
Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 37–56). Springer. https://doi.org/10.1007/978-94-017-9395-7_2
Hill, P., & Barber, M. (2014). Preparing for a renaissance in assessment. Pearson. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/uk/documents/educator/primary/preparing_for_a_renaissance_in_assessment_and_summary_text_december_2014.pdf
Hinton, C., & Fischer, K. W. (2010). Learning from a developmental and biological perspective. In H. Damont, D. Istance, & F. Beavides (Eds.), The nature of learning: Using research to inspire practice (pp. 113–129). OECD. https://doi.org/10.1787/9789264086487-en
Hinton, C., Miyamoto, K., & della Chiesa, B. (2008). Brain research, learning and emotions: Implications for education research, policy, and practice. European Journal of Education, 43(1), 87–103. https://doi.org/10.1111/j.1465-3435.2007.00336.x
Immordino-Yang, M. H., & Damasio, A. (2007). We feel, therefore we learn: The relevance of affective and social neuroscience to education. Mind, Brain, and Education, 1(1), 3–10. https://doi.org/10.1111/j.1751-228X.2007.00004.x
Jensen, E. (2005). Teaching with the brain in mind (2nd ed.). ASCD.
Jin, H., Shin, H. J., Hokayem, H., Qureshi, F., & Jenkins, T. (2019). Secondary students’ understanding of ecosystems: A learning progression approach. International Journal of Science and Mathematics Education, 17(2), 217–235. https://doi.org/10.1007/s10763-017-9864-9
Kaarakainen, M.-T., Kivinen, O., & Vainio, T. (2018). Performance-based testing for ICT skills assessing; A case study of students’ and teachers’ ICT skills in Finnish schools. Universal Access in the Information Society, 17(2), 349–360. https://doi.org/10.1007/s10209-017-0553-9
Kaponen, I. T., & Nousiainen, M. (2018). Concept networks of students’ knowledge of relationships between physics concepts: Finding key concept and their epistemic support. Applied Network Science, 3, Article 14. https://doi.org/10.1007/s41109-018-0072-5
Kay, K. (2010). 21st century skills: Why they matter, what they are, and how we get there. In J. A. Bellanca & R. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. xiii–xxxi). Solution Tree Press.
Kereluik, K., Mishra, P., Fahnoe, C., & Terry, L. (2013). What knowledge is of most worth? Journal of Digital Learning in Teacher Education, 29(4), 127–140. https://doi.org/10.1080/21532974.2013.10784716
Koenig, J. A. (2011). Assessing 21st century skill: Summary of a workshop. The National Academies Press.
Koh, K. (2014). Authentic assessment, teacher judgment and moderation in a context of high accountability. In C. Wyatt-Smith, V. Klenowski, & P. Colbert (Eds.), Designing assessment for quality learning. Springer. https://doi.org/10.1007/978-94-007-5902-2_16
Lai, E. R. (2011). Metacognition: A literature review. Pearson. http://images.pearsonassessments.com/images/tmrs/Metacognition_Literature_Review_Final.pdf
Lai, E. R., & Viering, M. (2012). Assessing 21st century skills: Integrating research findings. National Council on Measurement in Education. http://images.pearsonassessments.com/images/tmrs/Assessing_21st_Century_Skills_NCME.pdf
Lake, R. (2019, March 26). Preparing students for the uncertain future: Why America’s educators are ready to innovate—But their education systems are not. The 74. https://www.the74million.org/article/preparing-students-for-the-uncertain-future-why-americas-educators-are-ready-to-innovate-but-their-education-systems-are-not/
Leng, D. (2019, August 4–5). Education reform—Scottish style! (Paper presentation). Australian Council for Educational Research Research Conference, Melbourne, VIC, Australia. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1347&context=research_conference
Leutner, D., Fleischer, J., Grünkorn, J., & Klieme, E. (2017). Competence assessment in education; Research, models and instruments. Springer. https://doi.org/10.1007/978-3-319-50030-0
Levy, F., & Murnane, R. (2004). The new division of labor: How computers are creating the next job market. Princeton University Press. https://doi.org/10.1515/9781400845927
Levy, F., & Murnane, R. (2013). Dancing with robots: Human skills for computerized work. Third Way. http://content.thirdway.org/publications/714/Dancing-With-Robots.pdf
Lichtenfeld, S., Pekrun, R., Stupnisky, R. H., Reiss, K., & Murayama, K. (2012). Measuring students’ emotions in the early years: The Achievement Emotions Questionnaire – Elementary School (AEQ-ES). Learning and Individual Differences, 22(2), 190–201. https://doi.org/10.1016/j.lindif.2011.04.009
Lucas, B. (2016). A five-dimensional model of creativity and its assessment in schools. Applied Measurement in Education, 29(4), 278–290. https://doi.org/10.1080/08957347.2016.1209206
Martin-Loeches, M. (2015). Neuroscience and education: We already reached the tipping point. Psicología Educativa, 21(2), 67–70. https://doi.org/10.1016/j.pse.2015.09.001
Marulis, L. M., Palincsar, A. S., Berhenke, A. L., & Whitebread, D. (2016). Assessing metacognitive knowledge in 3–5 year olds: The development of a metacognitive knowledge interview. Metacognition and Learning, 11(3), 339–368. https://doi.org/10.1007/s11409-016-9157-7
Marzano, R., & Kendall, J. (2007). The new taxonomy of educational objectives (2nd ed.). Corwin Press.
Maxwell, G. S. (2006, May 21–26). Quality management of school-based assessments: Moderation of teacher judgments (Paper presentation). 32nd Annual International Association for Educational Assessment Annual Conference, Singapore. https://iaea.info/conference-proceedings/32nd-annual-conference-2006/
Maxwell, G. S. (2010). Moderation of student work by teachers. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 457–463). Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.00347-X
Maxwell, G. S. (2019). Managing the tension between performance standards and aggregate ranking. In J.-A. Baird, T. Isaacs, D. Opposs, & L. Gray (Eds.), Examination standards: How measures and meanings differ around the world (pp. 206–208). University College, Institute of Education Press.
Maxwell, G. S., & Cumming, J. J. (2011). Managing without public examinations: Successful and sustained curriculum and assessment reform in Queensland. In L. Yates, C. Collins, & K. O'Connor (Eds.), Australia’s curriculum dilemmas: State perspectives and changing times (pp. 202–222). Melbourne University Press.
Mayer, R. E. (2017). How can brain research inform academic learning and instruction? Educational Psychology Review, 29(4), 835–846. https://doi.org/10.1007/s10648-016-9391-1
Microsoft Partners in Learning. (2012). 21st century learning design: Student work rubrics. https://www.creatingrounds.com/uploads/9/6/2/4/96240662/21cld_student_work_rubrics_2012.pdf
Morris, A. (2011). Student standardised testing: Current practices in OECD countries and a literature review (OECD Working Paper No. 65). OECD. https://doi.org/10.1787/19939019
Nusche, D., Earl, L., Maxwell, W., & Shewbridge, C. (2011). OECD reviews of evaluation and assessment in education: Norway. OECD. https://doi.org/10.1787/22230955
Oatley, K., & Johnson-Laird, P. N. (2014). Cognitive approaches to emotions. Trends in Cognitive Sciences, 18(3), 134–140. https://doi.org/10.1016/j.tics.2013.12.004
Ontario Ministry of Education. (2016). 21st century competencies for Ontario: Foundation document for discussion. https://ocea.on.ca/wp-content/uploads/2018/02/21cl-21stcenturycompetencies.pdf
Organisation for Economic and Cultural Development (OECD). (2007). Understanding the brain: The birth of a learning science. OECD.
Organisation for Economic and Cultural Development (OECD). (2011). Educational evaluation and assessment in Norway: Strengths, challenges and policy pointers. http://www.oecd.org/edu/school/48632053.pdf
Organisation for Economic and Cultural Development (OECD). (2018). PISA 2018 assessment and analytical framework. https://doi.org/10.1787/b25efab8-en
Paralkar, V. K. (2020). The Tyler rationale: A model of curriculum architecture. In T. S. Poetter, D. C. Murray, P. Larrick, M. A. Moyer, E. C. Berlioz, & K. Waldrop (Eds.), Curriculum windows: What curriculum theorists of the 1950s can teach us about schools and society today (pp. 139–150). Information Age Publishing.
Partnership for 21st Century Learning (P21). (2019). Framework for 21st century learning definitions. Battelle for Kids. http://static.battelleforkids.org/documents/p21/P21_Framework_DefinitionsBFK.pdf
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. National Academy Press. https://www.nap.edu/download/10019
Pelligrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. The National Academies Press.
Pepper, D. (2011). Assessing key competences across the curriculum—And Europe. European Journal of Education, 46(3), 335–353. https://doi.org/10.1111/j.1465-3435.2011.01484.x
Pepper, D. (2013). KeyCoNet 2013 literature review: Assessment for key competences. Key Competence Network on School Education. https://obrazovanje.org/rs/uploaded/dokumenta/keyconet-2013-literature-review_-assessment-for-key-competences.pdf
Perkins, D. (1992). Smart schools: From training memories to educating minds. Free Press.
Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher, 18(1), 16–25. https://doi.org/10.3102/0013189X018001016
Prøitz, T. S. (2010). Learning outcomes: What are they? Who defines them? When and where are they defined? Educational Assessment Evaluation and Accountability, 22(2), 119–137. https://doi.org/10.1007/s11092-010-9097-8
Reeves, D. (2010). A framework for assessing 21st century skills. In J. A. Bellanca & R. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. 305–325). Solution Tree Press.
Reich, R. (1992). The work of nations: Preparing ourselves for 21st century capitalism. Vintage Books.
Roebers, C. M., Kälin, S., & Aeschlimann, E. A. (2020). A comparison of non-verbal and verbal indicators of young children’s metacognition. Metacognition and Learning, 15(1), 31–49. https://doi.org/10.1007/s11409-019-09217-4
Rovers, S. F. E., Clarebout, G., Savelberg, H. H. C. M., de Bruin, A. B. H., & Merriënboer, J. J. G. (2019). Granularity matters: Comparing different ways of measuring self-regulated learning. Metacognition and Learning, 14(1), 1–19. https://doi.org/10.1007/s11409-019-09188-6
Rowe, A. (2017). Feelings about feedback: The role of emotions in assessment for learning. In D. Carless, S. M. Bridges, C. K. Y. Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 159–172). Springer. https://doi.org/10.1007/978-981-10-3045-1_11
Russell, C. A. (2016). System supports for 21st century competencies. Asia Society, Center for Global Education. https://asiasociety.org/files/system-supports-for-21st-century-competencies-2016_0.pdf
Saavedra, A.R., & Opfer, V.D. (2012). Teaching and learning 21st century skills: Lessons from the learning sciences. Asia Society; RAND Corporation. https://asiasociety.org/sites/default/files/T/teaching-and-learning-21st-century-skills-rand.pdf
Scalise, K. (2018). Next wave for integration of educational technology into the classroom: Collaborative technology integration planning practices. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 239–255). Springer. https://doi.org/10.1007/978-3-319-65368-6_14
Schneider, M., & Stern, E. (2010). The cognitive perspective on learning: Ten cornerstone findings. In H. Dumont, D. Istance, & F. Beavides (Eds.), The nature of learning: Using research to inspire practice (pp. 69–90). OECD. https://doi.org/10.1787/9789264086487-en
Schwartz, M. C., & Paré-Blagoev, E. J. (Eds.). (2017). Research in mind, brain, and education. Routledge. https://doi.org/10.4324/9781315670621a
Schweig, J., Baker, G., Hamilton, L. S., & Stecher, B. M. (2018). Building a repository of assessments of interpersonal, intrapersonal, and higher-order cognitive competencies. RAND Corporation. https://doi.org/10.7249/RR2508
Scottish Government. (2010). Curriculum for excellence: Building the curriculum 5: A framework for assessment: Quality assurance and moderation. https://www.education.gov.scot/Documents/btc5-qa-moderation.pdf
Scottish Government. (2019). National standardised assessments in Scotland: Purpose and use. https://www.gov.scot/publications/scottish-national-standardised-assessments-purpose-and-use/
Shaw, S., Kuvalja, M., & Suto, I. (2018). An exploration of the nature and assessment of student reflection. Research Matters: A Cambridge Assessment Publication, 25, 2–8.
Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4–14. https://doi.org/10.3102/0013189X029007004
Siddiq, F., Gochyyev, P., & Wilson, M. (2017). Learning in digital networks: ICT literacy: A novel assessment of students’ 21st century skills. Computers and Education, 109, 11–37. https://doi.org/10.1016/j.compedu.2017.01.014
Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past: A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84. https://doi.org/10.1016/j.edurev.2016.05.002
Silva, E. (2008). Measuring skills for the 21st century. Education Sector.
Smith, J. K., Smith, L. F., Gilmore, A., & Jameson, M. (2012). Students’ self-perception of reading ability, enjoyment of reading and reading achievement. Learning and Individual Differences, 22(2), 202–206. https://doi.org/10.1016/j.lindif.2011.04.010
Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st century competencies: Guidance for educators. Asia Society; RAND Corporation.
Stehle, S. M., & Peters-Burton, E. E. (2019). Developing student 21st century skills in selected exemplary inclusive STEM high schools. International Journal of STEM Education, 6(1), 1–15. https://doi.org/10.1186/s40594-019-0192-1
Stobart, G. (2014). The expert learner: Challenging the myth of ability. Open University Press.
Suto, I. (2013). 21st century skills: Ancient, ubiquitous, enigmatic? Research Matters: A Cambridge Assessment Publication, 15, 2–8.
Tomas, L., Rigano, D., & Ritchie, S. M. (2016). Students’ regulation of their emotions in a science classroom. Journal of Research in Science Teaching, 53(2), 234–260. https://doi.org/10.1002/tea.21304
Tuck, J. L., & Moxley, J. H. (2017). A comprehensive reanalysis of the metacognitive self-regulation scale from MSLQ. Metacognition and Learning, 12(1), 79–111. https://doi.org/10.1007/s11409-016-9161-y
Tveit, S. (2009). Educational assessment in Norway: A time of change. In C. Wyatt-Smith & J. J. Cumming (Eds.), Educational assessment in the 21st century (pp. 227–243). Springer. https://doi.org/10.1007/978-1-4020-9964-9_12
Tveit, S. (2014). Educational assessment in Norway. Assessment in Education: Principles, Policy & Practice, 21(2), 221–237. https://doi.org/10.1080/0969594X.2013.830079
Tveit, S. (2018). Ambitious and ambiguous: Shifting purposes of national testing in the legitimation of assessment policies in Norway and Sweden (2000–2017). Assessment in Education: Principles, Policy & Practice, 25(3), 327–350. https://doi.org/10.1080/0969594X.2017.1421522
Tyler, R. (1949). Basic principles of curriculum and instruction. University of Chicago Press.
United Nations Conference on Trade and Development (UNCTAD) Howard-Suamico School District, Wisconsin. (2017). Information economy report 2017: Digitalization, trade and development. United Nations.
United Nations Educational, Scientific and Cultural Organization. (2013). Toward universal learning: What every child should learn. UNESCO Institute for Statistics; Center for Universal Education at Brookings.
United Nations Educational, Scientific and Cultural Organization. (2019). Education content up close: Examining the learning dimensions of education for sustainable development and global citizenship education. UNESCO.
van Deursen, A. J. A. M., Helsper, E. J., & Eyon, R. (2016). Development and validation of the Internet Skills Scale (ISS). Information Communication and Society, 19(4), 804–823. https://doi.org/10.1080/1369118X.2015.1078834
Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321. https://doi.org/10.1080/00220272.2012.668938
Wagner, T. (2008). The global achievement gap: Why even our best schools don’t teach the new survival skills our children need—And what we can do about it. Basic Books.
Wang, Y., Lavonen, J., & Tirri, K. (2018). Aims for learning 21st century competencies in national primary science curricula in China and Finland. EURASIA Journal of Mathematics, Science and Technology Education, 14(6), 2081–2095. https://doi.org/10.29333/ejmste/86363
Wilson, M., Scalise, K., & Gochyyev, P. (2015). Rethinking ICT literacy: From computer skills to social network. Thinking Skills and Creativity, 18, 65–80. https://doi.org/10.1016/j.tsc.2015.05.001
Wilson, M., Scalise, K., & Gochyyev, P. (2018a). Learning in digital networks as a modern approach to ICT literacy. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 181–210). Springer. https://doi.org/10.1007/978-3-319-65368-6_11
Wilson, M., Scalise, K., & Gochyyev, P. (2018b). ICT literacy as a 21st century skill: Learning in digital networks with agile development approaches. In H. Jiao & R. W. Lissitz (Eds.), Technology enhanced innovative assessment: Development, modeling, and scoring from an interdisciplinary perspective (pp. 249–264). Information Age Publishing.
Wraga, W. G. (2008). Curriculum development. In G. McCulloch & D. Crook (Eds.), The Routledge international encyclopedia of education (pp. 154–156). Routledge.
Yves, K. (2016). Perceived learning environments and metacognitive strategy knowledge at the upper secondary school level. Journal for Educational Research Online, 8(2), 212–232.
Zadina, J. N. (2015). The emergent role of educational neuroscience in education reform. Psicología Educativa, 21(2), 71–77. https://doi.org/10.1016/j.pse.2015.08.005
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Maxwell, G.S. (2021). Defining and Assessing Desired Learning Outcomes. In: Using Data to Improve Student Learning. The Enabling Power of Assessment, vol 9. Springer, Cham. https://doi.org/10.1007/978-3-030-63539-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-63539-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63537-4
Online ISBN: 978-3-030-63539-8
eBook Packages: EducationEducation (R0)