Simon Buckingham Shum
Ágnes Sándor
ABSTRACT
When used effectively, reflective writing tasks can deepen learners' understanding of key concepts, help them critically appraise their developing professional identity, and build qualities for lifelong learning. As such, reflecting writing is attracting substantial interest from universities concerned with experiential learning, reflective practice, and developing a holistic conception of the learner. However, reflective writing is for many students a novel genre to compose in, and tutors may be inexperienced in its assessment. While these conditions set a challenging context for automated solutions, natural language processing may also help address the challenge of providing real time, formative feedback on draft writing. This paper reports progress in designing a writing analytics application, detailing the methodology by which informally expressed rubrics are modelled as formal rhetorical patterns, a capability delivered by a novel web application. This has been through iterative evaluation on an independently human-annotated corpus, showing improvements from the first to second version. We conclude by discussing the reasons why classifying reflective writing has proven complex, and reflect on the design processes enabling work across disciplinary boundaries to develop the prototype to its current state.
- Aït-Mokhtar, S., Chanod, J-P., and Roux, C. 2002. Robustness beyond shallowness: incremental dependency parsing. Natural Language Engineering, 8, 2/3 (Aug. 2002), 121--144. DOI= http://dx.doi.org/10.1017/S1351324902002887 Google Scholar
Digital Library
- Boud, D., Keogh, R., and Walker, D. 2013. Reflection: Turning Experience into Learning. Routledge, Abingdon, Oxon.Google ScholarCross Ref
- Boud, D. and Walker, D. 1998. Promoting reflection in professional courses: the challenge of context. Studies in Higher Education, 23, 2 (Aug. 2006), 191--206. DOI= http://dx.doi.org/10.1080/03075079812331380384Google ScholarCross Ref
- Brun, C., and Hagège, C. 2003. Normalization and paraphrasing using symbolic methods. Proceedings of the Second International Workshop on Paraphrasing, Volume 16. Association for Computational Linguistics, 2003. DOI= http://dx.doi.org/10.3115/1118984.1118990 Google ScholarDigital Library
- Brun, C., and Hagège, C. 2004. Intertwining deep syntactic processing and named entity detection. Advances in Natural Language Processing. Springer: Berlin, 2004. 195-206. DOI= http://dx.doi.org/10.1007/978-3-540-30228-5_18Google ScholarCross Ref
- Brun, C., Popa, D. N., and Roux, C. 2014. XRCE: Hybrid classification for aspect-based sentiment analysis. Proc. 8th International Workshop on Semantic Evaluation (SemEval 2014), 838--842. (Dublin, Ireland, August 23-24, 2014).Google Scholar
- Burnett, M. M., and Scaffidi, C. 2011. End-User Development. The Encyclopedia of Human-Computer Interaction, 2nd Edition. (Eds. Mads Soegaard and Rikke Friis Dam). Interaction Design Foundation: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-edGoogle Scholar
- De Liddo, A., Sándor. Á., and Buckingham Shum, S. 2012. Contested collective intelligence: Rationale, technologies, and a human-machine annotation study. Computer Supported Cooperative Work (CSCW) 21.4-5 (2012): 417-448. DOI= http://dx.doi.org/10.1007/s10606-011-9155-x Google ScholarDigital Library
- Gibson, A. and Kitto, K. 2015. Analysing reflective text for learning analytics: an approach using anomaly recontextualisation. Proceedings of the Fifth International Conference on Learning Analytics & Knowledge. ACM: NY. DOI= http://dx.doi.org/10.1145/2723576.2723635 Google ScholarDigital Library
- Hatton N. and Smith, D. 1995. Reflection in teacher education: Towards definition and implementation. Teaching & Teacher Education. 11, 1 (Jan. 1995), 33--49. DOI= http://dx.doi.org/10.1016/0742-051X(94)00012-UGoogle ScholarCross Ref
- Huang, Z., ten Teije, A., van Harmelen, F., and Aït-Mokhtar, S. 2014. Semantic representation of evidence-based clinical guidelines. Proc. 6th International Workshop on Knowledge Representation for Health Care. 2014: Vienna (21 July 2014). p. 78--94. DOI= http://doi.org/10.1007/978-3-319-13281-5_6Google Scholar
- Lisacek, F., Chichester, C., Kaplan, A., and Sandor, Á. 2005. Discovering paradigm shift patterns in biomedical abstracts: application to neurodegenerative diseases. In Proceedings of the First International Symposium on Semantic Mining in Biomedicine (SMBM) (41-50).Google Scholar
- MacLean, M., Kathleen Carter, Lövstrand, L., and Moran, T. 1990. User-tailorable systems: pressing the issues with buttons. Proc. Conference on Human Factors in Computing Systems. ACM, New York, 175--182. DOI= http://dx.doi.org/10.1145/97243.97271 Google ScholarDigital Library
- Moon, J. 2010. Reflective Learning Workshop (Handout 10/07), University of Worcester, UK. http://worc.ac.uk/edu/documents/Jenny_Moon_RefLearnlong07.docGoogle Scholar
- Reidsema, C., Goldsmith, R., & Mort, P. 2010. Writing to learn: reflective practice in engineering design, Proceedings of ASEE Symposium Singapore, October 18-21Google Scholar
- Rodgers, C. 2002. Defining reflection: Another look at John Dewey and reflective thinking. Teachers College Record. 104.4 (2002): 842--866. http://www.tcrecord.org/content.asp?contentid=10890Google Scholar
- Russell, R. 2001. Reflection in higher education: A concept analysis. Innovative Higher Education. 21.6 (2001). DOI= http://dx.doi.org/10.1023/A:1010986404527Google Scholar
- Ryan, M. 2011. Improving reflective writing in higher education: a social semiotic perspective, Teaching in Higher Education, 16, 1 (Jan. 2011), 99--111. DOI= http://doi.org/10.1080/13562517.2010.507311Google ScholarCross Ref
- Ryan, M. 2010. The 4 Rs Model of Reflective Thinking, Version 1.5. Developing Reflective Approaches to Writing (DRAW) Project, Queensland University of Technology. http://www.citewrite.qut.edu.au/write/4Rs-for-students-page1-v1.5.pdfGoogle Scholar
- Ryan, M. & Ryan, M. 2012. Developing a systematic, cross-faculty approach to teaching and assessing reflection in higher education. Office of Learning and Teaching. http://www.olt.gov.au/system/files/resources/PP9_1327_Ryan_report_2012.pdfGoogle Scholar
- Sándor, Á., Kaplan, A., and Rondeau, R. 2006. Discourse and citation analysis with concept-matching. International Symposium: Discourse and Document. (Caen, 15-17 June 2006).Google Scholar
- Sándor, Á. 2007. Modeling metadiscourse conveying the author's rhetorical strategy in biomedical research abstracts. Revue Française de Linguistique Appliquée, 12, 2, 97--108.Google Scholar
- Sándor, Á., and Vorndran, A. 2009. Detecting key sentences for automatic assistance in peer reviewing research articles in educational sciences. Proceedings of the Workshop on Text and Citation Analysis for Scholarly Digital Libraries, 47th Annual Meeting of the Association for Computational Linguistics (Singapore, 2-7 Aug, 2009). Google ScholarDigital Library
- Scouller, K. 1998. The influence of assessment methods on students' learning approaches: multiple choice question examination versus assignment essay. Higher Education, 35, 4, 453--472. DOI= http://dx.doi.org/10.1023/A:1003196224280Google ScholarCross Ref
- Simsek, D., Buckingham Shum, S., Sándor, Á., Liddo, A. D., and Ferguson, R. 2013. XIP Dashboard: Visual analytics from automated rhetorical parsing of scientific metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, Apr. 8-12, 2013).Google Scholar
- Sumsion, J. and Fleet, A. 1996. Reflection: can we assess it? Should we assess it?. Assessment & Evaluation in Higher Education, 21, 2 (July. 2006), 121--130. DOI= http://dx.doi.org/10.1080/0260293960210202Google ScholarCross Ref
- Ullmann, T. D., Wild, F. and Scott, P. 2012. Comparing automatically detected reflective texts with human judgements. In 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning, Saarbruecken, Germany, September 2012, 101--116.Google Scholar
- Webster-Wright, A. 2013. The eye of the storm: a mindful inquiry into reflective practices in higher education. Reflective Practice, 14, 4 (May. 2013) 556-567. DOI= http://dx.doi.org/10.1080/14623943.2013.810618Google ScholarCross Ref
- Xerox Incremental Parser. Open Xerox Documentation: https://open.xerox.com/Services/XIPParserGoogle Scholar
Index Terms
- Reflecting on reflective writing analytics: assessment challenges and iterative evaluation of a prototype tool
Recommendations
Reflective writing analytics for actionable feedback
LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge ConferenceReflective writing can provide a powerful way for students to integrate professional experience and academic learning. However, writing reflectively requires high quality actionable feedback, which is time-consuming to provide at scale. This paper ...
Learning Analytics as a Metacognitive Tool
CSEDU 2014: Proceedings of the 6th International Conference on Computer Supported Education - Volume 1The use of learning analytics is entering in the field of research in education as a promising way to support learning. However, in many cases data are not transparent for the learner. In this regard, Educational institutions shouldnâ t escape the need ...
Effects of Learning Analytics on Students' Self-Regulated Learning in Flipped Classroom
The present article is aimed at analyzing the effects of learning analytics on students' self-regulated learning in a flipped classroom. An experiment was conducted with 96 engineering students, enrolled in a subject offered in the Flipped Classroom ...
Comments