Skip to main content

Advertisement

SpringerLink
Log in

Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

The Digital Competency Profiler (DCP) is an online application for surveying the technology preferences and abilities of students in higher education. To explore the DCP as a digital-learning-readiness tool, a mixed-methods research design was developed for relating self-reported digital competencies and online-learning activity. To this end, three authentic scenarios, comprised of six tasks mapped to self-report items, were constructed. Having submitted their survey data, each of 15 participants visited the EILAB to complete a randomly-assigned scenario with a tablet. Both the performance activity and post-activity interviews were recorded digitally using a unique activity-station setup, and task artefacts were gathered as performance outcomes. Analysis was conducted in three phases. In Phase 1, both the audio-video performance data and activity artefacts were coded, assessed and scored. Exploratory correlational analyses showed a pattern of positive relationships at the task and scenario levels for two scenario groups, suggesting some predictive value for the DCP in this context. For the third group, a positive correlation was found at the scenario level, but negative correlations were found at the task level. In Phase 2, detailed case-studies were conducted, incorporating self-report data, coded performance timelines, and post-activity interviews. Several situational influencers related to problem-solving strategy, device comfort, task difficulty and motivation, beyond the purview of the DCP, were identified. In Phase 3, the findings were interpreted to position the DCP as a tool for identifying segments of students with members who, without support, will likely struggle to engage fully in technology-rich learning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Time markers in this section are provided in minutes and seconds from the beginning of the synchronized audio-video recordings.

References

  • Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. In Seville: Institute for Prospective Technological Studies (IPTS). European Commission: Joint Research Centre Retrieved from http://ftp.jrc.es/EURdoc/JRC67075_TN.pdf.

    Google Scholar 

  • Al-Araibi, A. A. M., Mahrin, M., & Mohd, R. C. (2016). A systematic literature review of technological factors for e-learning readiness in higher education. Journal of Theoretical and Applied Information Technology, 93(2), 500–521.

    Google Scholar 

  • Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Journal of Educational Technology & Society, 19(1), 292–307. http://www.jstor.org/stable/10.2307/jeductechsoci.19.1.292

  • Asselin, M., & Moayeri, M. (2010). New tools for new literacies research: An exploration of usability testing software. International Journal of Research & Method in Education, 33(1), 41–53. https://doi.org/10.1080/17437271003597923.

    Article  Google Scholar 

  • Aydın, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: Reflections from an emerging country. Educational Technology and Society, 8(4), 244–257. http://www.jstor.org/stable/jeductechsoci.8.4.244

  • Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148. https://doi.org/10.1207/s15326985ep2802_3.

    Article  Google Scholar 

  • Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: Designing for 21st century learning. New York: Routledge.

    Google Scholar 

  • Bertaux, D. (1981). From the life-history approach to the transformation of sociological practice. In D. Bertaux (Ed.), Biography and society: The life history approach in Social Sciences (pp. 29–45). London: Sage.

  • Bhatt, I., & de Roock, R. (2014). Capturing the sociomateriality of digital literacy events. Research in learning technology, 21, 21281. https://doi.org/10.3402/rlt.v21.21281.

    Article  Google Scholar 

  • Blayone, T., Mykhailenko, O., vanOostveen, R., Grebeshkov, O., Hrebeshkova, O., & Vostryakov, O. (2017a). Surveying digital competencies of university students and professors in Ukraine for fully online collaborative learning. Technology, Pedagogy and Education. https://doi.org/10.1080/1475939X.2017.1391871.

  • Blayone, T., vanOostveen, R., Barber, W., DiGiuseppe, M., & Childs, E. (2017b). Democratizing digital learning: Theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 13. https://doi.org/10.1186/s41239-017-0051-4.

    Article  Google Scholar 

  • Bradlow, E. T., Hoch, S. J., & Hutchinson, J. W. (2002). An assessment of basic computer proficiency among active internet users: Test construction, calibration, antecedents and consequences. Journal of Educational and Behavioral Statistics, 27(3), 237–253. https://doi.org/10.3102/10769986027003237.

    Article  Google Scholar 

  • Bui, T. X., Sankaran, S., & Sebastian, I. M. (2003). A framework for measuring national e-readiness. International Journal of Electronic Business, 1(1), 3–22. https://doi.org/10.1504/ijeb.2003.002162.

    Article  Google Scholar 

  • Crompton, H., Burke, D., Gregory, K. H., & Gräbe, C. (2016). The use of mobile learning in science: A systematic review. Journal of Science Education and Technology, 1–12. https://doi.org/10.1007/s10956-015-9597-x.

  • Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/s15327965pli1104_01.

    Article  Google Scholar 

  • Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness: Reference model suggestions. International Journal of Progressive Education, 11(1), 173–194.

    Google Scholar 

  • Desjardins, F. J. (2005). Information and communication technology in education: A competency profile of francophone secondary school teachers in Ontario. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(1), 1–14. https://doi.org/10.21432/T2PG69.

    MathSciNet  Google Scholar 

  • Desjardins, F. J., & Peters, M. (2007). Single-course approach versus a program approach to develop technological competencies in pre-service language teaching. In M.-A. Kassen, L. Lavine, K. Murphy-Judy, & M. Peters (Eds.), Preparing and developing technology proficient L2 teachers (pp. 3–21). Texas: Texas State University.

    Google Scholar 

  • Desjardins, F. J., & vanOostveen, R. (2015). Faculty and student use of digital technology in a "laptop" university. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), EdMedia: World Conference on Educational Media and Technology 2015 (pp. 990-996). Montreal: Association for the Advancement of Computing in Education (AACE).

  • Desjardins, F. J., Lacasse, R., & Belair, L. M. (2001). Toward a definition of four orders of competency for the use of information and communication technology (ICT) in education. Paper presented at the computers and advanced Technology in Education. Canada: Banff http://eilab.ca/wp-content/uploads/2013/04/2001CATE.pdf.

    Google Scholar 

  • Desjardins, F. J., vanOostveen, R., Bullock, S., DiGiuseppe, M., & Robertson, L. (2010). Exploring graduate student’s use of computer-based technologies for online learning. In J. Herrington & C. Montgomerie (Eds.), EdMedia: World Conference on Educational Media and Technology 2010 (pp. 440-444). Norfolk: Association for the Advancement of Computing in Education (AACE).

  • DiGiuseppe, M., Partosoedarso, E., vanOostveen, R., & Desjardins, F. J. (2013). Exploring competency development with mobile devices. In M. B. Nunes & M. McPherson (Eds.), International Association for Development of the information society (IADIS) international conference on e-learning (pp. 384–388). Prague: International Association for Development of the Information Society.

    Google Scholar 

  • Ding, R., & Ma, F. (2013). Assessment of university student web searching competency by a task-based online test: A case study at Wuhan University, China. The Electronic Library, 31(3), 359–375. https://doi.org/10.1108/EL-03-2011-0044.

    Article  Google Scholar 

  • Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz-Primo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. Distance Education, 32(1), 29–47. https://doi.org/10.1080/01587919.2011.565496.

    Article  Google Scholar 

  • Esbjörnsson, M., Brown, B., Juhlin, O., Normark, D., Östergren, M., & Laurier, E. (2006). Watching the cars go round and round: Designing for active spectating. In. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, & G. Olson (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1221–1224). New York: ACM.

    Google Scholar 

  • Eshet-Alkalai, Y., & Amichai-Hamburger, Y. (2004). Experiments in digital literacy. Cyberpsychology & Behavior, 7(4), 421–429.

    Article  Google Scholar 

  • Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. Electronic Journal of e-Learning, 12(4), 375–382.

    Google Scholar 

  • Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8. https://doi.org/10.1016/j.iheduc.2005.11.001.

    Article  Google Scholar 

  • Gay, G. (2016). An assessment of online instructor e-learning readiness before, during, and after course delivery. Journal of Computing in Higher Education, 28(2), 199–220. https://doi.org/10.1007/s12528-016-9115-z.

    Article  Google Scholar 

  • Greene, J. A., Seung, B. Y., & Copeland, D. Z. (2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55–69. https://doi.org/10.1016/j.compedu.2014.03.008.

    Article  Google Scholar 

  • Hargittai, E. (2002). Beyond logs and surveys: In-depth measures of people's web use skills. Journal of the American Society for Information Science and Technology, 53(14), 1239–1244. https://doi.org/10.1002/asi.10166.

    Article  Google Scholar 

  • Hargittai, E., & Shafer, S. (2006). Differences in actual and perceived online skills: The role of gender. Social Science Quarterly, 87(2), 432–448. https://doi.org/10.1111/j.1540-6237.2006.00389.x.

    Article  Google Scholar 

  • Herrington, J., Reeves, T. C., & Oliver, R. (2006). Authentic tasks online: A synergy among learner, task, and technology. Distance Education, 27(2), 233–247. https://doi.org/10.1080/01587910600789639.

    Article  Google Scholar 

  • Horzum, M. B., Kaymak, Z. D., & Gungoren, O. C. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. Educational Sciences: Theory and Practice, 15(3), 759–770. 10.12738/estp.2015.3.2410.

    Google Scholar 

  • Hung, M.-L. (2016). Teacher readiness for online learning: Scale development and teacher perceptions. Computers & Education, 94, 120–133. https://doi.org/10.1016/j.compedu.2015.11.012.

    Article  Google Scholar 

  • Hung, M.-L., Chou, C., & Chen, C.-H. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080–1090. https://doi.org/10.1016/j.compedu.2010.05.004.

    Article  Google Scholar 

  • IEEE. (1990). IEEE standard computer dictionary: A compilation of IEEE standard computer glossaries. In (pp. 218). New York: The Institute of Electrical and Electronics Engineers.

  • Jayroe, T. J., & Wolfram, D. (2012). Internet searching, tablet technology and older adults. Proceedings of the American Society for Information Science and Technology, 49(1), 1–3. https://doi.org/10.1002/meet.14504901236.

    Article  Google Scholar 

  • Knoblauch, H. (2012). Introduction to the special issue of qualitative research: Video-analysis and videography. Qualitative Research, 12(3), 251–254. https://doi.org/10.1177/1468794111436144.

    Article  Google Scholar 

  • Leigh, D., & Watkins, R. (2005). E-learner success: Validating a self-assessment of learner readiness for online training. In. In ASTD 2005 research-to-practice conference proceedings (pp. 121–131). Alexandria: ATD.

    Google Scholar 

  • Lin, H.-H., Lin, S., Yeh, C.-H., Wang, Y.-S., & Jansen, J. (2015). Measuring mobile learning readiness: Scale development and validation. Internet Research, 26(1), 265–287. https://doi.org/10.1108/IntR-10-2014-0241.

    Article  Google Scholar 

  • Litt, E. (2013). Measuring users’ internet skills: A review of past assessments and a look toward the future. New Media & Society, 15(4), 612–630. https://doi.org/10.1177/1461444813475424.

    Article  Google Scholar 

  • Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum qualitative Sozialforschung/Forum: Qualitative Social Research, 11(3), 1–5.

    MathSciNet  Google Scholar 

  • Merriam, S. B. (1998). Qualitative research and case study applications in education. Revised and expanded from case study research in education. San Francisco: Josey-Bass Publishers.

    Google Scholar 

  • Mosa, A. A., Naz’ri bin Mahrin, M., & Ibrrahim, R. (2016). Technological aspects of e-learning readiness in higher education: A review of the literature. Computer and Information Science, 9(1), 113–127. https://doi.org/10.5539/cis.v9n1p113.

    Article  Google Scholar 

  • Parasuraman, A. (2000). Technology readiness index (TRI) a multiple-item scale to measure readiness to embrace new technologies. Journal of Service Research, 2(4), 307–320. https://doi.org/10.1177/109467050024001.

    Article  Google Scholar 

  • Park, Y. J. (2015). My whole world’s in my palm! The second-level divide of teenagers’ mobile use and skill. New Media & Society, 17(6), 977–995. https://doi.org/10.1177/1461444813520302.

    Article  Google Scholar 

  • Parkes, M., Stein, S., & Reading, C. (2015). Student preparedness for university e-learning environments. The Internet and Higher Education, 25, 1–10. https://doi.org/10.1016/j.iheduc.2014.10.002.

    Article  Google Scholar 

  • Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5–18. https://doi.org/10.1007/bf02504769.

    Article  Google Scholar 

  • Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22.

    Google Scholar 

  • Savin-Baden, M. (2000). Problem-based learning in higher education: Untold stories. Philadelphia: Open University Press.

    Google Scholar 

  • Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: A review of the history and current state of distance, blended, and online Learning Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf.

  • Sun, X., & May, A. (2013). A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices. Advances in Human-Computer Interaction, 2013, 2. https://doi.org/10.1155/2013/619767.

    Article  Google Scholar 

  • van Deursen, A. J. A. M. (2010). Internet skills: Vital assets in an information society. (Ph.D. Thesis), University of Twente, Enschede, the Netherlands. Retrieved from http://doc.utwente.nl/75133/1/thesis_van_Deursen.pdf.

  • van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2010). Measuring internet skills. International Journal of Human-Computer Interaction, 26(10), 891–916. https://doi.org/10.1080/10447318.2010.496338.

    Article  Google Scholar 

  • van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2015). Development and validation of the internet skills scale (ISS). Information, Communication & Society, 1–20. https://doi.org/10.1080/1369118X.2015.1078834.

  • vanOostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a fully online learning communities (FOLC) model. In G. Veletsianos (Ed.), EdMedia 2016: World conference on educational media and technology (pp. 665–673). Vancouver: Association for the Advancement of Computing in Education (AACE).

    Google Scholar 

  • Watkins, R., Leigh, D., & Triner, D. (2004). Assessing readiness for e-learning. Performance Improvement Quarterly, 17(4), 66–79. https://doi.org/10.1111/j.1937-8327.2004.tb00321.x.

    Article  Google Scholar 

  • Wilhelm, J. (2016). What is the minimum sample size to run Pearsons R? (Online Expert Database). Retrieved June 7, 2017, from ResearchGate: https://www.researchgate.net/post/What_is_the_minimum_sample_size_to_run_Pearsons_R.

Download references

Author information

Authors and Affiliations

  1. Collaboritsi.com and Educational Informatics Laboratory (EILAB), Oshawa, ON, Canada

    Todd J. B. Blayone & Olena Mykhailenko

  2. Faculty of Education and Educational Informatics Laboratory (EILAB), University of Ontario Institute of Technology, Oshawa, ON, Canada

    Roland vanOostveen & Wendy Barber

Authors
  1. Todd J. B. Blayone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olena Mykhailenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roland vanOostveen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wendy Barber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Todd J. B. Blayone.

Appendices

Appendix 1: DCP activity item list

Table 5 DCP activity items with scenario mappings and ranked average competency scores
Full size table

Appendix 2: DCP reported competency, and performance score summaries with correlations by scenario2

Table 6 Scenario 1. Presentation preparation
Full size table
Table 7 Scenario 2. Democracy data analysis
Full size table
Table 8 Scenario 3. Conceptual model drawing
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Blayone, T.J.B., Mykhailenko, O., vanOostveen, R. et al. Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity. Educ Inf Technol 23, 1377–1402 (2018). https://doi.org/10.1007/s10639-017-9662-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-017-9662-6

Keywords

Search

Navigation