Proposing a framework for evaluating digital creativity in social media and multimedia-based art and design education

The usage of emerging social and digital applications is growing rapidly among the current generation of students and academics, and researchers started exploring their effec tiveness in enhancing students’ creativity. However, examining the most effective criteria for evaluating digital-media-enhanced creativity in art and design still needs further exploration. This pilot study seeks to develop a framework to assess students’ creativity and another framework to assess the effectiveness of multimedia-based teaching approaches in art and design educational contexts. Sixteen design instructors participated in a survey, which aimed to identify their experiences with multimedia-based pedagogy as a potentially effective approach in fostering students’ creativity as well as educators’ innovation in teaching. The paper identifies and ranks the criteria, whi ch they thought are effective in assessing digitally-stimulated creativity in each field of graphic design. The ultimate goal is to provide educators with a set of evaluation criteria to guide the appropriate teaching and assessment of digital creativity.


Introduction
'The exhibition is presented in an extremely complex articulation' (Letocha, 1992, p. 36). We can only be determined to this affirmation by addressing the functionality of the expographic language. It happens that there are few works interested in this level of the research endeavour, still remaining relatively unaffordable. Talking in regards about the existence of a grammar of production persists, therefore, problematic. The notion of grammar involves putting into practice a set of rules defining a category of discourse. Yet, the exhibition designers devote, as highlighted by Veron and Levasseur (1996), in their work, a whole set of hypotheses associated, on the one hand, with the fundamental characteristics of the exhibition as media and aggregated, on the other hand, to rules allowing to build and transpose through this medium a varied discourse typology. This report leads us to perceive the conceptual approach as a complex approach The evolvement of teaching methods from traditional face-to-face teaching to computer-based learning made it important for researchers to establish frameworks for measuring the effectiveness of multimedia-based teaching methods, and appropriate criteria for the assessment of their creative outcomes. Previous research findings suggest that 'researchers still know relatively little about the most appropriate tests to administer in order to identify potential creativity or increases in creative behaviour among teachers and learners' (AlEnizi's, 2008, p. 13). Moreover, there is no consensus on the most appropriate assessment methods to test for creativity or measure its development (AlEnizi's, 2008).
Hardly any researchers focused on the development of rubrics that measure the effectiveness of multimedia-based teaching in enhancing creativity. Therefore, this research paper investigates and proposes key criteria for measuring the educator's effectiveness in employing multimedia-based learning to enhance creativity, and to also propose criteria for measuring the level of creativity in the learning outcomes of multimedia-based pedagogical approaches. The aim is mainly to broaden understanding of multimedia-based creativity assessment, making specific suggestions about the most appropriate criteria and offering a model for conceptualising and evaluating creativity in the digital classroom.
Educators' endeavors, however, need to be guided by the research that investigates the proper frameworks for evaluating creative multimedia-based pedagogical practice. This involves establishing models and rubrics that assist educators in their selection and evaluation of the educational strategies, resources and applications that they use or intend to use because 'designing creative learning environments involves not only facilitating student creativity, but also modelling creative pedagogical practice' (Cochrane & Antonczak, 2015). Accordingly, the rest of this study aims to investigate and provide guidelines for measuring digital creativity in multimedia-based art, design and media Education. The next section presents the literature review that establishes what is meant by the term 'digital creativity', and describes the main existing methods of assessing creativity, with emphasis on assessing digital creativity in art, design and digital media educational settings. The third and the fourth section outline how the research resulted in three rubrics; the first rubric offers the proposed criteria for measuring the students' effectiveness in employing multimedia-based learning methods to enhance their creativity. The second rubric proposes criteria for measuring the students' effectiveness in employing multimedia-based learning methods to enhance their creativity in various design fields. The third rubric proposes criteria for measuring the instructor's effectiveness in employing multimedia-based teaching methods in enhancing student creativity.
The rubrics drew on a survey we conducted with sixteen instructors specialised in various fields of art, design and digital media. The key questions included in the survey focused on how to assess creativity in each of the fields of graphic design, and which criteria are the most important to evaluate digital creativity. The last section presents Discussions and Conclusions in addition to future research directions.

Research problem
Despite the aforementioned technological advances, we have noticed (through investigating the views and practices of teachers at different universities in Bahrain) a lack of research knowledge about multimedia-based pedagogical practices that may nurture creativity in classrooms across Bahrain. Moreover, we could not find any assessment frameworks for multimedia-based creativity in the art, design or digital media classroom.

Research objectives
The primary objectives of this study are: 1. To identify key indicators of creativity as expressed among art, design and digital media students who use digital and social media applications to enhance their creativity. 2. To propose a framework that can be used to measure and assess educators' success in employing strategies that use multimedia-based learning to harness their students' creativity. 3. To equip art, design and digital media educators with a better understanding of how to assess their students' digital creativity and evaluate the outcomes of their multimedia-based learning. 4. To guide art, design and digital media educators into the best practices to implement and the most effective digital applications to employ in order to enhance the creative potential of their students.

Research questions
1. What evaluation rubric, framework or criteria can be effective in measuring and assessing a teacher's success in employing strategies that use multimedia-based learning to harness students' creativity and innovation? 2. What evaluation rubric or criteria can be effective in measuring and assessing an art, design or media student's creativity and innovation while applying strategies that use multimedia-based learning?

Research methodology
This pilot study attempts to find answers to the above questions through investigating the views and practices of teachers at various universities in Bahrain through surveying them. A semi-structured online questionnaire was distributed among a purposeful sample of 16 graphic design, fine arts and digital media instructors. We approached our enquiry as an extended pilot study as there are no largescale datasets and samples because the number of instructors working in the art, design and digital media fields in Bahrain is very limited.

Theoretical framework
The concept of creativity has been evolving through the centuries, and it has always been accompanied by continuous attempts to create appropriate methods of creativity assessment. The literature presents many reviews and discussions on the numerous instruments, procedures and methods that have been used to assess creativity. Several creativity assessment tools have been collected and described in a study by Barbot, Besancon & Lubart. (2011, p. 125): Petrosko's early review already mentioned hundreds of creativity tests. Haensly and Torrance identified over 200 instruments focusing only on verbal, figural, or 'general' creative abilities. Fishkin and Johnson's review compared strengths and weaknesses of numerous methods used to assess children's creativity. Later, a useful monograph on creativity assessment for educators, published by the National Centre on the Gifted and Talented reviewed thoroughly over 70 creative assessment techniques, selected from a list of over a hundred. (p. 125) In addition, the Wallach-Kogan Creativity Test is also a well-known test that measures divergent thinking as a proxy of creativity (Tan, Lee, Ponnusamy, Koh & Tan, 2016, p. 7). Similarly, Salamon (2008, p.77) thinks that it is important to incorporate divergent and convergent thinking into assessment frameworks because 'the processes of art and design often require mastering tasks and techniques together with creative and conceptual skills'.
Despite the diversity of the above-mentioned creativity assessment frameworks, a crucial issue for both creativity research and practical applications is to assess digital creativity, especially in the digital era where new fields and disciplines are growing. Meanwhile, existing digital creativity assessment tools are limited and lack theoretical frameworks for the assessment of students' digital creativity and the outcomes of their multimedia-based learning. In addition, there is a lack of frameworks for assessing educators' success in employing strategies that use multimedia-based learning to harness and foster their students' creativity.
The evolution of technology and emergence of various digital applications necessitate rethinking the concept of creativity and how to assess it. Indeed, the affordances of technologies may have a strong impact on creative processes and achievements as indicated by Loveless (2002, p. 64) who believes the following: …digital technologies can be tools which afford learners the potential to extend or enhance their abilities, allow users to create novel ways of dealing with tasks which might then change the nature of the activity itself, or provide limitations and structure which influence the nature and boundaries of the activity (p. 64).
The role of social media should be highlighted in the context of employing the technology to harness the students' creativity. Seo (2013)

stated
Educators recognize the power of social media to transform learning, and they are now integrating these online tools in their instructions. The features of social media complement the constructive philosophy of teaching and learning that allows learners to create, co-create and share knowledge with global audiences beyond classroom walls (p. 1).
One of the concepts that have been discussed and introduced in recent literature is the domainspecificity of creativity, which indicates that creativity is domain-relevant and is based on 'multivariate relationships'. This concept has been discussed by (Barbot et al., 2011) who reported Furthermore, it is accepted that the ability to produce creative work is partly domain-specific because the nature of creativity varies with the field, and could even vary as a function of the task within a domain [24] or the constraints within the task [25]. Consequently, the concept of creativity has evolved toward a 'multivariate' framework, in which creativity is viewed as a multifaceted and domain-specific ability, which can be developed and can be assessed properly by assessment tools tapping into the multidimensionality of this ability [1]." (p. 59) Although there are limited studies that approach assessment in terms of the relationship of creativity with technology, some studies attempted to bridge this gap by proposing definitions and strategies of digital creativity. The project of Barajas and Frossard (2017) defined digital creativity (based on NACCCE, 1999;Cremin, Clack & Craft, 2012) as 'purposive imaginative activity, mediated by digital technologies, generating outcomes that are original and valuable in relation to the learner' (2017, p. 8). Sefton-Green and Brown (2014, p. 7) defined it as: 'working creatively within a digital medium'. Nie and Liu (2016, 453) discussed the development of multimedia Computer-Aided Instruction courseware, and they provided a guide for evaluating multimedia courseware. Cropley (2003 as cited in Salamon, 2008, p. 74) suggested that 'focusing on properties closely related to the real-life practice of the discipline in question can prevent grade-chasing, and recommends performance-based assessment such as portfolios'. Salamon (2008) described an assessment framework for creativity called Research, Analysis, Development, Solution and Evaluation, which is used in computer animation and is designed to be used as a general framework for assessing a person's creativity in art, media and design. According to Salamon, this framework is linear and does not reflect the circular approach to creativity, which involves repeating some stages of the creative process until a solution is found. Therefore, Salamon proposed the 'Creative Spiral' framework which reflects a rounded approach, that considers and allows for repetition, review or refinement. Chuang and Huang (2015) developed a creativity assessment scale for digital game story design in an attempt to help teachers evaluate creativity in various aspects of digital game story design. Feeman, Marcketti & Karpova. (2017) applied the Consensual Assessment Technique (CAT), which was created by Amabile (1996) to the evaluation of fashion mood boards by faculty and by students. The purpose of their investigation was to develop a digital consensual assessment instrument that measures creativity as they suggested the difficulties in objectively evaluating the creativity in fashion design. These difficulties are similar to the difficulties and objectives that led to this research, which aims to propose effective criteria for measuring digital creativity in art and design. According to Jeffries (2017), the CAT is designed to assess creativity, through the consensual assessments of domain experts or judges. Jeffries, however, suggested that the use of CAT as a measure of creativity within graphic design research is relatively minimal. He was able to identify only24 papers that referred to both graphic design and the CAT, and only two of these papers were directly related to graphic design (Dineen & Niu, 2008;Silvia et al., 2008). Jeffries (2015) analysed and described the approaches proposed in these two studies in their attempt to apply CAT to graphic design: Silvia et al. (2008, p. 70) undertook a study to validate a new method of scoring divergent thinking tasks. Part of the study had participants who majored in arts subjects (accounting for 9% of participants in the study), of which some participants majored in graphic design. Whilst the new technique built the case for the validity of subjective rating by citing CAT studies, the method used was not the CAT protocol. Indeed, they acknowledged the importance of expert domain judges for 'studies of real creative product', but argued that this need not be the case for divergent thinking tasks assessment. In contrast, Dineen and Niu's work (2008, p. 17) utilised the CAT method, and involved participants in their second year of graphic design at a Chinese art and design higher education institution. The study explored the respective merits of UK creative pedagogy relative to traditional Chinese pedagogy. However, it is arguable, how closely aligned to graphic design the final outputs were relative to illustration design.
Jeffries also referred to research by Wojtczuk (2014), which also attempted to apply the CAT to graphic design and highlighted a number of findings on the influence of judges' background on design creativity assessment. One of the most important findings, according to Jeffries (2015) was …the low level of consensus achieved by designers' rating of graphic design creativity. Therefore, as yet, there is little precedent established for a task to reliably measure graphic design creativity using the CAT. This is particularly relevant in the light of debates surrounding the domain specificity/generality of creativity, and the role task selection plays in creativity assessment (Byrne, 2011, p. 17) As a result, Jeffries suggested that a number of factors, such as diversity of artwork, technical preference, task selection and sampling of judges must be considered while assessing creativity in graphic design. He conducted a pilot study, which revealed that judges' preferences for technical execution differed and caused variations in their judgments of graphic design creativity, and that their preference for technical quality seemed to be higher or lower relative to the medium of the artwork. This led Jeffries to recommend the inclusion of a 'caveat on technical execution' when applying CAT to graphic design creativity assessment (Jeffries, 2015, p. 26).
The above-mentioned studies lack a critical perspective on how technology-assisted creativity can be assessed, especially with technical execution as a dimension and a significant factor. Therefore, this study will focus on providing digital creativity assessment frameworks and rubrics in an attempt to help the educators to evaluate and assess students' digital creativity and the effectiveness of multimedia-based teaching approaches in art, design and digital media educational contexts.

Assessing the student's creativity in implementing multimedia-based learning and practice
Creativity is complex because of its multiple manifestations and dimensions. Hence, it is difficult to measure creativity. Starko (2014, p. 53), for instance, stated that 'Creativity cannot be seen with the naked eye but can be evaluated by observing an individual.' This suggests that creativity can be challenging to understand and measure because it is such a complex phenomenon and can be expressed in many ways. AlEnizi's (2008) study also referred to the importance of observation in assessing creativity and suggested that students' creative behaviour can be assessed by observing if they are questioning and challenging, making connections and seeing relationships, predicting or 'possibility thinking', being open-minded and exploring ideas and reflecting critically on ideas and actions. Testing creativity is considered to be a contentious issue.
Many creativity tests have four main components: divergent thinking (for example Guilford's Alternative Uses Task and Torrance Test of Creative Thinking (TTCT) (1974); convergent thinking (Mednick), artistic assessments (Barron-Welsh Art Scale) and self-assessments (Khatena-Torrance Creative Perception Inventory). Probably, the most widely used test is the TTCT (Torrance, 1979), which includes measurement of thinking creatively with pictures, figural testing and verbal testing. The main criteria in TTCT are Fluency, originality, flexibility and elaboration characteristic (Kim, 2006;Torrance, 1979). On the other hand, AlEnizi (2008) suggests that the main variables that are effective in measuring and testing creativity can be categorised as Cognitive Variables (Intelligence, knowledge, technical skills and thinking styles), Environmental Variables (politico-religious factors, cultural factors, socioeconomic factors and educational factors) and Personality Variables (motivation, extraversion/introversion, non-conformity, paradoxical traits and interests). Table 1 shows the criteria that are proposed in this research based on the above-mentioned literature to measure and evaluate student's creativity in art, design, and digital media courses. The above-proposed criteria and their significance, relevance and weight in measuring the students' effectiveness in employing multimedia-based learning methods in enhancing their creativity were determined through surveying a purposeful sample of 16 design, digital media and fine arts instructors from different universities in Bahrain. The survey consisted of a number of matrix questions which required the instructors to assign a score (from 1 to 5) to each criterion based on its importance in measuring creativity. The responses were used to calculate the weighted average score for each criterion. Based on this, each criterion was then assigned a percentage weight based on its weighted average score (the criteria were assigned percentage weights based on how their weighted average score related to the other criteria using the following formula: Percentage weight for each criterion = weighted average score for the criterion / the sum of the weighted average scores for all the criteria × 100 Some rounding was applied in order to have the percentage weights total up to 100%. Because of the small sample size and data set and because there are only few graphic design, fine arts and digital instructors in Bahrain, the findings can only be considered indicative and the survey is only a pilot survey. This also applies to the rest of the criteria proposed in this section.
Elaboration was considered by the respondents to be the most important criterion for measuring the student's creativity in implementing multimedia-based learning and practice, followed by originality and the rest of the criteria in Figure 1.

Figure 1. Respondents' rankings of the criteria for measuring the students' effectiveness in employing multimedia-based learning to enhance their creativity
The respondents also assigned a score to each of the criteria indicated in Table 2 according to the different listed fields of design. These proposed criteria and their weights may help educators who teach courses related to the listed fields in measuring the students' effectiveness in employing multimedia-based learning methods to enhance their creativity, and in determining the most important criteria for measuring creativity in each of the fields. The findings, for instance, indicated that flexibility is the most important criterion for measuring creativity in Logo and Corporate Identity Design; Lateral thinking and risk-taking are the most important criteria for Commercial Ad Design, and risk-taking is also the most important in Photography; Originality is the most important in Animation and Children Storybook Design; Elaboration is the most important in Infographics and Video

Measuring the educator's effectiveness in employing multimedia-based learning to enhance creativity
Morris (2014, p. 5) describes creative teaching as 'teachers using imaginative approaches to make learning more interesting, engaging, exciting and effective'. On the other hand, researchers at Cornell Centre for Learning and Teaching (1997;Pratt, 1997;Young & Shaw, 1999;Tigelaar et al., 2004 as cited in Office of Institutional Analysis & Planning, 2007) emphasize the following: …The competencies that have been consistently identified as universal and integral to excellent teaching can be classified into three broad groups: content expertise, instructional design skill (e.g., course design and planning, development of assignments and exams used to evaluate student learning) and instructional delivery skills, such as presentation skills and interactions with students both inside and outside of the classroom (p. 3).
In addition, Feldman's study (1976;cited in Young & Shaw, 1999, p. 674) noted that 'stimulation of interest and clarity of presentation were the two most highly related dimensions of good teaching' (1976; cited in Young & Shaw, 1999, p. 674). Interpersonal traits, such as helpfulness, friendliness and open-mindedness, are also considered by students to be important traits of effective teachers.
El Mhouti, Nasseh & Erradi (2013) proposed an instrument for the evaluation of the quality of digital learning resources. Their study presented evaluation criteria that were based on 'four dimensions of digital learning resources quality: academic quality, pedagogical quality, didactic quality and technical quality.' (El Mhouti et. al., 2013, p. 29). The academic quality variable entails information reliability and information relevance. The pedagogical quality variable is based on pedagogical formulation, pedagogical construction, pedagogical strategies and assessment methods. The didactic quality is based on learning activities and content, while technical quality is based on design, browsing, and technological ingenuity.
The above-mentioned evaluation models and criteria, however, mainly focus on teaching and resources quality and do not focus on evaluating the educator's creativity in implementing innovative pedagogical strategies and techniques that are particular for teaching multimedia-based subjects and courses. Therefore, in addition to adopting some of the criteria and variables that were proposed by the above-mentioned researchers, more relevant criteria are herein suggested for the study on measuring the effectiveness of multimedia-based teaching methods in enhancing creativity (Table 3). These criteria and their weights were proposed by the surveyed design, digital media and fine arts instructors. The instructor's ability to develop and build enhanced digital academic content that fosters creativity, and the optimal utilisation of new technological skills, tools, and resources in education to enhance the transfer of knowledge and foster creative learning outcomes.

2 3 4 5 Weight Efficiency
The instructor's optimal use of available resources to achieve a certain level of outputs at the shortest duration, and the ability to accomplish the intended learning outcomes with less time and effort.
No optimal use of available resources and time to achieve a goal Minimal optimal use of available resources and time to achieve a goal Moderate optimal use of available resources and time to achieve a goal Good optimal use of available resources and time to achieve a goal Excellent and frequent optimal use of available resources and time to achieve a goal 5.0%

Risk-Taking
The instructor's willingness to take risky actions in order to achieve specific goals.
Has no willingness to take risk Has little willingness to take risk Has moderate willingness to take risk Has strong willingness to take risk Is always very strongly willing to take risk 4.0%

Divergent Thinking
The instructor's ability to produce the largest number of responses to one question, helping to come up with a new and authentic idea.
Has no ability to produce more than one response to one question Has little ability to produce more than one response to one question Has moderate ability to produce more than one response to one question Has good ability to produce many responses to one question Has excellent and remarkable ability to produce many responses to one question 5.0% The digital skills and competence of the instructor was considered by the respondents to be the most important criterion for measuring the educator's effectiveness in employing multimedia-based learning to enhance students' creativity, followed by technological ingenuity, problem-solving, efficiency, and the other criteria indicated in Figure 2.
Although these proposed criteria and their weights are preliminary and only indicative, they may potentially help quality assurance bodies and universities in their attempts to evaluate the educator's effectiveness in employing multimedia-based learning methods to enhance their students' creativity.

Conclusion and future directions
The growing use of technology in art and design education is changing the concept of creativity and the way it is assessed. For this reason, exploring and developing assessment methods and tools for art and design students, who implement the latest technologies and digital trends in their academic work, is needed in the present educational landscape. However, existing creativity assessment frameworks are limited, especially for their lack of up-to date criteria for assessing digitally-enhanced creativity through the use of social and digital media in the art or design classroom. The research sought to extend existing creativity assessments and rubrics into the realm of digital and social media, pushing back on current conceptions of creativity and its relationship to education and technology. It proposed a framework for assessing students' creativity in educational contexts that involve the utilisation of social and digital media applications, and another framework for assessing the instructors' creativity, which may also be enhanced with the use of these applications and technologies.