A Generic Tool for Generating and Assessing Problems Automatically using Spreadsheets

A myriad of higher education degrees including science, engineering, business and economics could integrate activities using spreadsheet into the teaching-learning process. In the present article, we aim to introduce and describe a new tool that generates, individually assigns, corrects, provides feed-back and grades exercises and activities using spreadsheets all completely automatically. The system’s versatility means it can be applied in a range of areas of knowledge and subjects including mathematics, statistics, physics, accounting and economics, among others. It is usable for both formative and summative assessment based on students’ answers, providing them with the corresponding feed-back and/or the resultant grades. The proposed solution has been developed and implemented at the University of Girona (Spain) and forms part a wider family of tools that aim to contribute to the continuous improvement of higher education.


Introduction
Most university-employed teaching and research staff would agree that one thing that remains unchanged in education is constant transformation. Radical or incremental, it affects both content and format, continuously challenging educators to adapt to new generations of students and novel contexts. The reality of twenty-first century teaching practices is that it is not a matter of 'if' but of 'how' technology moderates the teaching-learning process. Assuredly there remain some sceptics, but most adopt and adapt to Information and Communication Technologies (ICT) irrespective of the subject or the educational level targeted. The proliferation of devices and connectivity and how they invade all areas of life makes the field of education irresistible to this propagation and concepts like active learning, problem-based teaching and higherorder cognitive thinking take on new meaning in the digital age.
Current higher education practice means that most universities dispose of global virtual learning environments based on some type of Learning Management System (LMS) that facilitates course management, provide access to study materials and assignments and establishes communication channels, among other uses. An important feature of this environment is Computer Based Assessment, a system from where effective assessment, be it summative and/or formative, is carried out. Too often, this assessment consists of typical questionnaires and quizzes with well-delimited answer options, where the novelty is simply the digitalization of a conventional/traditional assessment. For higher-order cognitive thinking involving analysis, evaluation and synthesis, assessing the degree to which a student has acquired a certain competence requires more sophisticated solutions and rather more complex activities and assessment procedures in terms of volume, depth and accuracy. Multiple ways of solving and several correct answer scenarios are often part of this process. Manual correction is time-consuming and there is a high risk of error on the part of the instructor, especially where large numbers of students are involved and if the course requirements mean multiple assessments per student. Typical examples are correcting a computer program, designing a circuit, a mathematical, statistical or physics problem, a business balance sheet and a diagram. A generic e-learning tool able to correct and assess these types of activities would be ideal within this context.
With this objective in mind, the University of Girona (Spain) has developed the web-based e-learning platform known as ACME, an acronym meaning Continuous Evaluation and Educational Improvement [1]. The ACME platform is an LMS designed to generate, assign and correct complex exercises automatically. When an activity is set up by a teacher, some exercises are selected from a repository and the system assigns individualized versions of them to the student workbook. The student receives the exercise, resolves the problem and sends the answer to the system, which then proceeds to correct it and provide feed-back. The platform is configured in specific modules for correcting exercises corresponding to different knowledge fields, such as mathematics [2], programming [3], database design [4] and diagrams [5], to mention just a few. Generating and correcting exercises in these subjects first requires that different types of exercises are set up, an essential step due to the different user interfaces and/or correcting tools needed. For example, for mathematics exercises a textbox is used to enter the solution (a mathematical expression) and symbolic computation software is used to correct it, and for inorganic chemistry and database subjects, a specific interface is used to enter the solution and a specific correction code is applied. The platform has a web environment for exercise management, which guides the user through the process.
Repeated requests from teachers of very diverse disciplines, including physics and business and finance, who all had in common student activities using spreadsheets, obviated the need to develop a module building on spreadsheet exercises. A series of tools have thus been developed that aim to correct and assess spreadsheets automatically, none of which, however, contain all the features available in the tool described in this paper.
The aim of the present paper is to describe the process and resulting tool for individually generating and assigning activities to be carried out on Excel spreadsheets, correcting, providing feed-back, assessing and grading these activities all automatically. This spreadsheet solution has been developed gradually: previous versions have been implemented in a variety of subjects, always functioning within the framework of the ACME platform. The new module developed is known as http://www.i-jet.org ACME_Spreadsheet 1 . Its usefulness lies in how it addresses teaching, self-regulated learning and tasks for assessment.
Regarding teaching, once the instructor has explained the theoretical and conceptual aspects of a subject, students use ACME_Spreadsheet for solving individually configured and assigned exercises with the instructor present to provide assistance and clarify pending issues. In this way, instructors can fairly assess what aspects have not been understood. ACME_Spreadsheet can also be used by students to practice and self-learn subject content. In this case, the teacher assigns problems to be solved by the student and the ACME_Spreadsheet returns opportune feed-back. Last, in terms of assessment, it provides the student with the numerical grade corresponding to the submitted solution.
In Section 2, the literature in the field of systems that aim to correct spreadsheets automatically is reviewed. The system requirements are highlighted in Section 3, how the system functions in terms of generating and correcting exercises is described from the perspective of both teachers and students in Section 4 and the architecture of the proposed model is presented in Section 5. Last, the results are discussed and conclusions drawn with the implications for the main stakeholders involved in Section 6.

Literature review
Approaching the core theme of this article, it is pertinent to highlight the importance of using spreadsheets in education, including the main benefits and drawbacks, as stated by the literature. Previous published experience of spreadsheet education in business subjects is also reviewed to determine the value proposition of the authors' contribution.
Spreadsheet is one of the generic software packages most used in companies: Thus the importance and relevance of university students being skilled in its design [6] and use. Moreover, spreadsheets have long been used in education to teach a variety of subjects and there is a large body of scientific literature, including the publications authored by [7], [8], [9], [10], [11] and [12] that attest to how they are used to facilitate learning in mathematics, statistics, economics and physical sciences. Given their importance and potential, several specific solutions already exist for correcting student responses using spreadsheets. Thus, while the idea of using spreadsheets in education is not new, ACME-Spreadsheet differs from other approaches, mainly in the technical setup of the system, the value chain covered by the solution -from exercise design to feed-back and grading -and the low level of computing skills needed for the teacher to set up the subject. The inherent definition that spreadsheets are a group of applications that allow for tabulated information management and calculations facilitates their natural integration into engineering education. This, complemented by today's data-driven world and the data management and analysis skills demanded by most employers [12], makes it mandatory for higher education institutions to address this subject.
As already mentioned, using spreadsheets in engineering education is aligned with a specific goal: to be as employable as possible. In this sense, the cross-cutting ability to use spreadsheets is often a required skill irrespective of the job position, geographical context and candidate's academic background, the latter being where and when this skill is supposedly acquired. In the field of engineering and other technical studies, the benefits of using spreadsheets are multiple. The inspiring work of [7] reviews the historical development of spreadsheets, tracing their origins to the demands of finance and accounting and later spreading into engineering education to reach today's status quo of omnipresence and multiple differentiated applications adapted to the needs of the different engineering sub-fields as the result of a series of characteristics described by [7:893] as follows: utilising spreadsheets in engineering education not only makes completing tasks more efficient, but also often achieves a higher degree of accuracy than do humans.
Both academic practice and research in this field has intensified over the last decade or so. Solutions have been sophisticated and have spread throughout the academic community. Practice shows their presence during the entire higher education lifespan starting with spreadsheet literacy for first-year undergraduates to later mastery of spreadsheet skills, thus generating future professional competence. Within this context, most educators take on an active role in developing these skills among their learner audience, even though the aim of many may not be to convert the experience into a publishable article. Nonetheless, solutions published in the international academic literature do exist and the ones that are most relevant to this article are highlighted below, aligned with the description of the solution as provided by its author: • [13]: Innovative approach for automated grading, especially for complex assignments. • [14]: Innovative approach based on Excel files to develop individualized problemsolving tasks for assignments. It is marked automatically and provides tailored feed-back on the problem-solving process and outcome. • [15]: Interactive spreadsheets with immediate feedback regarding the accuracy of solutions. • [16]: Individualised spreadsheet assignment with instant feedback on progress, either textual or numerical (i.e. a mark) or both. • [17]: Using Excel macros to create assignment questions where numbers or data sets used in the questions are randomly generated and the students' answers graded automatically. • [18]: Spreadsheet-based tool to generate a personalised assignment.
• [19]: An individualised, interactive, formative assessment. Students can, at their discretion and without instructor intervention, immediately receive feedback and assistance related to their individualised question. • [20]: The design and successful delivery of an effective application of spreadsheets and the practice of modelling and analysing basic business concerns. • [21]: E-learning system that allows for manual and automatically generated exercises and automatically marks the students' solutions without the intervention of a lecturer.
• [24]: Development of a programme that ensures independent learning and practice through a technique of 'individualised accounting questions.' • [25]: Development of an automatic grading system to assess spreadsheet and database skills. • [26]: Spreadsheet activities using VBA programming to automatically generate feedback, calculate grades and conditionally progress. • [27]: Automated grading system for Excel based modelling assignments. Table 1 is a tabulated way to characterise and compare existing solutions. The columns represent the criteria to be considered when analysing existing solutions, except for the first column which is the number corresponding to the paper/authors in the reference list. Columns take a value of " when the feature applies and # otherwise.
In summary, most of these solutions have probably been designed and are being used for very specific tasks and so they do not allow for an exhaustive coverage of subjects. Others have the handicap of needing specific software. Most of the compared tools require programming knowledge to adequately prepare exercises. The majority correct a single sheet, while in practice complex solutions require a variety of sheets organised in different workbooks. Since none of the existing solutions satisfied our needs, we opted for designing and developing a new tool that could be integrated into the already existing ACME platform. Not all the features considered are essential, but they are all the possible requirements the solution should address.

System requirements
As already stated, the system described in the present contribution is part of a wider family of tools that make up the ACME platform and share a common target, namely the continuous improvement of the teaching/learning process.
The requirements specifically related to the ACME_Spreadsheets are described below.
• Generic: the module is designed to be generic in the sense that it can manage any activity through which a student and/or a teacher can capture the results obtained in an Excel spreadsheet and/or workbook. • Simultaneous multi-sheet correction: as it is usual to work with multiple spreadsheets of the same workbook, the requirement set was that the solution should be able to simultaneously read and correct multiple sheets of the same workbook, an important consequence of which is that complex exercises can be solved and the results obtained on one sheet can be an input for another. • Open values: To cover a wide range of activities, the system must support different scenarios where data entry could be: i) input data originating from an exercise, ii) experimental values obtained in a lab or a bibliographic search, or iii) a formula to obtain the input data. • Individualised activities for students: To create a varied spectrum of exercises and possibly contribute to reducing plagiarism, the system requires that activities generated by the platform are different, the visible result of which is that each student can have their own individualized activity • Maximum freedom for the student: To empower students to proceed with their own solutions, the spreadsheet module must allow for intermediary calculations, annotations, etc. The system will not search to match the formula entered by the student and the one set as correct by the teacher, but will check for result coincidence. • Transparency: Students must be informed at all times and the system must provide them with details about what content, or which cell, in the spreadsheet is the current object of correction and/or assessment. • Communication: Where the activity is set as a formative one, the system must provide valuable feed-back corresponding to the erroneous cell. Where the activity is a summative one, the solution must compute and communicate a mark according to the correctness of the solution. • Teacher-friendly: Creating a new activity must be as easy as possible for the teacher. Being a cross-cutting solution, this requirement avoids the need for programming skills, so teachers with basic user-level Excel skills can manage the entire process from activity planning to final assessment. • Integrated: The educator must be able to manage the module on a platform that supports regular teaching activities, which translates into a system requirement of total integration into an LMS such as Moodle, the LMS used in our university.
Moodle is a free, open-source learning management system developed on pedagogical principles and used for blended learning, distance education and other elearning projects with the aim of integrating on-line content and activities. Full in-tegration into Moodle has the advantage of being a one-stop-shop for students and teachers alike, optimising work tasks for both collectives.
With the ACME platform already available, the easiest way to proceed with the newly developed module was for students to visualise the ACME_Spreadsheet activity through the web browser and then download the Excel workbook for completion according to the specifications provided in the exercise statement. The student's answers are then uploaded into the solution file on the ACME platform, where they are assessed with feed-back and marks are assigned as per the configuration. This procedure ensures that ACME and Moodle are both used to their full potential, since they already have a student subject management system that solves all these issues with the inherent advantages. Another option considered but later abandoned was to develop each activity individually using an Excel sheet with VBA macros to provide feedback and grading. This would have meant programming specific macros for each activity, thus over-complicating the design and construction of each activity, as well as losing the centralised management for students. A graphical representation of the system architecture is depicted in Figure 1.

System use
This section aims to describe how ACME works. The spreadsheet activity in a classroom setting is described from a double perspective, placing the emphasis on the two main stakeholders involved, namely the teacher (Section 4.1) and the student (Section 4.2).

Teacher perspective
The spreadsheet activity in teacher mode is described step-by-step in the illustration in Figure 2.

Fig. 2. A process view of teacher mode
The teacher must first create some exercises, for which ACME has a specific activity editor. Figure 3 illustrates how to edit a problem statement in ACME_Spreadsheet. A typical problem involving calculating the breakeven point for multiple products (A, B and C) is shown and will be used throughout the paper to illustrate how the system works. This editor is the element responsible for facilitating the task of generating individualised exercises automatically.
Once the teacher has created a spreadsheet activity, it is automatically saved in the open repository of ACME and is available for any teacher to use, accessed either through the Moodle platform or directly from ACME. Further configuration elements relate to selecting whether the activity is a test for self-assessment and trials, or if it is an assessment item/exam, in either case setting the timing of the planned activity. The system, completely automatically and randomly, takes the different problem statements and substitutes the parameters marked with values from a parameters list. Once the activity is assigned to the student's exercise book it can be visualised and the student Excel workbook is ready to download. Each student receives a different exercise from their peers, either with the same problem statement but different parameters, or with a different problem statement and different parameters. Students proceed with the exercises and teachers have a series of tools to simultaneously monitor and track their activity. At any time, the following details are available: exercises solved, errors committed, statistics, timing, etc. The student can communicate with the teacher by means of the ACME platform to resolve any questions.  Figure 4 shows the process followed by the student as a system user:

Fig. 4. Student process view
To proceed in the ACME work environment, the student must log in and access the assigned exercise book. The activity described in this article must be solved using spreadsheets after visualising and selecting the exercise needing to be completed. Once selected, the student can visualise the problem statement and the Excel workbook is ready for download, as shown in Figure 5.
Once the activity is assigned to the student's workbook, the problem statement can be visualized in the ACME platform, the student can download the workbook and proceed to complete the activity with the required sections/questions. On completion, the student selects the file (see the Browse button in Figure 5) and sends it to the system to be corrected, receiving immediate corresponding feed-back. If it is a summative assessment, the student will also receive the equivalent mark, and if it is a formative assessment, the student can resend solutions until the final correct one is found or they have exhausted the number of trials set by the teacher when configuring the exercise. The architecture of a spreadsheet exercise This section aims to describe the architecture of the ACME_Spreadsheet, distinguishing between an ACME baseline exercise (Section 5.1) and a specific ACME_spreadsheet exercise (Section 5.2 to Section 5.5).

5.1
An ACME baseline exercise structure The structure of a baseline exercise on the ACME platform consists of a series of elements described below.
Heading: Containing the metadata of the exercise, serving to classify it. Problem statement or the statements that define the exercise: Each exercise can have one or more different formulations that may even refer to the same exercise without affecting the way to solve it. A series of #Px tags (where x = 1,2,3 ... n) appear in the problem statement indicating the point where the parameter values are to be substituted (see Figure 3).
Parameters' values: They can be from a predetermined list of values or they can be calculated. Values can be numbers, text, a mathematical expression or formula, data files, code files or images. When an exercise is assigned, each #Px marking is substituted by a parameter value.
Correction code: This code varies according to the type of exercise. It can be a code written by the teacher, for example a mathematics type code, or it can be a standard code known only to the teacher who has prepared the exercise, in which case the teacher should only indicate the typology (for example, spreadsheet type). This code captures the solution submitted by the student and the values of the corresponding parameters, verifies if the solution is correct or not and provides relevant feedback.
Attachments: These can have different functions and can take the form of graphs or images that can be shown in the problem statement or a data file the student can download.
The new system typology described in this paper aims to correct exercises where the solution provided by students is a spreadsheet. The procedure to correct this type of exercise follows a matching approach and consists of comparing the workbook sent by the student with the solution workbook created by the teacher. Consequently, the teacher must create a solution for the exercise together with a template to indicate where the student must place the results to be corrected. The spreadsheet exercises follow the generic structure of an ACME exercise and include a heading, statements, parameter values and 4 attachments, which are the 4 Excel workbooks briefly described below for which the same example as the one described in Section 4.1 is used.

Student Excel Workbook
The Student Excel workbook is an empty template that is downloaded by the student from the exercise book. It can contain one or more spreadsheets for completion and is the only workbook that is accessible to the student. Figure 6 shows how the cells appear to the user.
• Cells with labels: These cells inform about what is represented in each row/column. • Yellow cells: These cells (for example, A3:C7, B12:B14, F12:F14) are to be completed by students with the parameters corresponding to the exercise. There is the option of not enabling yellow cells, in which case the student must create their own information, entering a value to be able to proceed with the calculations required to find the solution.  Other spreadsheets can also be used to organise any intermediary solutions needed to find the final one.

Solution Excel Workbook
The solution Excel Workbook contains the correct solution for the exercise, which can be formulated in one or more spreadsheets. Cells can contain a series of data (see Figure 7). The system will substitute these cells with the parameter values corresponding to each student when the exercise is corrected. The teacher must mark them with #Px (x = 1, 2, 3 ... n) to indicate where the parameters that need to be substituted are. These cells will be referenced in formulas to calculate the solution. • Pink cells: These cells contain experimental data provided by the student. The teacher should mark them as 'EXP' in the solution Excel workbook. The system will substitute these cells with the inputs provided by the student when the workbook is corrected. Optionally, the system can check if the values provided by the student fall into a predefined range of values defined by the teacher. • Green cells: The automatic correction system compares these cells with the corresponding ones in the student spreadsheet and feed-back will be generated by them if these values do not match. The teacher must provide the required formulas to obtain the results based on the parameter values and the experimental data provided by the student. These cells can contain formulas (see for example F17 and F18, Figure 7) or they can capture calculations in other spreadsheets (for example B21:E24 in Figure 7). The main advantage of using a separate spreadsheet is that the teacher can thereby create an extensive exercise with several questions where all the calculations are in this spreadsheet. Based on this, several questions can be derived to form a subset of the different questions and calculations. When assigning the exercises to students, the teacher selects some of the derivative problems and the system assigns them to each student. The varied questions, problem statements and parameters make it possible for each student to have an individual exercise with a distinct appearance and content from the others, helping to reduce or eliminate plagiarism.
• Blank/White cells: These cells can be used by the teacher to generate intermediary calculations if applicable or they can be left blank. Other spreadsheets can also be used to organise data and calculations where necessary.

Feedback Excel Workbook
The feedback Excel workbook showed partially in Figure 8 is a non-compulsory spreadsheet. Each cell contains the error message that is sent to the student when there is an incorrect answer. When this spreadsheet is non-existent, the system sends the alert message 'incorrect cell' indicating the column and row, for example B17. In a formative assessment this spreadsheet serves to indicate what is expected in the cell with the corresponding explicit message needed to guide the student. The position of these cells must match with that of the cells in the spreadsheet created by the teacher, which will be further compared with the one provided by the student.

Mark Excel Workbook
The mark Excel workbook captured in Figure 9 is another optional file. There is a value in each cell when the solution is generated (for example, B21:E24). When using the activity with a summative assessment in mind, the system subsequently calculates the corresponding grade. For formative assessment purposes, the system can test the coincidence of the student generated solution with the correct solution without generating a mark ('READ', for example B14, F14). A third option is that a cell is neither read nor scored (see B13). As in the previous case (see Feedback workbook in Figure  8), the position of these cells must match the position of those in the teacher provided solution workbook.
Once the teacher has prepared the problem statement, the list of parameters and the 4 Excel workbooks the baseline exercise is saved in the data base. The next step is to assign the problem statement to a student exercise book.

Discussion and conclusions
While plenty of solutions with some functionalities of spreadsheet activities exist, the one described in the present article has a multi-stakeholder integrative approach that stands out due to the following characteristics and advantages.
Individualised exercises: The system can generate an individual exercise for each student. An exercise can include various problem statements and parameters, resulting in a unique combination that ultimately appears as an individual exercise for each user.
Flexibility and freedom: The correction is performed in a series of specific cells highlighted in the student spreadsheet. In all other areas of the spreadsheet the student can proceed with the intermediary calculations considered necessary to generate the final solution. Students are thus empowered to proceed with their own solution and there are no restrictions, for example, on writing a specific, exact formula in a certain cell.
Complex: The system allows multiple spreadsheets in a workbook to be corrected. As the results are often organised in different spreadsheets, with some output data generated by a calculation in a spreadsheet serving as input in another spreadsheet, this feature allows for planning and carrying out a series of complex activities.
User-friendly: Creating a new exercise is easy and requires no programming skills. Basic, user-level spreadsheet knowledge is sufficient to plan and generate an activity once the functioning and structure of this type of activity is known.
Multi-disciplinary: The system is prepared to respond to any spreadsheet-based activity irrespective of the knowledge field. Currently, activities are in the field of mathematics, statistics, accounts, physics, mechanics and business, among others.
Integrated: All activities on the ACME platform are integrated into the generic Moodle platform, meaning that all activities can be created and managed from there.
Formative assessment: Activities can be used for a formative assessment since the system generates feed-back for students. Each cell configured to be corrected provides feed-back that is valuable to guide the student towards the correct solution.
Summative assessment: Activities can also be used for a summative assessment. When configured as such the system returns a numeric mark between 0 and 10 according to the value that the teacher has previously marked in each cell.
Going beyond these technical characteristics the solution is well perceived and welcomed by students. In publication [28], the authors discuss the spreadsheet activity from a rather pedagogical perspective, contributing with data related to grades and subjective perception of a previous version of the system.
Complementary to these positive aspects, like all innovative solutions our proposal has a series of technical limitations. First, the system does not allow dynamic table equivalent to be used because data in the spreadsheet can be tabulated in table format, but the dynamic table function is not usable. Second, files other than Microsoft Excel files generated by other spreadsheet editors are not supported by the system, but since Microsoft Excel is one of the most used applications this shortcoming is minor. The pedagogical limitations are related to a series of aspects categorized as preparation, simulation, functionality and perception. First, the use of ACME_Spreadsheet module implies a series of preparatory steps taken by the teacher in terms of ideating the 'what' and 'how' the student should learn, the 'what' and 'how' the student should be taught and from this what can be parametrized and systematised through automatization. Second, the teacher needs to foresee a variety of scenarios in relation to all the parameters of the exercise and its solution. Third, managing a novel technical application by a teacher always has a certain failure probability in some aspect that adds stress and tension to the scenario. Last, even if all the previous steps are executed successfully, there remains the uncertainty of how it will be perceived by students and the effort-reward balance.
Weighed against these limitations, which are also avenues for future development, and terms of the experience of using ACME_Spreadsheet once all the necessary steps according to the requirements are taken, the system is easy to use and the academic results are positive. Moreover, a conditional progressive approach is also planned to be applied to solutions wherein once a problem is solved in a first stage the problem passes to a second stage, which is activated only if and when the previous step has been correctly completed.