Oral-based rubrics design : A case study with undergraduate Spanish students in ESP settings

During several decades, considerable attention has been paid to speaking assessment procedures. Probably, the reason for this relevance lies in the difficulties in dealing with oral assessment and in determining the adequate type of assessment (Campbell, Mothersbaugh, Brammer & Taylor, 2001; Stoynoff, 2013; Schwartz & Arena, 2013). The main aim of this article is to analyze to what extent students’ oral competences (such as their English competence and fluency) affect their peers’ English oral production. More precisely, we intend to study effective procedures to assess oral production in ESP contexts. In order to assess our students, we have designed a speaking-based rubric as the main instrument (called ‘Speaking Diagnostic Test’), based on previous research (Wilson, 2006; Spandel, 2006). A total number of 10 participants were selected in the 1 year course at Universitat Jaume I (Spain). In order to carry out our investigation two different instruments were used for data collection: (1) A ‘Student Questionnaire’ to reflect students’ preferences towards taking an oral exam with their peers; and a (2) ‘Speaking Diagnostic Test’ so as to assess students’ oral production in English concerning the following variables: fluency, vocabulary, grammar, pronunciation, coherence, and communicative ability. The results obtained from both questionnaires will help us identify to what extent students’ preferences in working with other classmates influence their English oral production and therefore to what extent these results could lead to the reformulation and modification of the assessment methods.


Introduction
In recent years, teachers of English as a Second Language (ESL) have paid particular attention to the relevance of communicative competence as an essential aspect in their English language courses.Canale and Swain (1980) and Canale (1983) defined the term of communicative competence as a combination of language knowledge and social knowledge for communication.The former was referred to as knowledge on how to use the language in terms of syntax, morphology, and philosophy; whereas the latter implied the use of such knowledge for actual communication.As a result, ESL teachers have generally agreed on the idea of teaching students how to be communicatively competent in different social contexts and situations as the main goal of their English courses.Moreover, it is important to highlight students' perception on an existing gap between the relevance given to oral performance and the limited time devoted to develop it in the university context (Kavanagh & Drennan, 2008).
In this study, we will be paying special attention to the use of rubrics as an effective oral assessment tool in English for Specific Purposes (ESP) contexts since they have been commonly used during recent years in Higher Education English courses.Particularly, we will be analyzing to what extent students' oral competences affect their peers' English oral production.With peer interaction in mind, in our research we intend to study effective procedures to assess oral production in ESP contexts.

This article begins by outlining the development of
Communicative Approach (CA) and how it has influenced not only the teaching of English as a Foreign Language but also the way in which language competences are assessed.Then the article describes the design of the rubric which served as both assessment tool and as object of study concerning its efficacy.The next section is devoted to the study.This includes the context and participants who took part in the experiment; and procedure and data collection, and instruments employed.Next, the results and interpretation of the data obtained in this analysis are discussed.

Oral-based rubrics: Criteria to evaluate communicative competence
Communicative approaches have become highly significant, trespassing the barriers of rigid learning structures to a conception geared towards the promotion of communication in the classroom.At this point, we feel it is useful to look back into the development of the communicative approach in order to better understand how this approach is often seen going down the path of eclecticism, with illdefined borders.
According to Richards & Rodgers' (1986, p. 72) principles in the Communicative Approach represented a fundamental contribution in the language learning field, since they support learning by means of introducing real-life tasks that involve communication and supply the learners' educational needs.These principles are the following: 1.The principle of communication (activities involving real communication that promote learning).
2. The principle of tasks (those activities that involve using the language to carry out significant tasks that promote learning) 3. The principle of significant relevance (language becomes meaningful for learners in order to positively support their learning process).
As for the first principle, discursive competence was one of the dimensions taken into consideration as an optimal tool to promote communicative competence by the first authors who launched the notion of Communicative Competence (Halliday, 1970;Widdowson, 1978;Canale & Swain, 1980).
With regards to the second principle, the notion of tasks became central in the Communicative Approach.The materials used for task design were taken from real-life resources and not manipulated at all for pedagogical purposes.The introduction of realia would become an important aspect to be taken into account, since realia are believed to be the most convenient kind of material to promote language learning due to their discursive and cultural context.
Concerning the third principle, the Communicative Approach places the learner at the centre of the learning process.Therefore, learners' needs and objectives are the basis to construct a flexible and dynamic syllabus.By employing a Communicative Approach, an interactive and meaningful relationship among learners' needs, Bellaterra Journal of Teaching & Learning Language & Literature. 8.3 (Aug-Sept 2015) ISSN  objectives and the syllabus can be achieved.This multiple relationship can increase learners' involvement in their own learning process.
Within this context and in order to fulfil the communicative standards provided by the Common European Framework of Reference for Languages (CEFRL), an assessment tool (i.e.rubrics) is needed.For this reason, 'rubrics' have become very popular in the field of language education and have caused a great impact among teachers and students as a new tool to evaluate oral production in English.
Despite the vast amount of definitions for the word 'rubric' found, for the purpose of this research we use the term 'rubric' in the same way that researchers Allen and Tanner ( 2006) already pointed out to refer to "a type of matrix that provides scaled levels of achievement or understanding for a set of criteria or dimensions of quality for a given type of performance…" (p.197).Along these lines and concerning our study, our intention is to assess undergraduate students' oral performances with the design of a speaking-based rubric and then to analyze the efficacy of said rubrics.
In order to use the adequate type of rubric, we need to focus on a specific kind that establishes a link between a particular content and the objectives that account for a given subject matter.At this respect, Allen and Tanner's (2006) description of analytical and holistic rubrics may lead us to a quite open view of how our rubric might be categorized.Thus, according to these authors, analytical rubrics "use discrete criteria to set forth more than one measure of the levels of an accomplishment for a particular task", whereas holistic rubrics are defined as those that "provide more general, uncategorized […] descriptions of overall dimensions of quality for different levels of mastery" (p.198).
Many educators have mentioned and suggested relevant communication features so as to assess speaking in general speaking courses (Jones, 1994;Rubin, Graham & Mignerey, 1990).Moreover, students' learning assessment has traditionally and basically been focused on written exams; in contrast to non-traditional assessment procedures, such as portfolios and oral assessments.In fact, many oral assessment procedures require the use of a rubric; it is probably for that reason that non-traditional assessment is on the rise of many educators concerns at all educational levels, especially at universities.
Evaluators spend most of their time listening to students' speeches, and then discussing their oral assessment regarding each of the competencies reflected in a speaking rubric.In this respect, evaluators usually share the same points of view concerning what constitutes the features of communicative competence, but on some occasions they do not agree on the same students' score.For this reason, the use of rubrics has become a very influential tool for assessment procedures and results with regards to maintaining consistency among teachers (Dunbar, Brooks, & Kubicka-Miller, 2006).• Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.

Designing a Rubric for ESP
• Can deal with most situations likely to arise while traveling in an area where the language is spoken.
• Can produce simple connected text on topics that are familiar or of personal interest.
• Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans.
These descriptors were then developed into more detailed components of the language dimensions we aimed at evaluating, including: Fluency, Vocabulary, Grammar, Pronunciation, Coherence, and Communicative Ability (how these are used in the rubric is illustrated in the extract taken from the rubric in figure 1).This last  The rubric was then applied to the following activities: students were asked to perform both a monologue and two dialogues.In the monologue students performed they had to develop the topic "Why do you think studying English is important?".This was followed by a performance two dialogues (role-plays) with two different classmates (regarding their preferences in working with others, as they stated in the 'Student Questionnaire') on the topic of "The language of socializing: A night at the opera".The rubric was implemented and modified so as to include specific grammar and vocabulary in relation to these topics.Both communicative tasks were designed bearing in mind the importance of performing real-life situations Finally, it is important to emphasise that the same 'Speaking Diagnostic Test' was employed in the three tasks (i.e.monologue and two dialogues).Accordingly, by using the same rubric we aimed to obtain relevant data from students' performances in different tasks while evaluating the same components, descriptors, and rates.In other words, we were interested in establishing a contrast between students' results in the two different communicative tasks under the same assessment criteria, but under different circumstances.The results obtained point at discerning the validity of this rubric as an effective tool for assessment.

Context and participants
The aim of this study consists of the analysis of students' features (such as their English competence and fluency) in order to find out to what extent those features affect their peers' English oral production.
The study was carried out in four stages: 1. Elaboration and design of a 'Student Questionnaire' designed to find out a classmate they would like to take an oral exam with and another classmate they would not like to.

Elaboration and design of a 'Speaking Diagnostic Test'
3. Realization of a monologue on the topic "Why do you think studying English is important?"4. Realization of two dialogues (role-plays) on the topic "The language of socializing: A night at the opera".Concerning the students' participation, 10 subjects were selected in the 1 st year course at 'Universitat Jaume I' (Spain).
Although they gave their permission to use their data, an individual identification code was provided in order to safeguard their privacy.

Procedure and data collection
The study took place during the students' regular class time in the first semester of the 2013-2014 academic course.The module is compulsory for all the students; and their participation in this study, although not mandatory, was presented as one of the complementary activities to be given partial credit in addition to the final mark at the end of the semester.In order for results to be available, students had to fill out a questionnaire ('Student Questionnaire') to be able to observe what exactly the students had answered.

Instruments
The main instrument for our study has been (a) a speaking-based rubric, also called 'Speaking Diagnostic Test' (see appendix 1).
Nevertheless, we have used other instruments endorsed for data collection: (b) 'Student Questionnaire', and (c) 'Voice Recorder'.

'Student Questionnaire'
In order to determine the students' profile and information about their attitude towards learning English, a questionnaire was used, namely, 'Student Questionnaire' Appendix 2).This questionnaire elicits students' name and surname, age, gender, and two simple questions addressed to the students in order to get information about their preferences towards taking an oral exam with their classmates giving proper justifications.
Question 1 (Q1): Asks students to give a name and a reason for choosing a classmate they would like to take an oral exam with ('Ideal Partner').
Question 2 (Q2): Asks students to give a name and a reason for choosing a classmate they would not like to take an oral exam with ('Unwanted Partner').These questions will help us classify students in pairs regarding their preferences towards taking an oral exam with their peers.

'Speaking Diagnostic Test'
Students' oral production was measured by means of a 'Speaking Diagnostic Test' that was previously designed considering the following categories: Fluency (Fl.), Vocabulary (Voc.),Grammar (Gr.), Pronunciation (Pr.), Coherence (Co.), and Communicative ability (Comm.ability).The score for each category could vary from 1 to 3, 3 being the highest score that could be obtained in each category.
Thus, students could have a maximum of 18 points in this rubric, in case that their oral production was perfectly performed in the task.
All the variables included in this test are considered as relevant (as in conventional rubrics) in order to assign a certain score in each of them to every student.
The 'Speaking Diagnostic Test' was used in two different stages of our study.Firstly, the rubric was used in order to assess a speaking monologue task in which students were asked the following question: 'Why do you think studying English is important?'.
Secondly, it was used in a different task in which students had to perform a role-play that consisted of inviting a peer to go to the opera.
Hence, the rubric was used in two different tasks with the aim to analyse to what extend students' oral production may be positively or negatively affected by their oral interaction with other peers.In order to analyse such a possible existing influence, we compared students' scores obtained in the rubric in their individual task (monologue) and in the interaction tasks (role-plays).

'Voice Recorder'
Students' oral production in the different tasks of our analysis was recorded by means of a 'voice recorder', in which all students' monologues and dialogues performances were kept for the purpose of Bellaterra Journal of Teaching & Learning Language & Literature. 8.3 (Aug-Sept 2015) ISSN  data collection in our study.In addition, this tool helped us to review students' performance in terms of vocabulary, grammar, communicative ability, pronunciation, and so forth.Therefore, this instrument has been useful in order to check and compare the rubric's scores obtained by each student, so as to support their assessment result.

Results and discussion: Students' scores
The results reveal the different elements or variables included in that rubric, taking into account in both the monologue and dialogues performances on behalf of the students.The 'Speaking Diagnostic Test' shows the different variables that we considered in order assigning a certain score from 1 to 3 in each of them.The total score regarding all these variables is 18 points.Therefore, this test is our

Students' scores in the monologue
As for the students' scores obtained in the monologue, the table below (Table 1) reflects the total amount of students who participated in the experiment, as well as the scores obtained in each of the variables and final scores.

Comm. ability
Due to research purposes, we are not going to analyse in full detail the results concerning the students' scores in the monologue, but we will proceed to comment on them in general terms since these results are the product of an initial proficiency level task ('Speaking Diagnostic Test').
This table provides indicative results, because of 10 students that are indicated in the table; only 2 students achieved an average result (9/18).These results seem to confirm that the students' initial English proficiency level did not reach our desirable expectations.Concerning the two dialogues (role-plays), it is worth highlighting that in order to obtain relevant results, students indicated their preferences towards taking an oral exam with their peers in a 'Student Questionnaire' that was designed for the purpose of this research.

Students' scores in the dialogues
Thus, according to their preferences ('IDP'/'UNP'), the following pairs were selected: -  From these results we may say that only 5 individuals did not reach the average score (9/18) with regards to the final score ( 18points).Thus, some examples such as students 'A', 'B', 'F', 'I', and 'J' do not reach the average class level.Accordingly, we will now comment these students in full detail: 'Student A' does not show fluent speech, which is reflected in long pauses and breaks.Vocabulary expressions and grammar use is rather poor.Furthermore, he does not employ specific vocabulary demanded in the subject.He pronounces words incorrectly, but he tries to be coherent.However, he does not adapt to his partner's level, 'Student J' does not show fluent speech, which is reflected in long pauses, hesitations, and long breaks.Furthermore, vocabulary expressions and grammar use is poor.He does not employ the specific vocabulary demanded in the subject.Although he pronounces words incorrectly, he makes an effort in transmitting a coherent discourse and be understandable.However, he does not adapt to his partner's level, which shows a great communicative ability in English.Finally, for these reasons this student obtains 7/18 points in his final score.
On the other hand, students 'C' (13/18), 'D' (13/18), 'E' (12/18), 'G' (9/18), and 'H' (13/18) were perceived as having obtained the highest scores in this role-play task, in which they could express their preferences when choosing their pairs.The five of them showed better oral skills than the rest of the subjects taking part in the same task.They spoke mostly fluently with a good command of vocabulary (using appropriate topic vocabulary); they used basic grammatical structures, although they still made some key mistakes.
Furthermore, they were able to pronounce mostly all words correctly with correct intonation and word stress, so as their messages were mostly coherent but with few linking words.Finally, their communication ability was excellent, as these four students used turn taking and interrupting techniques very effectively.
As table 3 illustrates, the students who obtained lower scores in this case were 'A' (6/18), 'B' (6/18), 'C' (8/18), 'F' (8/18), 'G' (6/18), and 'J' (6/18).The six of them were below the average (9/18) when performing the task, as they were not able to speak fluently, because were very poor and contained many significant mistakes.In addition, they made many pronunciation mistakes with non-existing intonation and word stress in their production.In general, their messages were incomplete and incoherent in most cases, as they did not use connectors.And finally, their communicative ability was very poor too, since they were not able to use turn taking or interrupting techniques.To conclude, looking at the percentages of the 10 students who participated in this study concerning learners who performed the tasks with someone they did not initially want to interact with, we can observe that 60% had a lower performance while 20% had an invariable performance and another 20% had a higher performance.

Conclusion
The present work focuses on analysing to what extent students' oral competences (pronunciation, fluency, grammar, etc.) are affected by interaction with their peers, in relation to their peer's perceived level of English oral production.Furthermore, we intended to study effective procedures to assess oral production in ESP contexts.
We have seen that our study was based on the design of a rubric ('Speaking Diagnostic Test') that was employed to test students' initial performances in English.This rubric has been useful to assess students' performances in the two role-plays.This instrument (i.e.rubric) has also been relevant for several principles that imply conclusions must be taken into consideration in order to reformulate and modify the assessment methods.However, this study has some limitations, and the recognition of these should help refine future research efforts.Firstly, it is important to consider modifications and/or implementation of the different variables used in the rubric in order to focus more in depth on students' weaknesses.This fact may lead to reconsider students' needs in English ESP courses.Secondly, regarding the 'Student Questionnaire', we could include more specific questions and statements so as to obtain more precise information about the students' preferences.We could include a list of reasons why students would (not) choose a classmate, as we have found that students felt reluctant to provide such information.Thirdly, we concede that there have been a limited number of subjects in our study.Bigger groups of students were not available at the time, and with more students we could have more likely found more nuances.
Finally, we think it is necessary to investigate not only on the use of rubrics on the part of the teacher ('Teacher assessment'), but also their use on behalf of students ('Peer assessment').This last idea could contribute to the students' awareness on a series of limitations and aspects to consider when taking part in an oral exam in English for ESP contexts.
This study has only been a first step towards the design of more complex rubrics with bigger quantities of students as subjects.In the light of the results of our study, we view a quite open field to further explore the design and use of rubrics in ESP contexts.

Figure 1 .
Figure 1.Sample of part of the rubric

4. 1 .
Students perform the role-play with a classmate who they would like to take an oral exam with their 'Ideal Partner', (shortened as 'IDP').4.2.Students perform the role-play with a classmate who theywould not like to take an oral exam with ('Unwanted Partner', shortened as 'UNP').
main instrument in order to get relevant data concerning students' results in the two tasks they performed: (a) Monologue employed as a diagnostic placement test; and (b) Two dialogues (Role-plays) with classmates who wanted to work together ('Ideal Partners') and classmates who did not want to (Unwanted Partners').
'Ideal Partner' ('IDP'):'Student A' and 'Student E', 'Student   B' and 'Student C', 'Student C' and 'Student F', 'Student D' and   'Student I', 'Student E' and 'Student D', 'Student F' and 'Student C',   Student G' and 'Student B', 'Student H' and 'Student D', 'Student I'   and 'Student E', and 'Student J' and  'Student D'. -'Unwanted Partner' ('UNP'): 'Student A' and 'Student I', 'Student B' and 'Student A', 'Student C' and 'Student A', 'Student D' and 'Student A', 'Student E' and 'Student F', 'Student F' and 'Student E', 'Student G' and 'Student H', 'Student H' and 'Student G', 'Student I' and 'Student D', and 'Student J' and 'Student A'.Regarding Table 2 (Students' scores in the role-plays -'Ideal Partners'), the 10 pairs who wanted to take an oral exam together ('IDP') are shown, providing their score in each of the variables as well as their final score.

Figure 2 ,
Figure 2, below, illustrates the final percentages obtained with regards to the comparison between students who wanted to take an oral exam together ('IDP') and those who did not want to interact with certain classmates ('UNP'), to what their preferences in the 'Student Questionnaire' are concerned.

Figure 2 :
Figure 2: Percentage of results with 'Unwanted Partners' When designing the rubric, we first asked ourselves the following question: "How should educators use a rubric in ESP contexts with undergraduate students?"With this question in mind, we began to

Table 2 :
Students' scores in the role-plays -'Ideal partners'

Table 3 :
Students' scores in the role-plays -'Unwanted Partners'

Table 4 :
Students' scores in the role-plays -'Ideal Partners' and 'Unwanted