Unified ( Russian ) State Exam in English : Reading Comprehension Tasks

The article summarizes the study of Reading Comprehension Tasks utilized in preparation for Unified (Russian) State Exam. The corpus of reading tasks was analyzed with the use of the classification algorithm developed by Weir and Urquhart (1998), and aimed at determining the level of engagement (local or global) and type of engagement (literal or interpretative) of a reader performing a Reading Task. The analysis demonstrated manifestations of both levels and types of engagement: the majority of Reading Tasks of ‘Heading-Text Match’ type are found to be of ‘global-interpretative’ configuration, Tasks 2 in EGE Reading Section are typically of a ‘local-literal’ configuration, while Multiple Choice Tasks (Task 3) require both careful and expeditious (in rare cases) reading skills and two types of engagement—literal and interpretative.


Introduction
The main objective of the study is to discuss and evaluate the existing reading comprehension tasks formats utilized to measure the construct of reading in the Reading section of the Unified State Exam (EGE-'Ediny Gosudarstvenny Ekzamen') currently in use in the Russian Federation.In EGE reading is assessed as a constituent part of the overall language proficiency.At the polar opposite of the oral exams and essays that were the basis of the Soviet secondary school testing system, the new multiple choice testing is supposed 'to be much more transparent' (Medvedev, 2009).Besides, with the increasing pressure placed on Russian Universities and schools by the new education standards and quality assurance (Federal'nyy Komponent, 2014), few educators can avoid the increasingly central role that EGE plays in schools.

Literature Review
Reading has always been a key element in foreign languages testing.The modern views on reading as an activity imply that a wide variety of skills and knowledge is to be acquired by an individual to be able to process the printed text.Among the total of 266 skills distinguished by Munby (1978) the most important emphasized by Moore et al. (2012) are as follows: inferring the function of texts with explicit indicators; inferring links between different parts of texts through syntactical and morphological cohesion devices, contrast, distinction, juxtaposition etc.; locating specifically required information.Nuttall (1996) argues that the central ideas behind reading are as follows: the idea of meaning; the transfer of meaning from one mind to another; the transfer of a message from writer to reader; how we get meaning by reading; how the reader, the writer and the text all contribute to the process.Urquhart and Weir (1998, p. 22) define reading as 'the process of receiving and interpreting information encoded in language form via the medium of print'.
Supporting the idea of reading as a bottom-up process, C. Nuttall metaphorically states that it involves 'word attack' skill, 'sentence attack ' skill and 'text attack' skill (1996, pp. 41, 62, 78).Proponents of reading as a top-down process advocate the idea of readers calling upon their background knowledge and/or using contextual information (analysis-by-synthesis) with a focus more on macro-level constituents-genre, text structure, as well as the role of background schematic knowledge etc. (Hudson, 1998).
The modern interactive, hermeneutic approach on reading supported by a number of experts implies a degree of bidirectionality in the process (Hudson, 1998): top-down to predict the meaning and bottom-up to check it.

Methods
The analytical framework applied in the study is the taxonomy dveloped by Weir and Urquhart (1998), and applied in a number of recent studies into the IELTS academic reading test (Weir et al., 2009;Moore et al., 2012).The taxonomy is based on different levels of engagement in reading and types of reading (careful vs expeditious).The contrasted levels of engagement presented in the taxonomy are local and global.If operating at a local level (sentence, paragraph) a reader is scanning for some specifics, while the global level implies comprehending main ideas, the overall text or texts (Weir & Urquhart, 1998).Careful reading involves reading of texts for details, while expeditious presupposes 'quick and selective reading … to extract important information in line with intended purposes' (Weir & Urquhart, 1998, p. 101).The defining features of careful reading, according to Weir and Urquhart (1998, p. 103), are as follows: (a) the reader attempts to handle the majority of information in the text, that is, the process is not selective; (b) the reader adopts a submissive role, and accepts the writer's organization, including what the writer appears to consider the important parts; (c) the reader attempts to build up a 'macrostructure' on the basis of the majority of the information in the text.After Pugh (1978, p. 53) scanning is defined as a process aimed at finding a 'match' between what the reader seeks and what the text supplies.In scanning, very little information is processed for long term retention or even for immediate action (Rahman, 2004).Inferencing is understood as '… making use of syntactic, logical and cultural clues to discover the meaning of unknown elements' (Grellet, 1996, p. 14).
The methods used in the paper are similar to those applied in the research of the IELTS Reading tasks (Moore et al., 2012).

Unified (Russian) State Exam (EGE)
Having replaced the system of school leaving tests in foreign languages and an independent entrance exams administered by universities, Unified (Russian) State Exam (EGE) has significantly changed the Russian education system.It is designed as a high stake, timed, pen-and-pencil exam to 'combat corruption and provide all school leavers with equal opportunities to receive a quality [vocational] education' (Verbitskaya, 2014).Since 2009, EGE is the only form of graduation examinations in schools and a mandatory requirement for acceptance into higher education institutions.At present it is administered by the Federal Service for Supervision in Education and Science together with the local public education authorities of the regions of Russia.In July 2011, a special meeting of the Russian Public Chamber was devoted to improving the EGE.On August 22, 2011, in Maikop speech Dmitry Medvedev (2011) called for optimizing the EGE structure and content.Unfortunately the results of EGE 2014 (Information Telegraph Agency of Russia, 2014) proved that with all the efforts made by its developers to create a new transparent, internationally recognized test, the EGE still needs improvement.The issue made president Putin (2014) addresses its designers in his 9th All-Russia Press Conference and emphasize the need for changes in the EGE system.

Unified (Russian) State Exam (EGE) in English
English as one of the foreign languages taught in Russian high schools is a compulsory subject.The high school course in English is aimed at 'developing language skills, communicative, socio-cultural and academic competencies' (Russian Educational Portal, 2014) assessed at the Unified State Exam which is to become obligatory for all school leavers in 2020 (Livanov, 2014).Describing the approach applied in developing EGE in English, professor A. Green qualifies it as competence/skills-based approach … emphasizing reading and listening skills/strategies, … productive skills (writing and speaking), grammar and vocabulary in context (Green, 2014).
As a form of final assessment of academic progress in learning English by Russian high school graduates, EGE aims at determining the learners' level of foreign language proficiency and comparing it with the requirements of Federal educational standards on foreign language (Federal'nyy komponent, 2014;Kodifikator, 2014).

EGE Specifications (2014) in Foreign Languages
At the moment EGE in English has a four parts structure and consists of 'Listening', 'Reading', 'Grammar and Vocabulary' and 'Writing'.
The exam items complexity fits one of the three levels determined by Federal (Russian Institute of Pedagogical Measurement [FIPI]): Basic, Advanced or High, which correspondingly match levels A2+, B1, B2 of Common European Framework of Reference (CEFR).An item complexity is related to the complexity of the language used and the skills tested.

FIPI Criteria for EGE Reading Items
The genres of the texts used in EGE are as follows: 'mass media, fiction and pragmatic texts' (Spetsyfikatsyya EGE, 2014).The types of texts used on the three described levels are different: Basic level Reading comprehension is assessed through tasks on the language material of short informative texts or popular science texts, the Advanced level implies using mass media (e.g., a review) and popular science texts, fiction and mass media (e.g. an essay) texts are used for High level reading tasks.
The following EGE text selection criteria are to be considered and followed by item writers: 1) The text (or an extract of a bigger text) selected is to be worthy of close study due to its completeness, coherence and cohesion.
2) The ideas and issues explored in the texts are to be appropriate to the relevant age-group (16-18 years).3) The text's intellectual merit is to correspond to candidates' communicative, reading and life experience.4) The text genre is to correspond to the item genre.5) The texts must be free of language or images which would be judged as obscene or offensive by religious, ethnic or any other standards and expectations.6) The text must not be overloaded with terms, proper names and numbers.7) The text complexity is to correspond to the item complexity (Basic, Advanced or High) (Spetsyfikatsyya EGE, 2014).
The topic knowledge assessed at EGE and specified by the developers includes: A) Everyday life, household chores and duties.Shopping.B) City and country life.Urban and rural issues.C) Family and school.Family traditions, interpersonal relations.D) Health issues, medical services.A healthy way of life.E) Youth in modern society, interests and hobbies.F) Free time, extracurricular activities.Sports.Correspondence.G) Homeland and the foreign language country, their geography, climate, population, cities and villages, sights.H) Traveling home and abroad, sightseeing.I) Nature and ecological issues.J) Culture and history of homeland and the foreign language country.K) Russia and the foreign language country's contribution to science and culture.L) Professions and job market.M) Higher Vocational education: possibilities and requirements.N) Plans for the future, career choices.P) Foreign languages role in modern world.Q) Secondary school education.Compulsory subjects, favorite and unpopular subjects.Holidays.R) Scientific-and-technological advance, its perspective and consequences.S) IT.T) Holidays and memorable events in different countries (Spetsyfikatsyya EGE, 2014).
The skills assessed in the EGE Reading tasks are to enumerate in the test specifications in the following way: '2.3.1)Read authentic texts of different styles (publicist, fiction, popular science, pragmatic) and use different strategies/types of reading depending on the communicative problem; 2.3.2) Skim news messages, interviews, broadcasts, popular science texts, extracts from fiction texts; 2.3.3)Scan articles and advertising folders for the information requested; 2.3.4)Read pragmatic texts, popular science texts, extracts from fiction books for detail; 2.3.5)Recognize the main information and miner details of a text, recognize the significant details and facts; 2.3.6)Demonstrate the attitude towards the text read; 2.3.7)Infer chronological and cause-effect relationships, anticipate the development/result of the events/facts narrated, generalize facts and phenomena 2.3.8)Identify the writer's view and separate the main idea from the minor facts given, evaluate significance/novelty of the information, recognize the main idea and the problems raised applying elements of analytical tools' (Kodifikator, 2014).
The EGE in English in its current form provides three reading tasks in three different sections.Each section is based on a reading passage correspondingly averaging about 620 (Part 1), 350 (Part 2), 950 (Part 3) words.The passages are drawn from two main sources: mass media and fiction, with the topics being of general interest, created for general public.The reading passages are followed by a variety of tasks (20 in total) aimed at assessing candidates' skills to comprehend the material in the time (40 minutes) allocated.Candidates' understanding is tested in three different tasks.The tasks are as follows: matching the text (7) and the heading ( 8), text gap (6), multiple-choice tasks (7).
These tasks are specified in Demo EGE (2014) as follows: Part 1) Fill in the table matching Headings 1-8 with texts A-G.Use every number only once.There is one odd heading in the task.Part 2) Read the text and fill in the blanks A-F with phrases marked 1-7.There is one odd phrase in List 1-7.Part 3) Read the text and do tasks А15-А21.Draw a circle around the number 1-4 you chose (FIPI, 2013).

EGE Task Survey
The EGE reading task samples under study were sourced from: 1) the official site of FIPI (2003FIPI ( -2013)); and 2) practice test material (EGE in the English Language, 2013;Variant No. 125958, 2014).It is assumed by the authors that the FIPI website materials as those partly made up of past papers better correspond to the form, content of real EGE tasks than any commercial materials as they are based on the corpus used by EGE designers and item-writers and developed by the latter.A total of 17 complete EGE Reading sections tests were investigated, each made up of the three described task types.The analysis conducted was aimed at determining the level of engagement and type of engagement potential examinees are to involve themselves to fulfill the Reading task.

EGE Reading Tasks
The EGE corpus collected for the study is made of a total of 17 tests, with each of these tests comprising three Reading Sections based on three different texts.In all, the total number of reading tasks under study is 51, comprising 340 separate items.Table 1 lists the task categories identified in the Corpus and their absolute frequencies.The left hand column demonstrates the total number of each task category used in the corpus, and the centre column contains the figures reflecting the number of items in each category.In the current EGE format the task for examinees is to match a Text (usually a paragraph) with the Heading (a phrase or a word) that summarized its contention.An example of the task is given below.

Sample 1
Choose the correct heading for sections A-G from the list of headings (1-8) below.Fill in the table matching Headings 1-8 with texts A-G.Use every number only once.There is one odd heading in the task (The instruction is presented in Russian).

_____________ 8. _____________
A. Most people who spend a holiday traveling take a camera with them and photograph anything that interests them-<…>.Later looking through their albums they will remember the happy time they have had, the islands, countries and cities they have seen.
... D. The seas are in danger.They are filled with poison like industrial, nuclear and chemical waste.<…>.If nothing is done about it, one day nothing will be able to live in the seas.Every ten minutes one species of animal, plant or insect dies out forever.
In Sample 1 above, as the number of Headings exceeds the number of Texts, the fulfillment of the task implies (a) comprehension of the texts and (b) identification of the corresponding headings from the list provided.If the ratios (Heading-Texts) were balanced, the sequence would be reversed, with examinees beginning with the Heading and then matching each of these up with the corresponding Text.

Level of engagement
As the designated name of this task type indicates (i.e.Heading-Text match), the level of engagement is clearly at a supra-sentential level.In almost all the cases registered in the Corpus, the unit of text to be considered in the completion of tasks was one paragraph with the paragraphs being different in length.In the Sample above (1), for example, Text 1 is one sentence long (62 words); Text 2 is longer, running to 5 sentences (72 words).In the whole corpus, the average paragraph length was three sentences, typically 65 words.

Type of engagement
To accomplish Heading-Text match tasks, examinees are supposed to have the skills to match up a putative summary of a text with the propositional content of the text.The task also assesses the skill to recognize and comprehend a broad list of linguistic categories e.g.synonymy, generalization, cause-effect.Thus, in Text A (Correct answer 1), we see that the relevant category for the text in question is synonymy of the words with the same root ((1) memories-remember …); in Text D (Correct answer is 7), the category expected to be acquired is 'generalization' ((7) If nothing is done …).The task for examinees is to be able to recognize the link between the content of the text comprehended, and the category around which the heading is structured.
In Sample 2 (below), the task implies examinees to have the skill to recognize and identify the way significant lexical units such as 'a big break with 'tradition' equate to the 'Great changes'.

Sample 2
Choose the correct heading for sections 1-7 from the list of headings (A-H) below.Fill in the table matching Headings 1-8 with texts A-G.Use every number only once.There is one odd heading in the task (the instruction is presented in Russian).There are no musicals, no comedies and no Agatha Christie thrillers.The new management is staging three Shakespeare plays and two plays by the German playwright Berthold Brecht, in addition to a very modern pantomime which doesn't have either a Prince Charming or two ugly sisters.We wish the Hippodrome every success in its efforts to bring culture to the masses.
Thus, the task for examinees in the Text-Heading match category assesses the skill of a 'pragmatic' comprehension of the text described by Taylor (2009).A variation in the design of Heading-Text match tasks in different tests deserves a brief comment here.Whereas most IELTS summary prompts are typically realized in a neutral, academic style, EGE headings tend to sound more laconic and bear features of a more idiomatic, 'publicistic' style (for the description see Moore, Morton & Price, 2012).Sample 2 above exemplifies this 'publicistic' style.
These 'publicistic' rather than academic headings are characterized by a number of features: (a) lack of reference to larger rhetorical units (e.g.reason, anticipation etc); (b): a wide spectrum of grammatical forms from free word combinations to set phrases and collocations to different types of sentences.
Evidently comprehending and matching much less systematic categories of the type provided in EGE Samples 1 and Sample 2 requires well developed interpretative skills from an examinee.

Level of engagement
The majority of items in the gap text tasks above are aimed at locating some specific information.E.g. completing items A or B in Sample 3 above, test takers are supposed to recognize the syntactic structure of the sentence and the phrase (Correct answers are 6 and 4).The level of engagement with the text as we see in the Sample above is fairly local.
Very few of the Text gap tasks individual items are not to be completed on a sentence level, but are to be comprehended in relation to the whole text, or at least a paragraph level.Thus, for example, item C above (Sample 3, the answer is 2) is unlikely to be completed correctly if the examinee does not read the sentence in bold below to be confident about the answer.

Type of engagement
The way in which test takers completing Task 2 items need to engage with material is fairly literal.As we observe in Sample 3, the task implies nothing but matching the semantic plausibility and syntax of the corresponding sentence with the phrase in the list below.Thus, the majority of items in the Corpus are completed by juxtaposing the sentence in the reading passage with the phrase in the list provided.

Task 3: Multiple Choices
About 30% of items in the corpus are items of a multiple choice format, with typically a 4-option structure.
Examinees are to choose ONE 'correct' option.Sample 4 provided demonstrates a variety of multiple choice items based on the passage entitled 'Why I sent Oxford a rejection letter'.

Sample 4
Read the text and do tasks А15-А21.Draw a circle around the number 1-4 you chose (In Russian).

Why I sent Oxford a rejection letter
So, why did I apply in the first place?If you're achieving high grades at A-level (or equivalent), you can feel quite a lot of pressure to "prove yourself" by getting an Oxbridge offer.Coupled with the fact that I grew up on benefits in council estates throughout Bristol-not a type of heritage often associated with an Oxbridge interview-I decided to give it a try.

…
A 18 Judging by paragraph 7, the author comes from a family which is … educated. big. aristocratic.
not very rich.

Level of engagement
The Multiple choice task format in the EGE Reading texts corpus is, unlike that in Parts 1 and 2, not distinctive for one particular level of engagement.Any generalizations for the multiple choice tasks in the corpus studied are controversial as different items correspond to different levels of text.It is obvious in all the items samples provided above and below.
E.g. items A19 below exemplifies engagement at a paragraph level.

Sample 5
To me, withdrawing my application to an institution that is a symbol of unfairness in both our education and the legal system (which is so dominated by Oxbridge graduates) makes perfect sense, and I am reluctant to be part of a system so heavily dominated by such a narrow group of self-selecting elites.
A 19 The author believes that the selection to Oxbridge … is hard to understand.
is unfair.
needs improvement.
In item A17 a test taker is to demonstrate a more 'local', propositional level of engagement (See Sample 6).

Sample 6
Day-of-play tickets are sold on strictly one-per-person queuing basis.
A17.In paragraph 2 'day-of-play' means that: one person can buy many tickets for the match.
you can get a ticket on the day of the match.
you can book a ticket on the day of the match.
you must queue for a while.
Item A15 on the contrary implies engagement with a more extended part of the text-two paragraphs at least, as seen below (Sample 7).

Sample 7
A little over a month ago, I sent Oxford a rejection email that parodied the thousands that they send each year.Much to my surprise, it has become a bit of an Internet hit, and has provoked reactions of both horror and amusement.
In my letter I wrote: 'I have now considered your establishment as a place to read Law (Jurisprudence).I very much regret to inform you that I will be withdrawing my application.I realize you may be disappointed by this decision, but were in competition with many fantastic universities and following your interview, I am afraid you do not quite meet the standard of the universities I will be considering.' A 15.The email letter the author sent to Oxford was meant to be … desperate. respectful. mocking. regretful.
Finally, item A21 requires consideration not at a sentence level as could be the case with the questions of this type, but the text as a whole-a ten-paragraph text (Sample 8).

Sample 8
Why I sent Oxford a rejection letter ….
Although I share concern that not going to Oxbridge gives you a "chip on your shoulder", I did not write to Oxford to avoid the risk of being labeled as an "Oxbridge reject": I already am one.Last year I made an (admittedly weak) application to Cambridge and was inevitably rejected post-interview.
A year ago, I was in awe of the beautiful buildings of Oxbridge, but today I am in awe of the sheer number of people who, like me, have managed to not take it so seriously.Ultimately, I am not harming Oxford by laughing at it, and it is an amazing feeling to realize that so many people are enjoying my email.Actually, I was amazed to know how many people of different ages bothered to read it and even to leave their comments about it in Facebook.I had fun reading some of them, too.
A 21. "It" in 'have managed to not take it so seriously' in the last paragraph refers to … Oxbridge rejection. Oxford.
university studies.
university interview.
In rare cases as in Sample 9 below the text does not provide enough information and test-takers are expected to use not only their skill of recognizing and interpreting culturally specific references in texts but background linguistic knowledge of idioms as well.The associated text (not only the paragraph below) contains semantic links with at least 3 first options Sample 9 It was only at the interview that I started to question what exactly I was trying to prove.I was well aware that fantastic candidates are often turned down, and I did not believe that this was a true reflection of their academic potential.
Although I share concern that not going to Oxbridge gives you a "chip on your shoulder", I did not write to Oxford to avoid the risk of being labeled as an "Oxbridge reject": I already am one.Last year I made an (admittedly weak) application to Cambridge and was inevitably rejected post-interview.
A 20.The expression "chip on your shoulder" in paragraph 9 means … feelings of unfair treatment.
reflection of one's potential.
below-average performance.
record of achievements.

Type of engagement
EGE multiple choice tasks in our corpus offers a range of modes of engagement.Thus, in Item 17, for example, item-writers expect a potential examinee to find and identify quite specific information (i.e. the meaning of the collocation 'day-of-play').It is definitely literal.Item 15 is aimed at assessing examinees' skill to compress the concept of a whole section (a letter) in a word (i.e.mocking).The processing obviously implies 'interpretative' skills and 'interpretative' engagement.There are also a number of samples (Item 21) testing examinees' skill to identify the relations between a point made and the corresponding quotation from the text (i.e.chip on your shoulder).

Conclusions
In each Reading Task of the Unified (Russian) State Exam in English test-takers respond to reading in three different ways: selecting a heading from the bank to match with the identified text (Task 1. Heading-Text Match), filling in gaps in a text (Task 2. Gap Text), selecting a single 'correct' option from a four-option structure (Task 3. Multiple choice).Of all the skills to be acquired by test-takers and enumerated in EGE Specifications (2014) above, only three are really assessed in the existing EGE reading construct: Task 1 tests skimming skills, Task 2 aims at testing the skill of inferencing local (predominantly sentence level) relationships, in Task 3-the skill of reading for detail.
A sizeable proportion of the studied EGE Reading tasks has a local-literal orientation (Task 2, Task 3), but there are a few that require interpretative type of engagement (Task 1, Task 3).For a number of Task 2 items (i.e.Gap text), the engagement expected varied from highly 'local' to 'highly'.The items in the corpus studied that obviously belong the 'local-literal' category are Heading-Text Match items and Multiple Choice items.They both that require comprehension of the reading passage as a whole.
For the analysis overall, we have to acknowledge that the research is completed and, thus, the results are based on the Corpus of test tasks, which were not derived from real live test materials.Assuming however, that the data used represent some approximation of current item-design practices at EGE, we argue, that the research conducted provides a broad picture of EGE Reading Comprehension Tasks.

A
new winter season at the Hippodrome Theatre is a big break with tradition.

2004 there were more mobile phones in the UK than people. <…>
Task in discussion imply: a wide variety of syntactic structures: noun phrases (e.g., Useful Invention; US Younger Generation; Modern Branch of Industry; Historical Separation; Verbal Misunderstanding); separate nouns (e.g.Entertainment; Restaurant; Excursions); idiomatic collocations (e.g.Protected by law, All in One; Out and About) and implicit references (e.g.Old Enough, Hard to explain how they could).Neither sentence structures nor question forms are used as headings in EGE Text-Heading format tasks.4.6.2Task2: Text Gap Gapped text type tasks account for about one third of the reading section of the EGE.Six sentences, one in each paragraph in the text of about 300 words contain a blank to be restored by test takers with the phrases presented below (See Sample 3 below).<…>Michael Harrison phoned his father, Sir Ernest, to wish him a happy new year.Sir Ernest was chairman of Racal Electronics, the owner of Vodafone, A _______________________. <…>.The networks themselves were small <…>.Nobody had any idea of the huge potential of wireless communication and the dramatic impact B _______________________. Hardly anyone believed there would come a day when mobile phones were so popular C _______________________.But in 1999 one mobile phone was sold in the UK every four seconds, and by