Going Beyond Readability Formula: How Do Titles Contribute to the Readability of Essays?

The belief that genre specific reading provides numerous benefits for apprenticing writers is something we in the field of writing education value as an underlying constant. Accepting this, writing center directors stock their self-access library shelves with a variety of composition texts, to include rhetorics (and the essays therein). To select these materials, readability formulae (e.g., the Lexile Readability Formula) are often employed. However, such formulae only measure two of the many features that make up the readability of an essay (i.e., semantic, syntactic). Other important features such as the title are not considered. To address this, this article reports the results of a sequential, mixed-methods study conducted in an Asian postsecondary setting. The study found that titles influence readability both as (a) a primary (i.e., an isolated feature) and (b) a conjoined feature (i.e., consisting of two or more associated entities where the second impacts the first). The article also makes a recommendation for teachers, writing center staff, and the publishing industry that readability formulae be administered in a hybrid fashion to explore additional features such as the title when considering the difficulty of exemplars.


Introduction
The belief that genre specific reading provides numerous benefits for the preparation of apprenticing writers is something we in the field of writing education value as an underlying constant (Thaiss & Zawacki, 2006). That is, those who read more tend to write better (Krashen, 2004), demonstrating as much as a .50 to .70 correlation (Grabe, 2003). Following this, directors stock their self-access library shelves with a variety of composition texts, to include rhetorics (Baker, 2019), rhetorically organized anthologies of paragraphs and essays which explicate major rhetorical forms, present sample texts exemplifying major rhetorical patterns, and offer procedures to show student writers how to reproduce genre in their own writing (Ferris & Hedgcock, 2005). And like all those who select texts for prospective readers, writing center staff need to consider whether the selected texts will be a good match for potential readers (Gardner & Miller, 1999), "the study of which ... has come to be called readability" (Gilliand, 1972, p. 12).
Historically, readability has been defined as the consideration of the text, the reader, and the interaction between the two (Dale & Chall, 1949;Gilliand, 1972;Harrison, 1980;Klare, 1963;Kintsch & Vipond, 1979;Kintsch & Miller, 1981;Schirmer & Lockman 2001). Chall, Bissex, Conrad, and Harris-Sharples' (1996) metaphor-based definition is especially elucidating because they liken readability to an iceberg: beneath the water level there are "various sources of difficulty. The more difficult the passage, the greater the ice beneath" (p. 6). This definition is so appropriate because it points to the fact that readability, like an iceberg, is not one solid, homogeneously intransparent entity. Instead, it is a heterogeneous mix of features that make up what Goodman (1967) called the psycholinguistic guessing game of reading, an activity that requires a consideration of the many features that contribute to the complex phenomenon known as reading.
Early explorations of this phenomenon date as far back as 900 AD (Lorge, 1944) and became more frequent in the early 20 th century (Dubay, 2007). These early explorations were mainly quantitative (focusing on semantic features, syntactic features, or a combination of the two) and laid the theoretical base for the first readability formula (Lively & Pressey, 1923), a predictive device which provides quantitative, objective estimates of difficulty (Klare, 1963). Since then, formulae have employed these two features, as such features have been found to be reliable predictors of readability (Dubay, 2007). However, it was soon acknowledged that other features needed to be considered during a comprehensive assessment of the text/reader combination (see Ojemann, 1934), a concern which continues to be echoed today (Armbruster, 2016;Chall & Dale, 1995;Gunning, 2003;Fry, 2002;Kintsch & Vipond, 1979;Lexile, 2010;Meyer, 2003).
To address this concern, many researchers have supported a paradigm shift toward a hybrid approach, one where a readability formula is first employed and then other factors are considered qualitatively (Fry, 2002). This is because using a readability formula and subjective criteria together reduces the risk of presenting prospective readers "a seemingly appropriate book but one they cannot read due to format, language, structure, or content" (Weaver, 2000, p. 33). The popular readability formula employed in this study (i.e., the Lexile Readability Formula) has received similar attention (Gunning, 2003;Lexile, 2010). Gunning (2003), for instance, suggested that "although teachers might use Lexiles ... , they need to go beyond the numbers ... [and] complement the objective data yielded by the formula with subjective judgment" (pp. 182-186).
Revisiting Gunning's advice, Chall et al's (1996) iceberg reference is remarkably appropriate, especially when thinking about one feature of the essay, the title (the essay's tip), as this is the initial signaling device readers encounter and often utilize to begin processing a text (Noor, 2006), and this strategy has been found to provide the reader with valuable insight into understanding the text (Fan & Liu, 2008).

Titles and schema
Discussions regarding the effects titles have on reading comprehension in native English speaking (NS) postsecondary contexts often begin by referencing Bartlett's (1932) pioneering work with schema. Following this, Dooling and Lachman (1971) began what would be a limited series of discussions on the topic. Extending Bartlett's idea of schema, they presented students with a vague, short (77 word) crafted narrative in two conditions--(a) with a thematic title and (b) without a thematic title --and reported that titles improve "students' overall recall of texts which are vague and metaphorical" (p. 216).
Drawing on Dooling and Lachman (1971), Bransford and Johnson (1973) utilized a 180-word prose paragraph regarding how to wash clothes and added a third condition (providing readers access to the title after reading the text). They found that students who had access to the title before encountering the passage had higher recall whereas those who received the title after engaging the passage or not at all demonstrated lower performance.
Several studies have extended Bransford and Johnson's (1973) work in different directions. Alba, Alexander, Hasher, and Caniglia (1981), for instance, arguing that a test of comprehension was lacking, replicated Brandon and Johnsons' study and explained that having access to the title prior to reading produced higher comprehension than in the other two conditions.
Similarly, Schallert (1976) investigated the meaningfulness of titles with texts from a different procedural perspective. Using a text that could be read with alternative meanings (e.g., a baseball game/ a glass factory), he too presented the texts in each of the three conditions. However, the first condition was presented with alternate biasing titles regarding the two topics. He concluded that students recalled more as a result of receiving a stronger biasing title. He also maintained that having access to a title produced more recall than the other two conditions.

Titles and other variables
Presenting a title in alternate conditions or using biasing titles has been used to explore a number of areas: titles' relationship with (a) gender and age and (b) titles and their relationship to the text 2.2.1 Gender and age Sjogren & Timpson (1979) looking at the question of titles and schema through the lens of gender, utilized short, 145-word narratives and alternating gender biasing titles. They reported that gender specific titles helped readers identify the content of a passage that might otherwise be ambiguously read due to gender related schema.
More recently, Miller, Cohen, and Wingfield (2006), adapting short materials from others (Bransford & Johnson, 1972;Dooling & Lachman, 1971) presented material with (a) a title and (b) with no title. They too confirmed that the presence of titles improves reading comprehension and added that age and working memory were related to this effect: Younger readers (ages 18-34 years) did better than middle aged readers (ages 35-59 years) who did better than older readers (ages 60-85 years).

Titles and their relationship to the text
Work with titles has also focused on how titles affect NSs' understanding of a text's organization (Bock, 1980;Schwartz & Flammer;, sentence level concepts (Smith & Swinney, 1992), and structural areas (proposition, sentence, and word level) (Wiley & Rayner, 2000). Bock (1980), working with biasing titles, presented two different biasing titles and a newspaper story cut into strips. Drawing on students' organization of the texts, Bock concluded "titles provide the starting point for setting up text structures" (p. 308).

Organization
Similarly, Schwartz and Flammer (1981) investigated the effect of the presence or absence of titles with (a) organized, (b) slightly disorganized, and (c) very disorganized 247-word narratives (fairy tales). They found that titles can help readers understand the structure of organized and slightly disorganized texts but have no impact on unstructured texts.

Propositions, sentence level, and word level concepts
In a second experiment with a newspaper story, Bock (1980) explored the effects of biasing titles on NSs' identification of content words and the propositions that titles point to and found that titles can influence students' interpretation of which content words are important. He also observed that propositions marked as important in titles are better recalled. Smith and Swinney (1992), adapting Bransford and Johnson's (1972) materials, investigated the role titles (and the resulting schema the titles activate) play in students' processing of sentence level concepts. They discovered that the presence of a title can improve comprehension of sentence level concepts as well as reading speed.
Likewise, Wiley and Rayner (2000), using materials from other studies (Bransford & Johnson, 1972;Dooling & Lachman, 1971) found that titles can help students process texts at various structural levels: (a) proposition, (b) sentence, and (c) word levels (i.e., ambiguous words). Additionally, they reiterated that when a title is present, "the text is generally read faster, rated as more comprehensible, and recalled better" (Wiley & Rayner, 2000, p. 1,011).

Research with English language learners
Following the work with NS undergraduates, research on the effects of titles has also been done with ELL university students, and although generally more recent, it is much less abundant. Studies with ELLs have demonstrated that NNSs, like NSs, place a great deal of importance on titles and that they too look to titles first when approaching a text (Noor, 2006), but that student proficiency (Carrell, 1983;Fan & Liu, 2008) and the meaningfulness of the title (Fan & Liu;Zhang & Hoosain, 2001) both play a part.
An early work that included ELLs is Carrell (1983). Carrell adapted the materials used in Bransford and Johnson (1972) to examine the effects of titles in a North American setting with NSs (as well as Arabic, Chinese, Greek, Japanese, Korean, and Spanish-speaking ELLs) at two proficiency levels: higher and lower proficiency. She found that for both the NSs and advanced ELLs, "the presence of a title is the single best predictor of readers' perceptions of relative ease or difficulty of comprehension" (Carrell, 1983, p. 192), as it activates students' schema so that they can engage the text. Carrel (1983), noted, however, the same effect was not found for the lower proficient students. That is, the lower proficiency students remained linguistically tied to the text and thus were less able to take advantage of the benefits titles provide.
In a more recent study, Zhang and Hoosain (2001), working with Chinese ELLS in an Asian context, also acknowledged the value of titles, but they, like Bock (1980) and Schallert (1976), looked at the content of the titles themselves. Working with short narratives, they reported that a title, "particularly a meaningful one incorporating a substantive verb ... triggers the relevant schema, and this schema generates expectations about what is likely to happen" (p. 17), the result of which is faster reading speeds and higher recall.
In the most recent study found to date, Fan and Liu (2008), studied Chinese ELLs and short narratives and offered similar results about the importance of titles being meaningful (i.e., incorporating a verb), but they, like Carrell (1983), added that more proficient readers make better use of titles than less proficient ones.
The examination of the extant literature illustrates that early studies done with NSs and more recent work with ELLs point to the effects titles may have on the reading experiences of postsecondary ELL apprenticing writers in the Asian context with short texts (i.e., short crafted paragraphs, fairy tales, newspaper stories); however, empirical investigations with ELLs and longer academic texts (essays), were noticeably absent during the literature review conducted for this study. This article is intended to

Methods
This article reports the results of a larger sequential, mixed-methods study which explored the effects 16 features have on postsecondary ELLs' perceptions of difficulty when reading exemplars from rhetorics. Separating the study and publishing separate articles regarding each feature was done in the interest of length so as to give each unique feature's literature review, data set, and discussion within the length of one article. The purpose of this article is to explore what benefits and difficulties titles pose for postsecondary ELL students in an Asian context (e.g. Taiwan) when they read exemplars (i.e., essays) excepted from rhetorics.

Overall design
To collect and analyze the resulting data to answer the research question, a sequential mixed-methods approach was employed (Johnson & Onwuegbuzie, 2004). To order this mix, an adaption of Creswell's (2013) design was employed (Figure 1).

Figure 1. Sequential Mixed-Methods Research Design
Following the sequential mixed methods design, two sections (steps) and their relevant subsections are reported here: (1) Identifying Text and Participants for Study and (2) An Exploration of the Benefits and Difficulties Titles Provide Students When They Read Essays Excerpted from Rhetorics. These steps were conducted at a university in Taiwan. The university maintains a writing center and offers its 9,000 Step 1 A quantitative comparison of the texts' readability levels (using the Lexile Readability Formula) and students' reading levels (using the Scholastic Reading Inventory) to identify texts and participants Step 2 A qualitative exploration of the benefits and difficulties titles pose to the students who read exemplars from rhetorics Qualitative A cline-questionnaire procedure.

Qualitative
A further exploration of the results of the cline-questionnaire procedure via semi-structured retrospective interviews to add scope and breadth to the findings. students undergraduate and graduate degrees in 16 majors, including a Bachelor of Arts in Applied English (the degree the population of this study is seeking).

Identifying texts and participants
To identify potential texts and participants for study, three stages were performed: (a) an examination of the reading levels of the exemplars in the rhetorics, (b) an examination of the reading levels of the target sample, and (c) a comparison of the two. To identify the exemplars for study, the reading levels of exemplars (N=893) from 12 rhetorics available on the local market were examined. This was done using a non-fee based computerized version of the Lexile Analyzer available to researchers. To determine the reading Lexile level of potential participants, the SRI (Scholastic Reading Inventory) was administered to a purposive sample (N=91), students enrolled in five of the seven sections of sophomore composition at the university, as this group makes up the majority of visitors to the writing center. And to determine which exemplars are accessible to the reading levels of the selected participants, a comparison of the readability levels of the exemplars in the texts and target students' reading levels was performed.

Identifying participants
In keeping with the nature of qualitative theory, the informants were purposively selected to best help the researcher understand the problem (Creswell, 2013). Following Kvale's (1996) suggestion that the number of informants tends to be 15 ± 10 in interview studies, a smaller cluster sample (n=14) was identified from the larger sample. The informants were identified because they received SRI scores at the top of their class (828-928L, the top 15%), which allowed them to examine a wide range of exemplars and help the researcher to holistically explore the research question (Merriam, 1991).
After the informants were identified, they were asked by e-mail if they would be willing to participate in follow-up, post-course interview. Twelve informants assented and were provided with pseudonyms to protect their anonymity: seven females (mean age 20.14 years) and five males (mean age 20.8 years). The makeup of the sample was indicated by the students' Lexile measures which identified them as appropriate participants rather than any purposeful intent of the sampling procedure (e.g., ethnicity, gender, age).

Identifying exemplars
Five exemplars ranging from 610-1010L (Table 1) were purposively chosen to be below, within, and slightly above the informants' Lexile range (i.e., 828-928L). This number was chosen to allow enough variety for the informants to engage in thoughtful comparisons and small enough to be examined and discussed within a reasonable time via the cline questionnaire and interviews, reasonable in that valuable data could be gleaned but informant fatigue could be avoided.

An exploration of the benefits and difficulties titles provide students when they read essays excerpted from rhetorics
Once the texts and the participants were identified, the effects of titles were explored. This was done via an untimed two-stage process: (a) a quantitative cline-questionnaire procedure and (b) qualitative semistructured retrospective interviews.

The cline phase
The purpose of the cline phase was to have the students read the five essays and put them in a cline of difficulty (from easiest to most difficult) so that the students would be able to reflect on this activity while completing a closed-response, Likert questionnaire. The exemplars were presented to the informants in random order and criteria for ranking was withheld in order to ensure the informants engaged in the type of decision-making process "normally used when making such judgments" (Chall, et al., 1996, p. 77).

The questionnaire phase
After the informants ordered their clines, they completed the closed-response questionnaire. This phase was administered to help the informants reflect on why they created the cline the way they did and to relate it in such a way that would provide insight into what other factors beyond readability formulae (e.g., the Lexile Readability Formula), i.e., titles, they feel influence their perceptions of difficulty when reading exemplars excerpted from rhetorics.
The questionnaire addressed 16 features related to comprehension, one of which was the title. The question related to titles is listed in Table 2.  Table 2 Excerpt from the Questionnaire Instructions: Now that you have ordered the texts from 1 (easiest) to 5 (most difficult), please explain your reasons for arranging the texts in the way you did by completing the questionnaire below.
Titles: How well the title of each text described each text influenced my decision about how to arrange the texts in the way that I did.
a. Strongly Agree b. Agree c.
Neither Agree nor Disagree d.
Disagree e.
Strongly Disagree To ensure reliability of the questionnaire, it was translated into the students' L1 (i.e., Mandarin) using a back-translation procedure, checked with a second translator for accuracy, and a pretest was conducted with a small number of respondents who were not part of the sample in the study (n = 2).

Interview
To help the researcher triangulate the data from the questionnaire, the informants (after creating the cline and completing the questionnaire) participated in semi-structured retrospective interviews. Each interview, in accordance with Creswell (1994), began with structured questions from the questionnaire and was followed up with semi-structured prompts which later became open-ended (Nunan, 1996). A bilingual research assistant (translator) was present to assist with any language difficulties, and an observational protocol, which included both video and audio taping, was utilized to record the interviews. The interviews lasted for an average of 32.5 minutes (range 19.3 -57.4 minutes). Variation was dependent on how much each informant had to offer and how much translation was required.
After the interviews were completed, the audio recordings were transcribed, and the informants checked their transcripts. Once these steps were completed, the informants' responses were explored using Erlandson, Harris, Skipper, and Allen's (1993) emergent category analysis procedure, a procedure which allows "categories to follow data rather than precede them" (p. 112). "To add strength and fertility to the entire analysis" (Erlandson, Harris, Skipper, & Allen's (1993, pp. 128-129), a second-level group debate procedure was also included

Results
The purpose of this article is to report the benefits and difficulties titles pose to ELL apprenticing writers when they read exemplars (i.e., essays) taken from rhetorics. The first step helped to quantitatively identify potential texts (N=5) and participants for study (N=14). Twelve assented, 11 reported to the test site, and 10 successfully completed the procedures and thus provided useful data. The second step helped to answer the research question: What benefits and difficulties do titles pose for postsecondary ELLs students in an Asian context (e.g. Taiwan) when they read exemplars (i.e., essays) excepted from rhetorics?
Examining the informants' responses, it was found that the informants, as a group, perceived titles to be both a primary (i.e., an isolated feature) and (b) a conjoined feature (i.e., consisting of two or more associated entities where the second impacts the first). The latter being that the respondents felt that titles had a relationship with vocabulary, vocabulary in context, and logical organization.

Primary features
The results of the interviews indicated that the informants reported the title be one of 15 primary features to affect the readability of the exemplars in the rhetorics, albeit the title took a less prominent role in comparison to the first feature measured by the Lexile Readability Formula (i.e., semantic). Nine of the 10 informants (90%), for example, cited vocabulary and vocabulary in context as influential whereas 40% reported titles to be so. Conversely, the title took a more prominent role than the second feature utilized by readability formulae (i.e., syntactic-sentence length). Forty of the informants reported the title to be influential as a primary feature whereas only 10% found sentence length to be so. This surprising result can be accounted for in that the overall impact of sentence length cannot be considered only as a primary feature. Its relevance is also in relation to its conjoined properties (e.g., grammar, vocabulary, punctuation) (Dubay, 2007).

Conjoined features
The results of the interviews further evidenced that the informants found titles to be one of 10 features conjoined with other features. Here too titles played an important but smaller role than vocabulary (70%) and sentence length (50%). Only 30% percent of the respondents perceived tiles to be influential as a conjoined feature.

Titles and their relationship with vocabulary and vocabulary in context
As a conjoined feature, two features were reported to impact the amount of influence titles have on an informant's perception of difficulty: (a) vocabulary and (b) vocabulary in context. Two of the informants (20%) (Ben, Linda) explained that whether they could make out the meaning of the title of an essay helped them to determine how difficult an essay would be. Specifically, they explained that how difficult they found a title was influenced by how understandable the vocabulary in the title was. Ben offered an illustration, explaining that the vocabulary in the title of the essay he placed as easiest ("A Guide to Proper Hand-washing Technique") was easier to understand than the vocabulary in the title of the other essays. He further illustrated this by explaining that he had trouble with the conceptual meaning of the phrase freedom and security which made up the title of the essay he placed last, most difficult ("Freedom and Security"). Ben's report is in line with studies that have suggested that the meaningfulness of titles is a factor in how helpful they are (Bock, 1980;Fan & Liu, 2008;Schallert, 1976;Zhang & Hoosain, 2001).
Ben also noted that he could not initially make out the title of the "Freedom and Security" essay, but that he was later able to infer its meaning from context because of the meaning related to the terms in the text. His report is associated with literature that has purported that ELL readers rely on context clues to catch the meaning of unknown words (Cooper, 1999;Fraser, 1999;Nassaji, 2003;Paribakht & Wesche, 1999).

Titles and their relationship with logical organization
Titles were also reported to be related to the amount of difficulty caused by logical organization. Dan explained why he felt that the title of the "A Guide to Proper Hand-washing Technique" essay (the one he positioned as easiest) was helpful. He reported that the essay's title assisted him in understanding the essay's logical organization because he could anticipate the essay would be reporting a procedure. Dan's report is in accordance with Bock's (1980) work which explained that titles can help informants set up ideas about forthcoming logical organization.

Conclusions and Discussion
Accepting that readability is an important and current field of study (Mesmer, 2008), yet the readability of essay exemplars found in rhetorics is an understudied area in both L1 and L2 literature (Baker, 2019), this article explored what effects titles have on ELL apprenticing writers' perceptions of text difficulty when reading exemplars excerpted from rhetorics. The results indicated that titles impact readability as (a) a primary (isolated feature) and (b) a conjoined feature (consisting of two or more associated entities where the second effects the first, i.e., a feature influences the impact of titles or titles influence the impact of another feature).
These findings further literature which has reported the effects titles have on the readability of other types of materials (e.g., short crafted narratives, fairy tales, newspaper articles). As a primary factor, for instance, the findings corroborate Noor (2006) who reported that NNS, like NSs place a great deal of importance on the title when approaching a text. The findings also validate the views of Dubay (2007) and others who argue that the relevance of the title is related to conjoined features. The results regarding the conjoining feature vocabulary, for instance, are consistent with explorations done in both L1 (Miller, Cohen, & Wingfield, 2006) and L2 settings (Carrell, 1983;Fan & Liu;Zhang & Hoosain, 2001) that found that whether the reader can make out the meaning of the title is a good predictor of a reader's perception of text difficulty. Work with ELLs and vocabulary in context is also furthered (Cooper, 1999;Fraser, 1999;Nassaji, 2003;Paribakht & Wesche, 1999), as this study demonstrates that ELL readers rely on context clues to infer the meaning of unknown words in titles. The findings regarding the influence of titles on logical organization additionally advances the findings of Bock (1980) who reported that that titles can help readers anticipate a text's logical organization.
Taken together, the data presented in this article provides a unique contribution to the literature, as the study demonstrates how titles contribute to the readability of essays taken from rhetorics when read by ELL apprenticing writers. Specifically, it was found that that the title is an important contributing feature but one that is less influential than the two features employed in quantitative readability formulae (i.e., semantic, syntactic), and therefore the title needs to be considered when holistically thinking about readability and selecting essays from rhetorics. This result extends the work of Weaver (2000) who explains that using readability formulae and subjective criteria together reduces the risk of presenting students a seemingly appropriate text (based on the results of readability formulae) but one that they cannot read due to intervening effects of other features. The results further extend the stance taken by Gunning (2003) and others (Chall & Dale, 1995;Fry, 2002;Lexile, 2010;Meyer, 2003;Zakaluk & Samuels, 1988) that quantitative readability formulae are a good starting point for readability investigations, but that the other features formulae do not measure (e.g., titles) need to be considered in a sequential fashion: First employ quantitative readability and then qualitatively consider the influence of subject features readability formulae do not measure.
Regarding implications, the results presented in this article offer practical guidance for instructors, writing center staff, and the research community as a whole, as the teaching of writing (see Devanadera, 2018), writing centers (Paiz, 2017), and writing center self-access libraries (Baker, 2018) are becoming a welcoming commonplace in L2 settings. The data is also of use to members of the publishing industry as text selection is an ongoing concern (Baker, 2019) and reading difficulty has been cited as a factor to be considered when including exemplars in the 200 plus rhetorics published each year (Bloom, 1999). The resulting data, however, also raises additional questions which merit investigation, one of which is that in-depth discussions of other features that contribute to readability of essays found in rhetorics still need to be had (Baker, 2019). Another question related to this is a broader potential focus. That is, the title and its relation to readability has received a moderate amount of historically relevant attention in the North American context, attention which prematurely ceased in the late 1980s, and a limited amount of work in L2 studies thereafter, yet it is still an under-researched area in Asian and non-North American contexts. Thus, as this empirical study explored only one Asian context (Taiwan), but purposively provides an exhaustive literature review and detailed methodology section, it marks a starting point for replication studies and further discussions of the importance of the title outside the North American context.