Portfolio Based Language Assessment (PBLA) in Language Instruction for Newcomers to Canada (LINC) Programs: Taking Stock of Teachers' Experience

168-192


Portfolio Based Language Assessment (PBLA) in Language Instruction for Newcomers to Canada (LINC) Programs: Taking Stock of Teachers' Experience
At the time of publication, Immigration, Refugees, and Citizenship Canada (IRCC) is using a portfolio-based assessment approach in all of its funded Language Instruction for Newcomers to Canada (LINC) courses in order to 1) increase program efficiency, 2) monitor effectiveness, and 3) apply standards in task-based teaching, which are informed by the Canadian Language Benchmarks (CLB) (Chandrasekaran, 2018;Fox, 2014).The CLB are a descriptive, criterion-referenced scale of language proficiency in ESL, across 12 benchmarks or reference points, along a continuum from basic to advanced.
On the one hand, research (Fox, 2014(Fox, , 2017;;Fox et al., in press;Little, 2007) has provided evidence that portfolios (i.e., collections of a student's work over time, housed in an individual folder or binder, as evidence of learning and achievement) are primarily useful as formative assessment tools (i.e., informing teachers and learners about their ongoing learning).On the other hand, when portfolios are used for only summative purposes (i.e., as evidence of achievement), they can result in what some (e.g., Hargreaves et al., 2002), have referred to as portfolio prisons and may negatively influence teaching and learning.
Following on longitudinal studies which explored teachers' initial introduction to and use of PBLA from 2010-2014 (Fox, 2014;Fox & Fraser, 2012), the present study explores and documents LINC teachers' current use of and responses to Portfolio Based Language Assessment (PBLA).The results may help inform current thinking on this IRCC reform initiative, which has prompted considerable debate amongst language teachers and the larger language teaching community.For example, a petition on an English as a Second Language (ESL) teacher's website (Lachini, 2017) garnered over 700 signatures requesting that PBLA be withdrawn from LINC classes.From the petition statement and the comments posted on the website, it would appear that many LINC teachers are angry with PBLA.As Vanderveen (2018) points out their "current opposition to it is striking because it is widespread and strong" (p.9).The influence of assessment practices on classroom teaching and learning is generally referred to as washback (Bailey, 1996;Cheng, 2008Cheng, , 2014;;Messick, 1996) which can be positive, negative, or neutral.The reaction to PBLA, if traced to the assessment, could suggest a strong negative washback from PBLA on teaching and learning in LINC classrooms.
A decade after PBLA's initial development, it is important to take stock of what is happening at the chalkface with PBLA.In this exploratory case study, we investigated LINC teachers' accounts of and practices using PBLA for the purpose of making recommendations regarding leverage points: "points in a program -which are replete with curricular potential and need, and sites for positive intervention" (Fox, 2004, p. 1).Specifically, this study addresses the following research questions (RQ): Background: LINC and PBLA Below, we provide a brief description of the LINC Program in Canada and a general overview of portfolio use in classroom-based assessment.This is followed by a description of the use of PBLA within the Canadian language learning context highlighting its evolution in LINC programs.
LINC programs are federally funded ESL programs which aid new immigrants and refugees with their social and linguistic integration within the Canadian community (Fox, 2014;Fox & Fraser, 2012).LINC programs, in their present form, have been in place within the Canadian context since 1992, although similar federally and provincially funded programs for newcomers to Canada date back over fifty years.
In the early 1990s, Citizenship and Immigration Canada (CIC)[the federal department currently as IRCC] conducted country-wide consultations involving multiple stakeholders (teachers, learners, program administrators, immigrant service organizations and government officials) with regard to the needs of adult immigrants in Canada.One of the areas identified by the consultations was the need for greater standardization of language provision across the country.Thus, in 1993, the National Working Group on Language Benchmarks was established and by 1996, CIC published an initial Working Document (Canadian Language Benchmarks: English as a Second Language for Adults), which defined criterion-referenced standards of English proficiency for adult newcomers to Canada.The document was circulated to language teaching programs across Canada for feedback and comments.In 1998, through collaboration between federal and provincial governments, the Centre for Canadian Language Benchmarks (CCLB) was established to undertake the ongoing review and revision of the CLB.As a result, a new version of the CLB was published (Pawlikowska-Smith, 2000).
During the same period, working groups were set up by the CCLB to develop a comparable standards document in French.By 2006, the Niveaux de compétence linguistique canadiens was published to define criterion-referenced standards and inform language training programs in French as a second language (FSL) programs for adult newcomers to Canada.In 2010, in response to the call for a more standardized and regulated assessment system in LINC programs, and informed by successful experience with portfolio assessment practices in language and settlement programs for immigrants in Europe (e.g., Little, 2005Little, , 2007Little, , 2012) ) and the United States (e.g.Suárez-Orozco & Suárez-Orozco, 2009), the CIC developed and prioritized the PBLA approach.The theoretical framework informing the CLB/NCLC was updated in 2011 and subsequently, the CLB and the NCLC were revised and validated against the updated theoretical framework.

Portfolio Assessment Approaches
Within educational contexts, a portfolio is a "purposeful collection of students' work that demonstrates to students and others their efforts, progress, and achievements" (Genesee & Upshur, 1996, p.99).Along with Genesee and Upshur, Lam (2017) notes that the idea of portfolios is rooted in and derives from a number of disciplines including architecture, photography, and performing arts.Within these disciplines, portfolios "primarily serve to showcase a professional's talents and artistry via an array of exemplar works" (p.84).Within the field of education, however, Klenowski (2002, as cited in Lam, 2017) argues that portfolios are viewed as a pedagogical-cum-evaluative tool employed at the classroom level.
Some researchers (e.g., Silva, 1993) suggest that the rise of such pedagogical portfolios was a result of the call for a more contextualized form of assessment, particularly in writing (Hamp-Lyons & Condon, 2000).The impetus for more progressive, personally meaningful writing by students in classrooms, encouraged them to play a greater role in their learning process, and develop more self-awareness over time through reflection.
Portfolios were viewed as a potential means of better addressing students' individual needs (Silva, 1993) and highlighting for students where improvement could and should be targeted.
This approach in classroom-based assessment signalled a shift from the traditional forms of assessment (quizzes, tests) to alternative assessment practices (poster presentations, collaborative group projects).Despite the array of alternative assessment approaches, however, portfolio assessment has arguably become the most prominent and pervasive (Fox, 2017).The pedagogical portfolio (Davidson et al., 2018) has value not only in assessing students' achievement but also in documenting their development over time.Portfolios in language classrooms "provide a continuous record of students' language development that can be shared with others" (Genesee & Upshur, 1996, p. 99) and most importantly the students themselves, as a means of encouraging their reflection on and awareness of changes in their language over time.
The traditional view of portfolios tends to classify them by purpose, as either showcase or working portfolios (Fox, 2014;Fox & Artemeva, 2017).The former provides a locus for collecting artifacts at the end of a unit or course.As such, showcase portfolios serve summative purposes assessing "competencies in a given standard, goal or objective" (Johnson et al., 2010, p. 5).Working portfolios, however, serve formative purposes, because they are "repositories of work-in-progress" that highlight the development of students' learning over time (Fox, 2014, p. 69).In other words, they inform learners and teachers alike of what is working and what is not in the ongoing process of development.The focus is on learning -not achievement (although achievement is evident as an outcome of the process over time).Assessment in both types of portfolios requires judgment in evaluating performance and selecting what is to be assessed (Fox, 2014).However, in working portfolios, the student is directly involved in both evaluation and selection if learning is to be maximized; in showcase portfolios, it is often the teacher that selects and manages the portfolio.
Empirical studies of portfolio use have demonstrated that portfolio assessment can support a learner's development of better goal setting, increased self-awareness, and autonomy (Fox, 2014).For example, Little's (2002Little's ( , 2009) ) research regarding the European Language Portfolio, which is primarily used in classrooms for English as a second language (ESL) children, can play a fundamental role in enhancing students' awareness of their language development; increasing students' motivation level and ownership of learning; and promoting plurilingualism.Little and other researchers (Fox & Frazer, 2012;Jones, 2012;Nunes, 2004) report that portfolios can make the language learning process more transparent, provide more opportunities for creativity, promote critical thinking and "dialogic learning" between learners and their teachers and peers, and provide a productive learning experience by coherently integrating teaching and assessment.
Classrooms employing portfolio practices as pedagogy are often different from traditional teacher-centred and test-centred classrooms as they tend to be more "studentcentred, collaborative, and holistic" (Genesee & Upshur, 1996, p. 99).As Hamp-Lyons and Kroll (1996) point out, portfolio-based approaches are more longitudinal, personalized, and contextualized.Such approaches are characterized by three important stages: collection, selection and reflection (Hamp-Lyons & Condon, 2000).Explaining these stages, Lam (2017) notes that students within the collection stage are encouraged to think critically and to independently choose initial and final samples of their work.In the reflection stage, however, students are expected to choose a sample from their work that is the best representation of their competence, ability, or proficiency.
In sum, a pedagogical approach to portfolio assessment emphasizes Assessment for Learning (AfL) (or formative assessment) -operationalized through a working portfolio.Most scholars who are interested in increasing student learning would see this as the primary function of assessment in the classroom (Black & Wiliam, 1998;Cooper & Cowie, 2010)."AfL activities provide teachers with a window on student thinking, knowledge, understanding, and skill; and provide students with a primary resource for their learning" (Fox, et al., in press).The focus is on the process of learning and interventions during the process to improve overall learning.The student is an active participant in the development of the portfolio, takes ownership of its contents, uses it to reflect on and improve learning.Conversely, showcase portfolios place the emphasis on achievement, and use of summative or Assessment of Learning (AoL) approaches to demonstrate what a student knows and can do.Often, teachers select what will be included in a portfolio as evidence of the student's best work.The focus is on showing evidence of the outcomes or products of learning.

PBLA in the LINC Program (2010-2020)
From its initial conceptualization a decade ago, the use of a portfolio approach in the LINC program seems to have evolved from an emphasis on the development of individual learner's goal setting, self-reflection, increased self-awareness, and autonomy (characteristics of AfL/formative approaches and working portfolios) to its current emphasis, namely, addressing perceived deficiencies in the provision of language instruction in LINC (Fox, 2014).These deficiencies included the need for: standards-based, CLB-informed assessment, "curriculum, teaching materials, and guidelines for assessing learners' progress" (Chandrasekaran, 2018, p. 38); increased consistency across LINCoffered programs and classrooms; and systematic assessment evidence to improve monitoring and accountability (CIC, 2013).This emphasis calls for a showcase portfolio approach with evidence of achievement derived from summative/AoL.
Looking back, the literature supporting the initial implementation of PBLA explains the emphasis on formative/AfL (Pettis, 2014).Informed by a rich and extensive review of classroom-based research (Leahy et al., 2005), Pettis identified five main AfL strategies as the basis for the development and implementation of PBLA within the Canadian LINC context (see Pettis, 2014, p.15 -18 for a thorough explanation of the AfL emphasis intended for PBLA).Holmes (2015) discusses these AfL strategies, indicating that the PBLA was intended to: 1) clarify learning intents and criteria for success; 2) incorporate classroom activities that elicit evidence of learning; 3) provide action-oriented feedback that moves learners forward; 4) activate learners to become instructional resources for one another; and 5) activate learners to become owners of their learning.
However, the evolution of PBLA intent suggests less of a focus on the support of learning and more emphasis on accountability.
We speculate that changes in the overall purpose of the portfolio approach in LINC classes may have been prompted in part by the 2010 report, Evaluation of the language instruction for newcomers to Canada (LINC) program (CIC, 2010).This report evaluated the LINC program based on interviews with 24 key informants involved in program implementation and operation (e.g., CIC regional officers, provincial representatives), and a large-scale, cross-Canada survey of LINC stakeholders (e.g., administrators, teachers, students) that drew responses from 141 LINC-providing organizations, with responses from 139 Administrators, 651 LINC learners, and 56 LINC teachers.
There were several encouraging findings in the report.For example, although it identified variability in modes of delivery and program emphasis across LINC-providing organizations, it related this to the need for a reasonable requirement for flexibility within local and regional contexts.Further, it reported that the "use of the Canadian Language Benchmarks (CLB) is often cited as an important issue in the field of ESL by language experts" (p.16), and results of the Administrator's survey (N=139) indicated that "the great majority of LINC providers use the CLB for each of five purposes" (p.16), namely: intake assessment, ongoing assessment, exit assessment, developing curricula, and developing learning materials.The report stressed the variability inherent at the level of the LINC classroom was due in large part to continuous intake and differing assessment practices.Further, there were a large number of multilevel classes: " … of the classes surveyed … a quarter had students at two LINC levels, virtually all one level apart" (p.19).Multilevel classes are generally more difficult to teach as they require more planning for groups with different capabilities, which nonetheless share the same classroom and the same teacher.Multilevel classes with continuous intake and fluctuations in attendance, dramatically increase the challenges faced by teachers.
In addition, across the interviews with the 24 key informants, the following weaknesses of relevance to the PBLA initiative were highlighted: 1) the lack of progress and exit tests; 2) the relationship between LINC levels (which relate to commonly used curricular documents) and CLB criteria; 3) the lack of clear developmental milestones which, the informants argued, "would improve clarity for learners and could motivate them to complete more levels" (p.62).The recommendations emphasized the importance of developing a standardized language test that could enable officials to accurately measure language proficiency at key milestones of learners' development.In addition, the 2010 LINC evaluation report underscored the importance of developing a uniform/standardized approach for assessing newcomers' needs while allowing for CIC to track learners' progress against intended program outcomes.
By the time PBLA was fully implemented across the LINC program in Canada (over three years from 2014-2017), messaging from the government had changed dramatically.For example, a CIC operational Bulletin in 2013 stated that PBLA was "being introduced to address the need for a standardized in-class language assessment protocol" (CIC, 2013).This emphasis clearly departed from the AfL strategies envisioned at PBLA's inception.

The Washback of PBLA on Teaching and Learning In LINC
Despite the use of portfolio approaches in language teaching across the world and the well-documented potential effectiveness of portfolios in promoting and enhancing language learning, PBLA practices were novel to many Canadian LINC teachers (Fox, 2014;Ripley, 2012).In addition, as noted by Fox (2014), research exploring the PBLA approach in LINC had been largely within the context of initial field testing and implementation (Fox & Frazer, 2012).By 2016, PBLA was being implemented in all LINC programs across Canada and had evolved into a large-scale top-down assessment reform initiative promoting the use of the CLB, task-based instruction, standardization, and accountability (Fox, et al., in press).
Over the years, a number of researchers (Fox & Fraser, 2012;Fox, 2014;Desyatova, 2017) have investigated the washback of PBLA.Ripley (2012) and Fox (2104) reported that their teacher participants recognized both benefits and challenges in relation to the PBLA implementation.On the one hand, they reported that PBLA had promoted the employment and development of rubrics in student assessment (Ripley, 2012); increased planning and awareness of assessment; and helped to clarify teachers' articulation of goals (Fox, 2014).On the other hand, the physical portfolio itself (initially known as the Language Companion) and teachers' preoccupation within its organization had distracted teachers from opportunities promoting learner engagement and autonomy.In large part, teachers were preoccupied with unit/module outcomes, summative assessment, and rubrics (Ripley, 2012).
Furthermore, Fox (2014) and Ripley (2012) argued that the formative potential of the PBLA was limited because students were not involved in selecting their own artifacts to include in their portfolios due to the increased emphasis on the summative assessment.Fox (2014) and Lachini (2017) further note that the increased emphasis on summative assessment brought about further demands on teachers and excessive workloads, contributing to a stressful and unhealthy working environment.The excessive workload was also brought about by the lack of resources and issues with using the CLB as an index of proficiency (Fox, 2014).Vanderveen (2018) further noted the drawbacks of PBLA as being "costlier, more time-consuming" (p.5), and with more teacher pushback and outrage.
In a similar vein, according to Chandrasekaran (2018), rubrics and requirements for the collection of artifacts began to far outweigh the laudable, overall goals of portfolio assessment, such as fostering learner autonomy, reflection, self-awareness, and engagement.She suggests that both the initial roll-out emphasis of the PBLA (Pettis, 2010) and the 2014 PBLA Guidelines (Pettis, 2014) may have contributed to a shift in teachers' attention which ultimately had negative consequences.She explains, for example, that Pettis stated that eight artifacts per skill (i.e.32 artifacts total) were required in order for a teacher to make an informed decision about an individual student's CLB skill levels.Consequently, teachers focused most of their attention on the generation of artifacts, through the development of assessment tasks and concomitant rubrics, to meet these suggested portfolio requirements.However, teachers also began to complain that they were 'testing not teaching', and this fueled negative response, raised questions about the true purpose of PBLA (Chandrasekaran, 2018;Fox, 2014;Lachini, 2017;Morrissey 2016Morrissey , 2018)).Negative responses to PBLA may also have been exacerbated by "policymakers' countermeasures to enforce PBLA use, rather than addressing evidence arising from classroom experience, teachers' concerns, and feedback from the chalkface" (Chandrasekaran, 2018, p. 8).
In response to the evidence of negative washback, many recommendations were put forward by several researchers including Desyatova (2017), Fox (2014), andRipley (2012) to help support the PBLA initiative and its implementation within the LINC program.These recommendations include: • Clarifying the purpose of the Language Companion binder; • Creating a method for integrating PBLA into the curricula; • Offering continuing support to teachers on implementing PBLA; • Providing resources and materials for teachers • Fostering student involvement; • Rethinking and rearticulating the purpose of portfolios in order to prevent it from becoming just a "bulky file of student work" (Fox, 2014, p. 81); • Realizing the PBLA potential as a "dynamic mediating support for learning" (Fox, 2014, p. 81); and • Acknowledgement of the practical challenges facing teachers.
In the present study teachers' accounts of the PBLA initiative were considered because, as noted by Jones and Egley (2004), they are on the "frontlines and in the best position to help policymakers understand" (p.4) how policies are affecting teaching and learning.

Method
In order to 1) explore teachers' accounts for evidence of washback from PBLA on teaching and learning in LINC programs, and 2) identify potential leverage points which might be responsive to intervention and support, this comparative qualitative case study took a developmental evaluation approach (Patton, 2012) by comparing earlier (baseline) published and publicly available findings regarding the initial implementation of PBLA (Fox, 2014;Fox & Fraser, 2012;Ripley, 2012) with the current situation.Only published research was used for the comparison.Patton (2012) defines developmental evaluation as "improvement-oriented, formative evaluation [which] focuses on making an intervention or model better" (p.127).

Participants
The participants in the current study were recruited by, 1. Posting research posters on university bulletin boards; 2. Sending emails to potential participants; 3. Snowballing, wherein LINC teachers informed other LINC teacher colleagues of the study.
Subsequent to the approval of the university's Research Ethics Board (REB), seven participants, former and current LINC teachers, were recruited from programs across the province (Ontario).Individuals who were interested in participating in the study contacted the researchers.Teacher participants (n = 7) were identified by applying a purposive/judgemental sampling technique (Tashakkori & Teddlie, 2003) with three selection criteria: they engaged in post-graduate and graduate study to inform their teaching; they had the required certification from the Province of Ontario (TESL Ontario) and/or TESL Canada in order to teach in LINC programs, and they were from different geographical locations within the research context.In addition, variations in participants' ages and teaching experience were also taken into consideration.Five of the participants were enrolled in, or had graduated from, a university's Teaching English as a Second Language (TESL) Program, or were in master's or doctoral programs in Applied Linguistics with TESL specializations.Three among them were observing LINC classes as part of their research for their degrees and were working with LINC teachers who were using PBLA.All of the participants had previously taught ESL in other contexts.

Instruments and Procedures
Data were elicited using one-on-one, semi-structured interviews.The participants were asked a series of pre-determined, open-ended questions (Appendix A) about their current teaching practices, their accounts and beliefs about the influence of PBLA on their teaching, and if they had made any changes in their approach to teaching as a result of the IRCC initiative.Some participants were recruited for two interviews over the course of the project, which corresponded to two points in the academic year (Fall and Winter; Winter and Summer).The purpose of the follow-up interviews was to take stock of any changes in participants' responses to PBLA over time.
Each interview session took approximately 30 minutes and was audio recorded for analysis.During the interviews, the researchers took "field notes" and recorded comments of interest in relation to the research questions, or personal observations which were of relevance to the study's purpose.

Analysis
A modified constructivist grounded theory approach (Charmaz, 2006;Mills, et al., 2006), was applied to the microanalysis of the teachers' responses.Following Woods' (1996) procedures for analysis of teachers' accounts of their decision-making in the classroom, we considered: the frequency of issues occurring, explicit mentioning, and tone of voice (which signalled highly loaded issues).All the interviews were transcribed and coded.In accordance with Charmaz (2006), the following sequence of coding practices was followed: 1. Open Coding: first opening up of the data.2. Axial Coding: positioning the data collectively in novel modes.3. Selective Coding: analytically unfolding categories to identify essential trends that are surfacing.
We developed a coding scheme for two transcribed interviews, identifying themes, and creating coding categories within the themes.Any disagreements in coding between the two researchers were settled by a third person who had not originally coded the responses.After coding the responses, and in accordance with Jones and Egley (2004) we re-analyzed the coding categories and re-read the responses to rule out redundancy and overlap.

Results and Discussion
As noted above, in the present study washback was viewed as the effects of assessment practices on teaching and learning.Further, following Spratt (2005), washback effects can be "accidental and intentional … positive or negative" (p.8).Therefore, any effects evident in the participating teachers' accounts of PBLA, whether neutral, positive or negative, intended or unintended, were considered to be indications of washback.Findings are discussed in relation to the literature that framed this study, guided by the research questions.

Theme 1: Washback of PBLA on Teachers and Teaching
In response to research question one, the LINC teachers' accounts indicated that the effect of the PBLA initiative on instructional practices was far-reaching for some teachers, but not for all.PBLA appeared to have a negative washback effect on some instructional practices, as it encouraged teachers to adopt approaches they would not otherwise have adopted.Some LINC Teachers stated that they devoted almost all of their teaching time to "teaching to the test".TA: They tell us to not teach to the test but … we actually … teach to the test.
In addition to devoting more time to assessment rather than teaching, they emphasized that they tended to teach and test language skills in isolation more often as a result of PBLA (rather than integrating them), because of the skill division that was defined in the 2014 PBLA Guidelines (i.e., eight artifacts per language skill).As one teacher put it: TA: … where all this testing is going to lead us.Why can't we just do it as before… [PBLA] creates an imbalance amongst the skills.
Four teacher participants argued the skills-based approach had the effect of narrowing the curriculum, because the required focus on skill-based testing dominated their approach to teaching and, in their view, marginalized language learning.Ironically, the emphasis on summative (AoL) rather than formative (AfL) purposes for PBLA meant that teachers viewed it as distinct from teaching and learning practices in the classroom: TA: I am wasting a lot of time testing, … [PBLA] is test-oriented, even when planning to teach something I think about how I can test this, what competencies shall I be testing?Everything is geared towards what is to be assessed rather than seeing how the students are doing or learning.
To varying degrees, there was evidence in the accounts of the teachers who participated in this study, that PBLA may be undermining their spontaneity and creativity.Four teachers reported becoming more like "technicians" (Hargreaves, 1994), in that their fundamental love of supporting learning through ongoing, dynamic teaching was replaced by the need for ongoing testing and accountability.This, in turn, reduced their enjoyment of the teaching and learning experience (Polesel et al., 2012), as evidenced by the comment of one participant: TA: They come to classes to get by in the community, … why do they need to be tested?Now it's become like a school for a non-credited program you test them for everything you do.You teach a task you test it.
Five of the LINC teachers reported increased feelings of stress, frustration, and dissatisfaction as a result of its use.In particular, increasing stress had caused consequential health issues for two teachers.Essentially, the PBLA for these LINC teachers was "the ferocious master of the educational process, not the compliant servant [it] should be" (Madaus, 1989, p.85).They forcefully argued that the PBLA initiative had not sufficiently considered the diversity of the student population, or institutional factors such as class size, continuous intake, differing proficiency levels (e.g., the capability of level 3 students versus level 5 students), and the frequency of multi-levels within one class.
All seven LINC teacher participants reported that continuous intake and fluctuations in attendance made PBLA needs assessment and learner-centred long-term planning components almost impossible to implement.Furthermore, four teachers reported that learners' previous learning cultures and school experiences caused challenges in promoting learner autonomy and agency because it was difficult to move their "conception of learning beyond rote memorization of facts and measures, to learning as an adventurous, engaging experience … [and] shifts the passive learner role to that of active enquirer" (Abdulhamid, 2018, p. 295).Some LINC teacher participants (n = 6) reported that it was difficult to promote student autonomy, especially with lower-proficiency language learners because they did not have the language to articulate their needs.
All participants agreed that the PBLA initiative seemed to have employed a broad notion of a one-size-fits-all portfolio assessment model that neglected a number of sociocultural factors.Such factors, as Hamp-Lyons, (2007); Klenowski (cited in Lam, 2017), and Lam (2013) point out can hinder teachers' use of innovative portfolio assessment practices.One teacher reported: TD: During the initial roll-out of PBLA … the experience was more of a stressful one for me … to figure out what the requirements of me (as a teacher) were and how to implement them in my classroom.
Another recounted, TB: … the students found it difficult to understand what I wanted them to try and do … they did not have the language to clearly communicate their goals … I remember sitting down at the end of the class and saying to them: I understand that you are frustrated.I know that you understand there is something I want you to do, and I understand that you're not able to do it and accept that because it's not your fault.I just remember that sense that didn't work.

TB further commented:
As a teacher, I feel like I am serving two taskmasters.One being the students and the other being the government expectations and the PBLA.
The remarks are consistent with those of another teacher: TE: It's a struggle, to create assessment with different levels and classes.
Four teachers reported that because of the stakes associated with the PBLA and the collection of eight artifacts per skill (32 artifacts in total), they felt a great deal of pressure which could lead to a "diminished sense of professional worth and feeling of disempowerment and alienation" (Blazer, 2011, p.6).The LINC teachers' accounts suggested that PBLA's demands forcing teachers to organize skill-based lessons, develop assessment tasks, mark assessment tasks and provide continuous feedback took up considerable out-of-class unpaid time.In addition, all teachers reported that the expectations of the PBLA initiative had generated multiple roles that increased teacher stress, frustration leading to an unhealthy working environment.LINC teachers reported being needs analysts, curriculum and materials/task developers, and above all assessors and assessment task developers.In essence, the information provided by our participants supports Chandrasekaran's (2018) observation, that the PBLA initiative expected teachers to "make the road by walking" (p.69) without sufficient time or support of human and material resources.The following excerpts from interview data reflect the anger and frustration similar to that reported by Chandrasekaran: TA: PBLA is taking so much time and we are not respected for that.They aren't willing to pay us for our prep time.… In a mixed level class, it's not possible to do what the PBLA is asking us to do.TB: Absolutely, absolutely without any doubt, PBLA has increased my workload.Trying to meet the expectations … about the number of tests we have to put in the portfolio.Having assessments … tailored to CLB levels, to integrating students in the class and completing the About Me section … without any doubt … has increased the workload significantly.
TC: [PBLA] is disempowering and devaluing teachers … who are in the frontlines of enabling our immigrant population to integrate into our society … who are teaching the students how to speak the language are under-resourced, overworked… have no infrastructures, or training to rely on … can't be expected to do their work with continuous intake … expected to be therapist, housing expert … all manner of things for a very vulnerable population.
These findings mirror Lachini's (2017) explanation for LINC teachers' resistance to the PBLA and continuous intake: increased focus on assessment rather than teaching; teachers' demotivation; increased workload and stressful working environments.
Although the LINC teachers indicated that the PBLA initiative was not taking the LINC program in the right direction, they agreed that accountability was important.Some identified other forms of accountability, which they felt would be more fair, ethical and effective (such as external, validated tests at milestone or transition points).This is a noteworthy observation since, as with Jones and Egley (2004), none of the respondents reported they were against accountability; they clearly understood the importance of accountability in the profession.Note for example: TC: The goals of PBLA were good … a formative purpose.But … I don't agree it is … most of the teachers would believe it's an accountability system.… Accountability is important, and I understand why the government wants it, but they could use other accountability measures.
Another suggested, TA: … accountability it could have been gotten in many other ways.We could have given them reports about each student and their progress in a clear picture.
All participants expressed negative sentiments towards the PBLA due to their lack of assessment training and experience.They mentioned one or more of the following as time-consuming, challenging, and difficult: setting CLB-related tasks for multilevel classes; developing and aligning rubrics with the CLB criteria; identifying learning outcomes; aligning portfolio assessment tasks to CLB requirements; and, identifying alternative assessment approaches.They all emphasized that their limited knowledge in constructing assessment tasks meant they had little confidence the tasks were at the correct CLB level or were representative and accurate indicators of students' progress.The participants further expressed concern about the quality of their portfolio assessment practices and indicated that implementing portfolio assessment required advanced knowledge and practical skills they had not developed as part of their language teaching certification: TA: Testing is tough, and marking those papers takes up so much of my personal time.… I spent three hours marking papers.… I believe the samples I have collected from the students are not a true representation of the students' levels.
TB: It is very confusing.… With PBLA very few teachers feel comfortable, competent, [or] confident in making assessment tasks.Even with having a background in psychology, testing is a problem.TE: Even after 10 years with PBLA, I still struggle and find rubrics difficult.I am still learning.
Hamp-Lyons (2007) and Sweygers et al. (2009) argue that teachers who are assessors require specific assessment training, but portfolio assessment requires particular knowledge and skills in providing feedback, supporting students' reflection on learning, encouraging selfawareness, or identifying next steps.
Despite the negative comments reported above, four teachers believed that the implementation of the PBLA also had positive washback by bringing more structure to their classes and increasing their orientation towards task-based teaching.In keeping with Fox's (2014) findings, three of the teachers reported that they spent more time explaining their learning goals -explaining why they wanted their students to do an activity -and, better aligning their activities with the CLB.
TA: … I have learnt through PBLA that these students come from different cultures and they want to see structure, and with PBLA I am able to explain my outcomes, what I am doing, and for what reason.
TB: PBLA has made me more organized in delivering my lessons.… because of PBLA, I am more task-based in my approach, and much more consistent in my attempt to evaluate students' progress and setting learning outcomes.TD: Because of PBLA I have more structure in my classes, a much more academic orientation than before.
Unlike other participants, one teacher reported no additional anxiety or stress as a result of PBLA: TB: … my class runs well, I … accommodate and negotiate several aspects of PBLA in my classroom.… If we do a task and I find most of the students have not been able to successfully complete the task, I will admit … it is probably my failure and not theirs … making sure they feel good about themselves.This difference in teachers' reactions may be due to their working conditions.Whereas two participants had job security and benefits, the others did not.These two tended to be more positive on the whole, as their comments suggest: TB: … absolutely my job security plays a role in me not sounding as frustrated as other teachers.I have the luxury of the number of hours in my contract.If something doesn't get done, it doesn't get done.If an IRCC inspector wants to come in and I have a student who has an incomplete About Me section, I will just defend it … many LINC teachers don't have this option.TD: I am … relaxed … stuck here alone with one class and seven levels, I don't stress about what gets done.What gets done, gets done.This supports Green's (2007) washback model that presents washback in relation to the context of assessment, which is "realized through and limited by … participant characteristics" (p.25).In essence, consideration of context moves the model of washback from "a recipe for achieving positive washback, towards a descriptive and partially explanatory tool addressing what goes on in order to cause the various washback effects" (Saville, 2009, p.32).

Theme 2: Washback Effects of PBLA on Learners and Learning
As for learners, five teachers reported that their students were so distracted by their grades that they ignored the formative potential of portfolios assessment: TA: The students are also nervous because every time I say assessment, they get stressed and ask how many tests, especially Level 3 students, they are not happy, it's really stressful … It's just about scores, they just worry about scores and do not read the feedback.I try not to have scores and just have comments, but they say how many marks did I get and start to get angry.This is what they worry about.TC: PBLA contributes to an atmosphere of infantilization of adults' feelings…The way it's used now is like a kindergarten project.I think PBLA isn't fair for students with literacy issues.I think it unfair for the government to link PBLA and learning with meeting the citizenship requirements.TE: The learning is gone; it is not a learning process anymore… The minute you say assessment, and everything gets stressed.
TF: In the mixed-level class the students were lost and didn't know what they were doing.
This finding is consistent with Klenowski (2002) and Murphy (1997) who argued that students may not value formative feedback when the stakes are high, and portfolios are used for grading.Essentially, the PBLA in some LINC programs may have "become another form of testing for teachers and a means to achieve better grades for students" (Lam & Lee, 2009, p. 58).
The reports of the teachers, however, are dramatically different and add support to evidence that teachers play a critical role in mediating washback effects on learners (Shohamy et al., 1996).The teacher who did not experience feelings of anxiety or stress (TB), for example, commented that collecting portfolio evidence helped learners understand that language learning is a process that takes time.
Negative washback on learners and learning also manifested itself as "cheating", which in turn had inflated students' scores and increased confusion over their actual ability.It had also prompted increased competition.Students used their assessment results as a "means of comparing" themselves with others, as indicators of whether they were "smarter", or necessarily "dumber" (Marchant, 2004, p. 2).TA expressed: The real problem that I have in my class is cheating, I have 28, and my students cheat.These are adults who cheat… The cheating was not evident before in LINC classes.
As the PBLA stakes become more "consequential for students", the pressure to do well may increase correspondingly (Paris, 2000, p.3).According to Abdulhamid (2018), this may be an important factor in understanding the washback of the PBLA on LINC teaching and learning.
The participants reported that their students had difficulty understanding the purposes, assessment procedures and benefits of portfolio-based learning: TA: The students are not interested and with the time we have there is no chance to get students engaged … TC: … it was more of a hindrance than a help.… Students had no sense of ownership over the portfolio.… It was more of an obligation, something they did in class.
The participants also reported that their students' lack of understanding of the formative purposes of portfolio-based learning played a role in making the selection of artifacts "heavily teacher-directed" (Chandrasekaran, 2018, p. 2).TF remarked that when she tried to explain the benefits of student autonomy in selecting the assignments to be included in portfolios, her students objected.Although she persisted in her attempts to increase her students' self-reflection and self-assessment, they argued, "if I can learn on my own, why am I here?"Similarly, TA emphasized the importance of students' involvement in and responsibility for their learning.She recounted a strategy that had worked for her over time.Before an activity, she explained why she was asking her students to undertake it and responded to their questions about its purpose.After completing the activity, she reminded them of her intended purpose and asked for their feedback -either in class discussion or in anonymous questionnaires or written reflections, which became a source for class discussion.Eventually, some students became vocal advocates of increased engagement in, and awareness of their learning.

Leverage Points and Possible Future Interventions
Leverage points, as noted above, are "sites for positive intervention" and renewal which are evident in the discursive accounts of key stakeholders (Fox, 2004, p. 1).Leverage points were evident in the accounts of the LINC teachers participating in this study.In response to research question two, we have selected three key leverage points and suggested potential interventions to address them.Addressing other significant leverage points (e.g., issues relating to continuous intake, lack of job security) would require substantive changes to the administrative structure of the LINC program and are beyond the scope of this discussion.

Leverage point 1: Increased Teacher Workload Due to Assessment-Driven Teaching for Accountability Purposes
By nature, portfolios "lack standardization" (Mudzingwa, 2017, p. 16).PBLA currently requires teachers to plan, develop, and administer their own variable assessment tasks in order to evaluate their own unique and varying students.In a LINC program context, the use of the PBLA for standardization and accountability is "neither achievable nor desirable" and undermines "the ability to tailor portfolios to the needs of the target group" (Sweygers et.al, 2009, p. 101).We argue that while accountability cannot be the primary purpose for the PBLA, it has clear potential as a formative assessment resource.Further, positive washback from the PBLA on teachers' use of the CLB, namely, taskbased teaching, and increased planning related to explicit learning outcomes was evident in the teachers' accounts.
IRCC could take other approaches to achieve accountability and track progress in relation to intended outcomes within LINC programs.One option is to develop standardized, milestone assessments as recommended by the LINC Evaluation Report (2010), for use at key transition points in a program.This would greatly lessen the increased, assessment-driven workload the teachers attribute to the PBLA.Milestone assessments should be developed by assessment specialists, validated, and monitored.They would provide IRCC with information regarding standards and accountability, as needed, and provide teachers and programs with reliable information about the relative gains in proficiency of their students over time.If teachers actively collaborate with technical experts in the development of milestone assessment tasks and participate on an ongoing basis in marking milestone tasks, then the assessments can serve as professional development opportunities and support teachers' increased assessment awareness and capability.

Leverage point 2. Concerns Over Teacher Professional Worth, Resources, and Training
As recommended by Chandrasekaran (2018), the PBLA initiative ought to take a dialectic approach (Connelly & Clandinin, 1988) where teachers are given the flexibility to adapt the PBLA in their classrooms based on their context and learners' needs.In essence, we argue for a fundamental need for "rethinking and rearticulating the purposes for portfolio assessment in LINC, to prevent it from becoming just a bulky file of student work, and to realize its potential as a dynamic mediating support for learning" (Fox, 2014, p. 81).
To increase the positive potential of the PBLA, training in its use as a formative assessment approach is essential for LINC teachers who are working within such dynamic and complex teaching contexts.Specific training should address the issue of student resistance to teaching practices which differ from those experienced in other educational cultures (point 3, below).Further, assessment courses, as argued by Coombe et al. (2012), ought to be a focus of language teacher education programs and a certification requirement (TESL Ontario and CTESL Certification).
In accordance with Chandrasekaran (2018) and Holmes (2015), we further call for local and online spaces that offer teachers opportunities to reflect on and exchange novel and successful practices.LINC teachers should be encouraged to articulate their own needs and then offered training opportunities that directly address them.Drawing on and responding to teachers' expressed needs reinforces the value of their professional judgment, enhances their professional development, and will ultimately improve the quality of their teaching.

Leverage point 3. Learner Agency and Engagement
Teachers employing PBLA in their classrooms are expected to advocate and promote self and peer assessment (Yang, 2003).Without learner engagement within the current LINC programs, the PBLA initiative may be offering limited to no opportunities for language learners to (1) be fully involved in planning; (2) "tap into their metacognitive skills and knowledge" of goal setting; and (3) "retrieve, adapt, and deploy prior goalmaking strategies when developing language-learning goals" (Mudzingwa, 2017, p. 19).Teachers need to be supported in the development of effective approaches that address student resistance to what, for most students, is a dramatic shift in educational culture.A promising outcome of this study is the reports of teachers regarding strategies they have used to address such resistance, like the one reported above.These need to be formally collected, expanded, and circulated to all teachers.A portfolio can be an important formative assessment resource, when: • Artifacts of student performance are jointly selected by teachers and students; • Work and assignments are reviewed by students and teachers, over time, as means of informing teaching and learning; and • Students understand and can explain the purpose of what they are doing.
Portfolio collections could continue to have required sections, but in order to support student agency and engagement, evidence accumulated within them should be flexible and responsive to an individual students' level of language proficiency, interests, and needs.If students understand that at key transition points within a program their portfolios will serve as critical triangulating evidence of their proficiency alongside milestone test performance, they will invest more effort in them.

Conclusion
Although portfolio assessment is not new in L2 learning contexts, as evidenced by the data presented here, the PBLA implementation remains "a thorny issue" (Lam, 2017, p. 92) for LINC teachers.Overall, the above findings provide evidence of washback.However, teachers' accounts of the PBLA was highly contextual; the working context determined the direction and intensity of the PBLA washback they reported.
Besides, the data presented here is based on teachers' self-report of their classroom events.We drew no evidence from classroom observations.As Breen et al., (2001) put it, teachers, subconsciously or not, "… may wish to promote a particular image of themselves … there is often a difference between espoused theory (theory claimed by a participant) and theory in action (what a participant does in the classroom)" (p.345).Furthermore, this study involved only seven participants in one particular context, as such, the results cannot be generalized to other contexts.However, the results of the study point to evidence of the washback of the PBLA on teaching and learning in the classrooms of seven highly trained and experienced LINC teachers.It suggests leverage points and potential interventions that may improve the quality of teaching and learning with the larger community of ESL/LINC teachers in Canada and highlights the need for further research.

RQ 1 .
How do LINC teachers account for the use of PBLA in their LINC classrooms?Is there evidence of washback?If so, of what kind?RQ 2. What leverage points are evident in the accounts of LINC teachers working with PBLA?What interventions might enhance the quality and effectiveness of teaching and learning?
Note.B.A. = Bachelor of Arts degree, M.A. = Master of Arts in Applied Linguistics with TESL specializations, TA, TB = abbreviated pseudonyms for Teacher A, Teacher B and so forth Table 1 below summarizes the seven teachers' background information and classroom characteristics collected from teachers' interview responses.