Integrating translanguaging into assessment: students’ responses and perceptions

2.1 Multilingualism in language assessment

One of the most influential SLA theories that informs a communicative approach to language teaching is the Output Hypothesis (Swain 1985). The Output Hypothesis asserts that language learners must be pushed to produce output in the L2 so that they can develop the desired lexical and grammatical accuracy. Students’ L2 output represents the outcome of their L2 learning. A historically entrenched emphasis on monolingual (i.e., target language) output is arguably based on the assumption that learners have the same goal for learning a new language as for learning and using their L1 – to use it on par or close to the L1 standard. From an assessment perspective, this means that students’ output proficiency is judged against benchmarks (such as the Common European Framework of Reference [CEFR] or the ACTFL standards) that include L1 or near L1 proficiency as a top-tier goal. That is, the construct of communicative competence (see, e.g., Canale 1983; Canale and Swain 1980) informs the facets of proficiency that we might wish to see and examine in a given communicative test, and that construct traditionally focuses on aspects of the target language without the influence of L1. Leung and Scarino (2016), however, highlighted the importance of reconceptualising the goals and outcomes of learning an L2. They emphasised recognising the diversity of goals that people from different backgrounds learning languages might have in diverse settings and who may be interested in developing different capabilities and achieving different outcomes.

This paper follows Shohamy (2011), who advocated for a recognition of multilingualism as a construct in language testing. This perspective challenges the traditional view of language as a “monolingual, homogenous, and often still native-like construct” (Shohamy 2011: 419). By integrating a heteroglossic language ideology within language assessments, we are able to view dynamic forms of languaging as an asset for elicited performances in assessments while also being attuned to engagement with languages (Schissel et al. 2019). Another purpose for incorporating multilingual constructs into language testing is to recognise and reflect a fuller linguistic and semiotic repertoire for every individual learner, including those at the beginning stage of learning a new language. It is with this in mind that translanguaging is defined as “the deployment of a speaker’s full linguistic repertoire without regard for watchful adherence to the socially and politically defined boundaries of named languages” (Otheguy et al. 2015: 283). Furthermore, Toohey (2019: 937) pointed out that “knowledge/knowing and language/languaging are relational, processual, and entangled”. Language educators and researchers need to reorient their attention to recognise that the boundaries between named languages are not rigid and abandon their efforts in maintaining a static, binary, and instrumentalist view of language learning. Toohey (2019: 950) also argued that L2 teaching needed to take account of new insights and practices that would allow L2 learners to “participate (perform) before they were competent”. This paper builds on the premise that it is time for us to integrate a broader model of language use into language testing to unlock the learning potential of all learners, especially at the initial stage of learning.

Taking assessment into account will ensure that translanguaging becomes and remains a sustainable framework that teachers can confidently implement throughout the entire learning process (Schissel et al. 2021). Translanguaged multimodal assessment opens spaces for meaningful communication by stimulating students’ interest and participation in completing meaningful tasks. If we were to embrace a translingual framework as the foundation of the entire language learning process, we need to extend our attention to its roles in designing and implementing assessments and push our efforts beyond the current theoretical and pedagogical ‘only target language’ basis. We suggest that designing writing assessment tasks through a translanguaging lens will enable students to demonstrate not only what the new language knowledge means to them individually, but also to connect the new linguistic forms with their existing linguistic repertoires. It also enables test developers to assess more broadly what students know and can do both conceptually and linguistically without limits imposed by external societal definitions of the standard. However, although an increasing number of studies have pointed out the potential of using translanguaging in assessment, García and Li (2014: 133) found that “these tests have not been developed”, at least not as fully as we might like, for L2 assessment.

Some recent studies have shown that the translanguaging approach can be integrated successfully into assessment design. For example, in the linguistically diverse state of Oaxaca in Mexico, Schissel et al. (2021) explored a translanguaging assessment design through an action research case study with 40 pre-service language teachers. The study took into consideration the weightings of the translanguaged test and drew on a rubric to ensure multilingual students could be assessed equitably and fairly. In another study in U.S. public schools, Lopez et al. (2017) presented a digital classroom assessment system that allowed late-arriving emergent bilingual students to demonstrate their content knowledge and skills by drawing on their entire linguistic repertoires. The latter study also discussed how test developers and teachers could help mediate the translanguaging mode in classroom assessment. Lopez et al. used a computer-based testing platform to integrate translanguaging into their assessment design, and this seems to have offered a practical and promising way to further advance efforts in promoting multilingualism in a world where pen-and-paper testing may be replaced by computer or cloud-based testing in the foreseeable future. Shohamy and Pennycook (2019: 36) also suggested that language tests should incorporate not only multilingualism but also “multimodal and multisensorial assemblances” in order to capture the full scope of resources that students use to engage in real-life communication, which is frequently digital and highly mobile due to the rapid development of technology. The need for creative solutions to the traditional notions of testing language proficiency has become more pressing lately, especially since many educators and institutions are more open to digital transformations after two years’ remote emergency teaching as a consequence of the pandemic.

2.2 Creativity in assessment design

In a special issue on creativity in language learning and teaching, guest editor Rodney Jones (2020) incorporated the notion of translanguaging in moving our creative efforts forward. He pointed out that previous endeavours and considerations regarding creativity in language education are confined within the framework of dominant monolingual ideologies. In an earlier publication, Jones (2018) argued that creativity resides in an honest engagement with the ‘messiness’ of most situations in which people are trying to learn new knowledge or make meanings. According to Jones (2018: 82), messy creativity is an acknowledgment of the ultimate nature of language as a messy system, in which “contradictions, contingency, and indeterminacy” are the norm. Specifically, in applied linguistics, Jones (2018: 82) described messy creativity as

a kind of creativity that shakes us out of our comfortable assumptions about the way applied linguistics ought to be carried out, and challenges us to develop ‘messy’ methods to confront this messy creativity, methods that go beyond trying to ‘make sense’ of it through traditional conceptual categories and attempt to approach it from the less traditional perspectives of embodiment and entanglement, affect and action.

Our current research on ‘messiness’ and creativity focuses on developing tools and concepts that make messiness seem neat and orderly, rather than confronting messiness as a natural state or even enabling messiness as legitimate language practice.

The translingual and trans-semiotic messiness may be described as ‘embodied creativity’. Such a paradigm-shifting view of creativity may be viewed as a theoretical response to the translanguaging reality in L2 education (Wang 2020), often involving enactment of one’s full repertoire to make meaning in the entire circle of language teaching, learning, and assessment. However, existing research has found that L2 assessment is largely missing out on theoretical advancement illuminated by a translanguaging lens.

Integrating translanguaging into language assessment requires a creative design that can engage learners in participating in meaningful tasks and can allow them to freely express their minds and identities in the process of demonstrating their evidence of language learning. So-called task-based language assessment (TBLA) may potentially offer a favourable advantage that enables learners to draw on their own resources as they engage in a real-life communication task (East 2021: 494; Norris and East 2021). Indeed, this was used as an assessment design principle in a study conducted by Schissel et al. (2018) on creative multilingual testing design for multilingual speakers. Closely linked to a TBLA approach, formative assessment data can help teachers identify students’ areas of strength or weakness (Ellis 2003; Ke 2006) and understand what the new languaging experience can embody through individualised learning evidence. Overall, being creative involves using existing knowledge or skills in a particular subject or context to experiment with new possibilities in the pursuit of valued outcomes.

A further weakness in the innovation in assessment arena has been the limited attention that has been given to assessment design and innovation in languages other than English. Despite some scholars having explored creative teaching and assessment designs (e.g., García et al. 2017), almost all research has been carried out with students learning English as L2. Most students in existing translanguaging research are newly arrived migrants (Lopez et al. 2017), refugees (van Viegen 2020), and minoritised or marginalised youth (Yilmaz 2021), as well as endangered-language speakers (Hickey et al. 2014). The predominant focus on minority speakers in the Global North has been noted by Turner and Lin (2017), who remarked that translanguaging should go beyond being a subaltern concept for Eurocentric issues, and that more studies are needed to move translanguaging theory forward. The relative lack of research attention on L2 learning in the overall translanguaging advocacy reflects the hegemony of English in applied linguistics research (Macedo et al. 2015) and results in a noticeable gap between the theory advancement led by English teaching and the ground level practices of multilingual education. There is also a growing concern that globalisation, and global English, have resulted in an increasingly low motivation for learning an L2 other than English (Lanvers et al. 2021), and the alarmingly low enrolments in other languages have reinforced English monolingualism in mainstream education. Because of the language learning crisis, what is even more concerning is that Anglophone countries will have more monolingual English speakers in a world where plurilingualism is common in many other countries. East (2015) has warned us that “monolingualism implies inflexibility, insensitivity and arrogance.” This may aggravate the social problems that we have, and the ongoing humanitarian crises caused by populist decision-makers and that work against L2 learning.

The remote emergency teaching precipitated by COVID-19 has served as a catalyst for making profound and creative changes that have thus far been considered impossible in language teaching and assessment (Bond et al. 2021; East et al. 2023; Wang and East 2020). While language teaching methods have evolved drastically over the past few decades, language assessment design remains largely unchanged, especially in the higher education context. University language courses are normally informed by standard proficiency frameworks with aligned outcomes that determine students’ eligibility to apply for scholarships, internships, and study abroad experiences, and that are also linked to prerequisites for higher level courses that require a certain entry-level proficiency. These outcomes have been frequently measured in traditional ways. Because of the disruptions brought about by prolonged emergency remote teaching, many universities have decentralised their assessment policies for L2 programmes, allowing course directors to make more flexible decisions for assessing students equitably and inclusively. Course developers and teachers now have more freedom to explore creative assessment strategies that provide students with a more interactive and authentic learning experience, an experience that is often not well supported in the centralised paper-based examination condition.

The pandemic has pushed many educational institutions to take digital and even unconventional solutions to deal with widescale emergency remote teaching (Bond et al. 2021). Many frontline teachers have navigated creative digital solutions to redesign or even place their conventional paper-based proctored examinations alongside creative online tests to ensure that, as far as possible, learning objectives can be achieved in fair and reliable conditions (East et al. 2023; Wang and East 2020). The huge shift to remote teaching has posed unprecedented challenges for beginning language courses that cover only limited words, simple grammar structure, and basic cultural concepts. These problems have also prompted new and bold educational innovations in areas that have been an unquestioned norm for decades or even centuries.

2.3 Assessing Chinese writing

One of the impacts on L2 assessment as a consequence of online teaching is teaching and assessing Chinese writing in the digital environment. The linguistic nature of the Chinese language makes the assessment of its writing more challenging and complex than that of English and other alphabetic languages (Xu et al. 2021; Zhang and Lin 2017). This is due to the lack of a phonology-to-logograph link in the Chinese writing system. For European language speakers, orthographical skill and knowledge in Chinese are disconnected from their prior experience of literacy development and writing practice. Particularly for beginners, learning to write Chinese characters by hand has proven to be a considerable challenge (Wang and Li 2022), requiring a tremendous amount of practice time in order to gain long-term memory and retrieve information from doing basic reading and writing tasks. One of the fundamental learning components of a beginner Chinese language course has been to train students to hand-write Chinese characters stroke by stroke.

In writing assessments, beginner Chinese learners are expected to demonstrate their ability to unify the phonological, semantic, and orthographical aspects around one particular character. In completing these assessments, students need to recall the strokes of a character, alongside its meaning and pronunciation. Due to the complexity of the Chinese writing system that requires a prolonged process for initial learners to gain basic skills, beginner Chinese courses often do not have the capacity to integrate informative task-based assessments that develop students’ literacy and communication skills beyond lexical and sentence levels. The centrality of handwriting in Chinese writing assessment has been criticised by many previous learners and has been regarded as a major reason that holds Chinese learners back from being confident users of the language (Gil 2020). It is suggested that future Chinese writing assessment developers should utilise digital solutions to “save time and efforts” for students learning Chinese (Gil 2020: 39).

It is important to note that writing in Chinese is not assessed until students reach Level 3 in the current version of the standardised Chinese proficiency test, Hanyu Shuiping Kaoshi (HSK). In contrast to the various benchmarks set for English writing assessment, the objective of the Chinese writing test is to assess students’ ability to hand-write Chinese characters. Writing tasks in Levels 3 and 4 (equivalent to CEFR B1 and B2) are limited to the sentence level. Essay and story writing are included in Levels 5 and 6 (equivalent to CEFR C1 and C2). Furthermore, the sentence-construction writing tasks in HSK are largely “context-free, with little interaction with how language is used in a real-life situation” (Peng et al. 2021: 329). It is also apparent that Chinese writing assessment for non-L1 speakers has operated under a monologic ideology, with focus on the accuracy and harmony of the script.

The global pandemic has profoundly challenged print-based handwriting-centred Chinese language teaching and learning. In a recent survey of Chinese instructors in U.S. postsecondary institutions, Xu et al. (2021) found that teachers usually welcomed the digital approach to writing instruction during the emergency remote teaching. Zhang’s (2021) research found that learners using keyboard typing were able to perform significantly better in writing assessment of Chinese. Students not only produced longer and more accurate sentences but also had better Chinese character retention rate. However, Wang and Li (2022) pointed out that innovative approaches based on multimodality remain unknown to many Chinese language teachers.

2.4 Second language education in New Zealand: key contextual factors

The present exploration builds on the premise that translanguaging perspectives and practices are bound by macro-meso ideological, sociocultural, and institutional contexts (Shohamy 2011). Given deeply entrenched monolingualism in mainstream education (East 2009; Major 2018), the creativity of the multilingual approach in language teaching and testing can be easily overshadowed or even inhibited by many contextual factors, such as controversies around the translanguaging approach, strict assessment policies in tertiary institutions, and the decline in students’ motivation in learning an L2.

Recent research has explored the implications of, and creative solutions with, translanguaging pedagogies in L2 teaching in the New Zealand context. Wang (2020) surveyed 237 students from three universities and found a large discrepancy between the prescribed monolingual principles in language teaching and the experienced translanguaging reality in day-to-day classroom interaction. In another study related to L2 learning, Walker (2018) analysed the translanguaging patterns in learners’ online interactions and concluded that translanguaging was used as an affordance strategy to expand learners’ semiotic repertoires for meaning co-construction. Focusing on creative learning materials design, Seals and Olsen-Reeder (2020) presented a case of using translanguaging materials to support children from indigenous and marginalised minority groups as part of revitalising and maintaining vulnerable languages. It seems that the translanguaging space co-created by teachers and students is mostly well-received by language learners and users in New Zealand. However, none of these studies has tapped into assessment.

Decisions on major assessments in formal education are usually informed by centralised policy and subject to the educational institution’s infrastructure and environment. Assessment within a university is more or less a series of administrative and pedagogical legacies to ensure that the mainstream knowledge system is safeguarded. It also has the power to shape how alternative knowledge can be adapted to fit into the mainstream system and conform with the assessment regulations that are deemed effective for assessing English and European languages. For example, in the authors’ university, the assessment policy required all stage-one (first year) courses to be partially formally assessed under examination conditions. It has been stipulated that, for these courses, “at least 50 % of course assessment must occur in invigilated settings, normally achieved through formal examinations and/or tests sat under examination conditions” (Assessment [Coursework, Tests and Examinations] Policy 2019, Item 20). Moreover, the university does not have the capacity to administer computer-based examinations on campus for large-scale courses and prior students in the focal course in question here had traditionally been completing high-stakes paper-based final examinations.

Given the continuous decline in L2 learning in New Zealand, we contend that the rationale to set up centralised high-stakes assessments based on the L1 speaker standard merits a reconsideration. A recent study found that learners of Chinese as L2 at a leading university in New Zealand were found to be more interested in learning the language for personal interest than for material gain or career enhancement (Wang 2020). This finding is in line with de Burgh-Hirabe’s study (2019) which found low instrumental motivation among Japanese language learners at university. Furthermore, recent studies have attested to the fact that learners of languages other than English are more often motivated by personal development to “expand their meaning-making repertoire for identity expression rather than wanting to gain proficiency in a particular language” (Zheng et al. 2020: 796).

At the authors’ university, an internal on-site investigation was conducted with 1,188 former students who took Chinese between 2014 and 2017. In total, 141 valid responses were collected. It was found that 47% of students were primarily motivated to study Chinese to fulfil a “desire for self-satisfaction”, such as travel, entertainment, stepping outside the comfort zone, or because they simply enjoyed learning an Asian language. This was followed by 23.1% of students motivated by a “desire for career enhancement” and 11.9% who had a “desire to be integrated with the Chinese culture”. The findings indicated that the majority of students at the university were not learning Chinese for career-related purposes. Having high-stakes assessments irrelevant to real-life communication could therefore do a disservice when it comes to maintaining students’ learning motivation. In the same research site, looking at oral assessment for beginners in Chinese as L2, Wang (2019a) found that students had been assessed in a decontextualised, form-focused, teacher-led, and memorisation-based setting, often arranged in a nerve-wracking one-on-one meeting in a lecturer’s office, twice a semester. This echoes what Shohamy (2007) described as her unpleasant experiences in taking language tests. She argued (2007: 142), “[i]t was tests that were responsible for turning the enjoyment and fun of learning into pain, tension, and a feeling of unfairness”.

The question therefore arises around how L2 assessment designs can be adapted or transformed when emergency remote teaching realities come up against traditional monolingual ideologies. It is also important to take into consideration students’ responses to new and creative assessment designs (see, e.g., East 2016). Towards this purpose, this study aims to shed light on the two following research questions: (1) How do beginning L2 learners perform on an online writing test designed based on the concept of translanguaging? (2) How do they perceive the new test design?

4.1 Students’ performances

It should be noted that typing in Chinese is not as straightforward as typing in alphabetical languages. Students are faced with several challenges as part of the complex process of decoding and encoding the script while composing in Chinese digitally. Typing one character or word in Chinese involves six steps (Zhang and Min 2019): transforming ideas into Chinese words; recalling pronunciation; using Pinyin (the Chinese romanisation system) to transliterate the Chinese word; typing the romanised letters on the keyboard; selecting; and finally confirming the correct character based on its orthography from a list of homophones prompted in an input method interface.

Students’ overall performance on their first translanguaging test were nonetheless deemed to be successful. All 163 students completed the writing task and submitted the test and their survey comments within the time limit. Table 1 shows two examples of students’ writing output. Some emerging findings are discussed below.

4.2 Students’ perceptions

Students’ comments on the translanguaging design provided insight into their performance on the writing task. The total number of comments (171 supportive comments and 42 sceptical comments) outnumbered the total student number (163) because some students commented from both sides. Overall, it seems that students were supportive of the digitally-mediated translanguaging assessment design.