Sessions

Assessing Speaking Tasks: How can teachers evaluate students’ speech more effectively?

This study examined the relationship between two approaches to assess oral performances: analytical complexity, accuracy, and fluency (CAF) indices and human raters’ evaluations. CAF indices are analyzed with transcribed data and used frequently in second language (L2) speaking research; however, speaking tasks are communicative and goal-oriented, so the degree to which students achieve those communicative goals need to be assessed (Pallotti, 2009). The participants were Japanese university students (N = 48), who completed opinion-based speaking tasks. The recorded audio data were transcribed and analyzed using CAF measures. In addition, 11 human raters evaluated the same recorded oral performances on the following domains: topic organization, complexity, accuracy and fluency. The raw scores produced by the raters were analyzed with multifaceted Rasch analysis and computed into one logit, which is considered communicative adequacy. The results of a multiple linear regression analysis showed fluency accounted for a significant amount of the communicative adequacy, but other measures (lexis, complexity, accuracy) explain only a small portion of the variance. I conclude by suggesting that researchers and language teachers can better understand the extent to which learners achieve communicative goals by incorporating human ratings of communicative achievement into speaking assessments.