Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions
Hannah Craighead, Andrew Caines, Paula Buttery, Helen Yannakoudakis
NLP Applications Long Paper
Session 4A: Jul 6
(17:00-18:00 GMT)
Session 5B: Jul 6
(21:00-22:00 GMT)
Abstract:
We address the task of automatically grading the language proficiency of spontaneous speech based on textual features from automatic speech recognition transcripts. Motivated by recent advances in multi-task learning, we develop neural networks trained in a multi-task fashion that learn to predict the proficiency level of non-native English speakers by taking advantage of inductive transfer between the main task (grading) and auxiliary prediction tasks: morpho-syntactic labeling, language modeling, and native language identification (L1). We encode the transcriptions with both bi-directional recurrent neural networks and with bi-directional representations from transformers, compare against a feature-rich baseline, and analyse performance at different proficiency levels and with transcriptions of varying error rates. Our best performance comes from a transformer encoder with L1 prediction as an auxiliary task. We discuss areas for improvement and potential applications for text-only speech scoring.
You can open the
pre-recorded video
in a separate window.
NOTE: The SlidesLive video may display a random order of the authors.
The correct author list is shown at the top of this webpage.
Similar Papers
Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations
Karan Singla, Zhuohao Chen, David Atkins, Shrikanth Narayanan,

Multimodal and Multiresolution Speech Recognition with Transformers
Georgios Paraskevopoulos, Srinivas Parthasarathy, Aparna Khare, Shiva Sundaram,

Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
Changmao Li, Jinho D. Choi,

Learning to Understand Child-directed and Adult-directed Speech
Lieke Gelderloos, Grzegorz Chrupała, Afra Alishahi,
