Implicit Discourse Relation Classification: We Need to Talk about Evaluation

Najoung Kim, Song Feng, Chulaka Gunasekara, Luis Lastras

Abstract Paper Share

Discourse and Pragmatics Short Paper

Session 9B: Jul 7 (18:00-19:00 GMT)
Session 10B: Jul 7 (21:00-22:00 GMT)
Abstract: Implicit relation classification on Penn Discourse TreeBank (PDTB) 2.0 is a common benchmark task for evaluating the understanding of discourse relations. However, the lack of consistency in preprocessing and evaluation poses challenges to fair comparison of results in the literature. In this work, we highlight these inconsistencies and propose an improved evaluation protocol. Paired with this protocol, we report strong baseline results from pretrained sentence encoders, which set the new state-of-the-art for PDTB 2.0. Furthermore, this work is the first to explore fine-grained relation classification on PDTB 3.0. We expect our work to serve as a point of comparison for future work, and also as an initiative to discuss models of larger context and possible data augmentations for downstream transferability.
You can open the pre-recorded video in a separate window.
NOTE: The SlidesLive video may display a random order of the authors. The correct author list is shown at the top of this webpage.

Similar Papers

The Paradigm Discovery Problem
Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, Nizar Habash,
A representative figure from paper main.695
A Two-Step Approach for Implicit Event Argument Detection
Zhisong Zhang, Xiang Kong, Zhengzhong Liu, Xuezhe Ma, Eduard Hovy,
A representative figure from paper main.667