Fatality Killed the Cat or: BabelPic, a Multimodal Dataset for Non-Concrete Concepts

Agostina Calabrese, Michele Bevilacqua, Roberto Navigli

Abstract Paper Share

Resources and Evaluation Short Paper

Session 8B: Jul 7 (13:00-14:00 GMT)
Session 9A: Jul 7 (17:00-18:00 GMT)
Abstract: Thanks to the wealth of high-quality annotated images available in popular repositories such as ImageNet, multimodal language-vision research is in full bloom. However, events, feelings and many other kinds of concepts which can be visually grounded are not well represented in current datasets. Nevertheless, we would expect a wide-coverage language understanding system to be able to classify images depicting recess and remorse, not just cats, dogs and bridges. We fill this gap by presenting BabelPic, a hand-labeled dataset built by cleaning the image-synset association found within the BabelNet Lexical Knowledge Base (LKB). BabelPic explicitly targets non-concrete concepts, thus providing refreshing new data for the community. We also show that pre-trained language-vision systems can be used to further expand the resource by exploiting natural language knowledge available in the LKB. BabelPic is available for download at http://babelpic.org.
You can open the pre-recorded video in a separate window.
NOTE: The SlidesLive video may display a random order of the authors. The correct author list is shown at the top of this webpage.

Similar Papers

Personalized PageRank with Syntagmatic Information for Multilingual Word Sense Disambiguation
Federico Scozzafava, Marco Maru, Fabrizio Brignone, Giovanni Torrisi, Roberto Navigli,
A representative figure from paper demo.69
Dialogue-Based Relation Extraction
Dian Yu, Kai Sun, Claire Cardie, Dong Yu,
A representative figure from paper main.444
Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen
Yixin Cao, Ruihao Shui, Liangming Pan, Min-Yen Kan, Zhiyuan Liu, Tat-Seng Chua,
A representative figure from paper main.100
What Does BERT with Vision Look At?
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang,
A representative figure from paper main.469