GAIA: A Fine-grained Multimedia Knowledge Extraction System

Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman

Abstract Paper Demo Share

System Demonstrations Demo Paper

Demo Session 4A-1: Jul 6 (17:00-18:00 GMT)
Demo Session 5A-1: Jul 6 (20:00-21:00 GMT)
Abstract: We present the first comprehensive, open source multimedia knowledge extraction system that takes a massive stream of unstructured, heterogeneous multimedia data from various sources and languages as input, and creates a coherent, structured knowledge base, indexing entities, relations, and events, following a rich, fine-grained ontology. Our system, GAIA, enables seamless search of complex graph queries, and retrieves multimedia evidence including text, images and videos. GAIA achieves top performance at the recent NIST TAC SM-KBP2019 evaluation. The system is publicly available at GitHub and DockerHub, with a narrated video that documents the system.
You can open the pre-recorded video in a separate window.
NOTE: The SlidesLive video may display a random order of the authors. The correct author list is shown at the top of this webpage.

Similar Papers

Cross-media Structured Common Space for Multimedia Event Extraction
Manling Li, Alireza Zareian, Qi Zeng, Spencer Whitehead, Di Lu, Heng Ji, Shih-Fu Chang,
A representative figure from paper main.230
A Re-evaluation of Knowledge Graph Completion Methods
Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, Yiming Yang,
A representative figure from paper main.489
A Transformer-based Approach for Source Code Summarization
Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang,
A representative figure from paper main.449
Syntactic Search by Example
Micah Shlain, Hillel Taub-Tabib, Shoval Sadde, Yoav Goldberg,
A representative figure from paper demo.44