Learning to execute instructions in a Minecraft dialogue
Prashant Jayannavar, Anjali Narayan-Chen, Julia Hockenmaier
Language Grounding to Vision, Robotics and Beyond Long Paper
Session 4B: Jul 6
(18:00-19:00 GMT)
Session 5B: Jul 6
(21:00-22:00 GMT)
Abstract:
The Minecraft Collaborative Building Task is a two-player game in which an Architect (A) instructs a Builder (B) to construct a target structure in a simulated Blocks World Environment. We define the subtask of predicting correct action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.
You can open the
pre-recorded video
in a separate window.
NOTE: The SlidesLive video may display a random order of the authors.
The correct author list is shown at the top of this webpage.
Similar Papers
Mapping Natural Language Instructions to Mobile UI Action Sequences
Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, Jason Baldridge,

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Emily M. Bender, Alexander Koller,

CraftAssist Instruction Parsing: Semantic Parsing for a Voxel-World Assistant
Kavya Srinet, Yacine Jernite, Jonathan Gray, Arthur Szlam,

Relation-Aware Collaborative Learning for Unified Aspect-Based Sentiment Analysis
Zhuang Chen, Tieyun Qian,
