TU Darmstadt / ULB / TUprints

Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning

Chalvatzaki, Georgia ; Younes, Ali ; Nandha, Daljeet ; Le, An Thai ; Ribeiro, Leonardo F. R. ; Gurevych, Iryna (2023)
Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning.
In: Frontiers in Robotics and AI, 2023, 10
doi: 10.26083/tuprints-00024479
Article, Secondary publication, Publisher's Version

[img] Text
frobt-10-1221739.pdf
Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (35MB)
Item Type: Article
Type of entry: Secondary publication
Title: Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning
Language: English
Date: 11 September 2023
Place of Publication: Darmstadt
Year of primary publication: 2023
Publisher: Frontiers Media S.A.
Journal or Publication Title: Frontiers in Robotics and AI
Volume of the journal: 10
Collation: 15 Seiten
DOI: 10.26083/tuprints-00024479
Corresponding Links:
Origin: Secondary publication DeepGreen
Abstract:

Long-horizon task planning is essential for the development of intelligent assistive and service robots. In this work, we investigate the applicability of a smaller class of large language models (LLMs), specifically GPT-2, in robotic task planning by learning to decompose tasks into subgoal specifications for a planner to execute sequentially. Our method grounds the input of the LLM on the domain that is represented as a scene graph, enabling it to translate human requests into executable robot plans, thereby learning to reason over long-horizon tasks, as encountered in the ALFRED benchmark. We compare our approach with classical planning and baseline methods to examine the applicability and generalizability of LLM-based planners. Our findings suggest that the knowledge stored in an LLM can be effectively grounded to perform long-horizon task planning, demonstrating the promising potential for the future application of neuro-symbolic planning methods in robotics.

Uncontrolled Keywords: robot learning, task planning, grounding, language models (LMs), pretrained models, scene graphs
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-244795
Classification DDC: 000 Generalities, computers, information > 004 Computer science
Divisions: 20 Department of Computer Science > Ubiquitous Knowledge Processing
Zentrale Einrichtungen > hessian.AI - The Hessian Center for Artificial Intelligence
Date Deposited: 11 Sep 2023 12:35
Last Modified: 30 Oct 2023 07:16
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/24479
PPN: 512750394
Export:
Actions (login required)
View Item View Item