Séminaire ayant lieu dans le cadre du doctorat en informatique cognitive, en collaboration avec le centre de recherche CRIA et l'ISC

Casey KENNINGTON

Jeudi le 9 novembre 2023 à 10h30

PK-5115 (aussi possible d'y assister à distance, pour ce faire, vous devez vous inscrire ici)     

Titre : Robotic Grounding and LLMs: Advancements and Challenges

Résumé

Large Language Models (LLMs) are primarily trained using large amounts of text, but there have also been noteworthy advancements in incorporating vision and other sensory information into LLMs. Does that mean LLMs are ready for embodied agents such as robots? While there have been important advancements, technical and theoretical challenges remain including use of closed language models like ChatGPT, model size requirements, data size requirements, speed requirements, representing the physical world, and updating the model with information about the world in real time. In this talk, I explain recent advance on incorporating LLMs into robot platforms, challenges, and opportunities for future work. 

Biographie

Casey KENNINGTON is associate professor in the Department of Computer Science at Boise State University where he does research on spoken dialogue systems on embodied platforms. His long-term research goal is to understand what it means for humans to understand, represent, and produce language. His National Science Foundation CAREER award focuses on enriching small language models with multimodal information such as vision and emotion for interactive learning on robotic platforms. Kennington obtained his PhD in Linguistics from Bielefeld University, Germany. 

References 

Josue Torres-Foncesca, Catherine Henry, Casey Kennington. Symbol and Communicative Grounding through Object Permanence with a Mobile Robot. In Proceedings of SigDial, 2022. 

Clayton Fields and Casey Kennington. Vision Language Transformers: A Survey. arXiv, 2023.

Casey Kennington. Enriching Language Models with Visually-grounded Word Vectors and the Lancaster Sensorimotor Norms. In Proceedings of CoNLL, 2021

Casey Kennington. On the Computational Modeling of Meaning: Embodied Cognition Intertwined with Emotion. arXiv, 2023.