2006-MultimodalCollab-UISTDoctoral

« back to Publications

Tse, E. (2006)
Multimodal Co-located Collaboration. In UIST Doctoral Consortium, Adjunct Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2006), October 15–18, Montreux, Switzerland. Includes minute madness video, duration 0:57.
Type: Refereed conference (doctoral consortium). Additional materials: minute madness video.

Download as: Pdf

Additional materials: Video (minute madness presentation)

Abstract.
People naturally perform multimodal interactions in everyday real world settings, as they collaborate over large visual surfaces such as paper maps on walls and tables. While a new generation of research technologies now supports co-located collaboration, they do not yet directly leverage such rich multimodal interaction. Even though a rich behavioural foundation is emerging that informs the design of co-located collaborative technologies, most systems still typically limit input to a single finger or pointer. This problem is partly caused by the fact that single point interaction is the only input easily accessible to researchers through existing toolkits and input APIs. In this research, I distil existing theories, models and ethnographic studies on co-present collaboration into behavioural foundations that describe the individual and group benefits for using gesture and speech multimodal input in a co-located large display setting. Next, I will develop a toolkit that facilitates rapid prototyping of multimodal co-located applications over large digital displays. Finally, using these applications, I will conduct a number of studies exploring design in a multimodal setting.

Keywords: Multimodal Interaction, Co-located Interaction, Tabletop Interaction, Whole handed gestures, Verbal Alouds.

Scroll to Top