« back to Publications
Tse, E., Greenberg, S., Shen, C., Forlines, C. and Kodama, R. (2008)
Exploring True Multi-User Multimodal Interaction over a Digital Table. In Proceedings of the ACM Conference on Designing Interactive Systems – ACM DIS’08. (Cape Town, South Africa), ACM Press, pages 109-118, February 25-27.
Type: Refereed conference (full paper).
Earlier version: Report 2007-877-29, August.
Additional materials: video (shown at conference).
DOI: 10.1145/1394445.1394457
Download as: Pdf
Download as: Video
Abstract.
True multi-user, multimodal interaction over a digital table lets co-located people simultaneously gesture and speak commands to control an application. We explore this design space through a case study, where we implemented an application that supports the KJ creativity method as used by industrial designers. Four key design issues emerged that have a significant impact on how people would use such a multi-user multimodal system. First, parallel work is affected by the design of multimodal commands. Second, individual mode switches can be confusing to collaborators, especially if speech commands are used. Third, establishing personal and group territories can hinder particular tasks that require artefact neutrality. Finally, timing needs to be considered when designing joint multimodal commands. We also describe our model view controller architecture for true multi-user multimodal interaction.
Keywords: Tabletop interaction, multimodal speech and gesture interfaces, computer supported cooperative work.
