« back to Publications
Tse, E. (2007)
Multimodal Co-located Interaction. PhD thesis, Department of Computer Science, University of Calgary, Calgary, Alberta, Canada, December.
Type: PhD thesis.
Note: Tse was under my supervision.
Download as: Pdf
Abstract.
With the advent of very large, high resolution and affordable digital displays researchers are investigating interaction techniques suitable for collaborative work. This dissertation explores the design and technical development of technologies that support multiple co-located people collaborating with multimodal speech and gesture commands over a digital table. I first explain why it would be useful to use speech and gesture commands in a co-located setting through a set of behavioural foundations summarizing theories, empirical and ethnographic research on how people collaborate in everyday settings. I then described how to design multi user speech and gesture interfaces atop existing single user applications, and observed how people use these wrappers for collaborative work. To examine the effectiveness of speech filtering for selection over a large digital wall display I performed a controlled evaluation and found that multimodal commands improved selection efficiency, accuracy and user preference over two gesture-only selection techniques. The remainder of this thesis described further technical explorations in the design of multimodal co-located applications, including GSI DEMO, a multimodal split view tabletop, and an application to the brainstorming activities of industrial designers.