Saul Greenberg

Readings in Proxemic Interactions

UNDER CONSTRUCTION

These readings were used during a version of the course CPSC 781 - Advanced Topics in HCI (Focus on Proxemic Interactions, and as a general resource.

Overview

The course is based on a selection of readings from the list below. Thus the order follow (roughly) what students would be reading as the term progresses. Most of the course will be spent on the foundational material. Depending upon student interest, we select papers from particular case studies, The case studies are also good resources to inspire your project topic and its intellectual foundations, and for students to choose and present their Ubicomp topic.

Before the class, student responsibilities are to:

  • read them thoroughly
  • add an insightful discussion point or two about it on the blog,
  • be prepared to discuss it in class
  • selectively summarize and lead the discussion of the main points.

On a more general level, many of the readings on this list - particular the parts before the case studies - could also double as a foundational PhD reading list to Ubicomp.

Proxemics Introduction I: Exemplar Systems

  1. Vogel, D. and Balakrishnan, R.
    Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. Proceedings of the 17th annual ACM symposium on User interface software and technology, ACM New York, NY, USA (2004), 137-146. PDF and Video
    • The paper that (in my view) introduced many aspects of Proxemics as applied to HCI
  2. Ballendat, T., Marquardt, N. and Greenberg, S. (2010)
    Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. In Proceedings of the ACM Conference on Interactive Tabletops and Surfaces - ACM ITS'2010. (Saarbruecken, Germany), ACM Press, pages 121-130, November 7-10.
    • Our initial paper describing Proxemic Interactions
  3. Ju, W., Lee, B.A., and Klemmer, S.R. (2008)
    Range: exploring implicit interaction through electronic whiteboard design. Proceedings of the ACM conference on Computer supported cooperative work, ACM New York, NY, USA (2008), 17-26.
    • Much of Proxemic Interaction assumes some balance between implicit and explicit actions of a person. Ju discusses this.

Proxemics Introduction II: Early Notions

These systems illustrated some of the potential directions of Proemics. The are important not only for the systems they demonstrate, but for the philosophy behind them. The first two on reactive environments in part use ideas such as presence, proximity, and exploiting spatial layouts of spaces. The previous work by Vogel and by Ballendat represents both seminal and state of the art proxemic-aware system. Tat Mobile is a futuristic video where some of its devices are proximity-aware.

  1. Cooperstock, J., Fels, S., Buxton, W. & Smith, K.C.
    Reactive environments: Throwing away your keyboard and mouse, Communications of the Association of Computing Machinery (CACM), 40(9), 65-73. (1997)
  2. Buxton, W. (1997).
    Living in Augmented Reality: Ubiquitous Media and Reactive Environments. In K. Finn, A. Sellen & S. Wilber (Eds.). Video Mediated Communication. Hillsdale, N.J.: Erlbaum, 363-384. An earlier version of this chapter also appears in Proceedings of Imagina '95, 215-229.
  3. Tat Mobile UI
    Future of Screen Technology. This video only has a few segments dealing with proximity, but its an interesting thought piece that could trigger you into thinking about different ways that proximity could be used.

Proxemics Intro III: Hall's Theory

Edward Hall lays the sociological foundations to proxemics

  1. Edward Hall (1968)
    Proxemics. Current Anthropology 9, 2/3 (, 83-108.
  2. Hall, E.T. (1966)
    The Hidden Dimension. Doubleday, Garden City, N.Y, 1966. (chapters 9 and 10).

Proximity Toolkit

A brief introduction why toolkits are generally so important, and to the proximity toolkit in particular.

  1. Greenberg, S. (2007)
    Toolkits and Interface Creativity. Journal Multimedia Tools and Applications (JMTA), 32(2):139-159. (Special Issue on Groupware) Springer, February.
  2. Diaz-Marino, R. and Greenberg, S. (2010)
    The Proximity Toolkit and ViconFace: The Video. In Video Showcase, DVD Proceedings of the ACM Conference on Human Factors in Computing Systems - ACM CHI'10. (Atlanta, Georgia), ACM Press, 5 pages, April 10-15. Video and paper, demonstrated live at CHI. Duration: 4:11. (short paper and wmv).
  3. Marquardt, N., Diaz-Marino, R., Boring, S. and Greenberg, S. (2011)
    The Proximity Toolkit: Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies. In ACM Symposium on User Interface Software and Technology - UIST'2011. (Santa Barbara, CA, USA), ACM Press, 11 pages, October 16-18. Include video figure, duration 4:19.

Ubiquitous Computing: History

Mark Weiser is considered the 'father' of ubiquitous computing. These and the other papers by him represents his vision and early thinking. Unfortunately, an early death curtailed him from seeing the massive influence he had on the CS world. The other two papers summarize Ubicomp since then and reflect on the original vision.

  1. Weiser, M. (1991)
    The computer for the 21st Century. Scientific American. 94-110, September.
  2. Weiser, M. and Brown, J. (1996)
    Designing calm technology, Powergrid Journal, v1.01, July.
  3. Want, R. (2010)
    An Introduction to Ubiquitous Computing. In Ubiquitous Computing Fundamentals, Krum, J. (Editor), CRC Press. 2010.

Optional

  1. Weiser, M. (1993)
    Some computer science issues in Ubiquitous Computing. Communications of the ACM 36(7) July.

Tangible Bits

  1. Ishii, H. and Ullmer, B. (1997)
    Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. Proc ACM CHI'97 Conference on Human Factors in Computing Systems, 234-241.
  2. Morris, D., Saponas, T.S., and Tan, D. (2010)
    Emerging Input Technologies for Always-Available Mobile Interaction Foundations and Trends inHuman–Computer Interaction, Vol. 4, No. 4 (2010) 245–316

DONE TILL HERE

To read for October 29: Controlling Devices (David Ledo)

  1. Brad Myers
    Mobile Devices for Control, CMU.
  2. Rukzio and colleagues
    An Experimental Comparison of Physical Mobile Interaction Techniques: Touching, Pointing and Scanning
  3. Videos: Cristal and PICOntrol
  • Schmidt, D., Molyneaux, D., and Cao, X. PICOntrol: Using a Handheld Projector for Direct Control of Physical Devices through Visible Light. Proc ACM Symposium on User Interface Software and Technology (UIST ’12), ACM (2012). Video and Paper

To read for October 31: Personal Informatics (Bon Adriel)

  1. Ian Li, Anind K. Dey, and Jodi Forlizzi,
    Understanding my data, myself: supporting self-reflection with ubicomp technologies, UbiComp 2011, ACM, 2012.

Supplemental readings

  1. Ian Li, Anind K. Dey, Jodi Forlizzi,
    A stage-based model of personal informatics systems, CHI 2010, ACM, 2010
  2. Various videos: UbiGreen and UbiFit Garden

To Read for Nov. 5: Ambient Displays

Required Reading:

Wisneski, C., Ishii, H. Dhaley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P. (1998)Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information Proceedings of the First International Workshop on Cooperative Buildings (CoBuild '98), February 25-26, Springer.
http://pages.cpsc.ucalgary.ca/~saul/wiki/uploads/HCIPapers/wisnesky-ambient-1998.pdf

Supplementary Reads:

The above paper and more papers on this topic are listed under “Ambient Displays” section at the bellow link:
http://pages.cpsc.ucalgary.ca/~saul/wiki/pmwiki.php/CPSC781/Readings

Here is an ambient information display created by the Fraunhofer Institute:
Prante, T., Röcker, C., Streitz, N., Stenzel, R., Magerkurth, C., Van Alphen, D., & Plewe, D. A. (2003, October). Hello. wall–beyond ambient displays. In Adjunct Proceedings of Ubicomp (pp. 277-278).
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.3459&rep=rep1&type=pdf

Video:
http://youtu.be/QHNA_9i8I9I

To Read for Nov. 14: Device to device interaction

Required Reading:

Marquardt, N., Ballendat, T., Boring, S., Greenberg, S. and Hinckley, K. (2012) Gradual Engagement between Digital Devices as a Function of Proximity: From Awareness to Progressive Reveal to Information Transfer.In Proceedings of Interactive Tabletops a& Surfaces - ACM ITS. (Boston, USA), ACM Press, 10 pages, November 11-14. Paper and Video

Schilit, B., Adams, N., and Want, R. Context-Aware Computing Applications. Mobile Computing Systems and Applications, IEEE Workshop on, IEEE Computer Society (1994), 85-90.

To read for Human Robot Interaction

  • Walters, M.L., Dautenhahn, K., te Boekhorst, R., Koay, K.L., Syrdal, D.S., and Nehaniv, C.L.

An Empirical Framework for Human-Robot Proxemics. Proceedings of AISB Convention 2009.

  • Saulnier, P. Sharlin, E. and Greenberg, S. (2011)

Exploring Minimal Nonverbal Interruptions in HRI. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man 2011). (Atlanta, Georgia), IEEE Press, 8 pages, July 31-Aug 3

To Read for Privacy

For a quick overview on the problem

For a more extensive reading on the issue

Collection of papers that we may select from

XX: Ubiquitous Computing: History

Mark Weiser is considered the 'father' of ubiquitous computing. These and the other papers by him represents his vision and early thinking. Unfortunately, an early death curtailed him from seeing the massive influence he had on the CS world. The other two papers summarize Ubicomp since then and reflect on the original vision.

  1. Weiser, M.
    The computer for the 21st Century. Scientific American. 94-110, September. (1991)
  2. Weiser, M. and Brown, J.
    Designing calm technology, Powergrid Journal, v1.01, July, 1996.

Optional

  1. Weiser, M.
    Some computer science issues in Ubiquitous Computing. Communications of the ACM 36(7) July. (1993)

XX: Ubiquitous Computing: Reflection

These two papers summarize Ubicomp since Weiser and reflect on the original vision.

  1. Want, R.
    An Introduction to Ubiquitous Computing. In Ubiquitous Computing Fundamentals, Krum, J. (Editor), CRC Press. 2010.
  2. Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A., and Lopes, C.
    Making sense of sensing systems: five questions for designers and researchers. Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, ACM (2002), 415-422.
  3. Rogers, Y.
    Moving on from Weiser's vision of of calm computing: engaging UbiComp experiences. In: P. Dourish and A. Friday (Eds.) Ubicomp 2006 Proceedings, LNCS 4206, pp. 404-421, Springer-Verlag. (2006)

Optional

  1. Other readings in Marquard's Section 6

XX. Device to Device Proxemic Interactions

How can one device connect to an interact with another device based on proximity?

  1. Ramos, G., Hinckley, K., Wilson, A., and Sarin, R.
    Synchronous Gestures in Multi-Display Environments.
    Human-Computer Interaction 24, 1 (2009), 117.
  2. Kortuem, G., Kray, C., and Gellersen, H.
    Sensing and visualizing spatial relations of mobile devices.
    Proceedings of the 18th annual ACM symposium on User interface software and technology, ACM New York, NY, USA (2005), 93-102.
  3. Kray, C., Rohs, M., Hook, J., and Kratz, S.
    Group Coordination and Negotiation through Spatial Proximity Regions around Mobile Devices on Augmented Tabletops.
    3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems, 2008. TABLETOP 2008, (2008), 1–8.

Do a quick read on these (no blog entry required)

  1. Holmquist, L., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M., and Gellersen, H.
    Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts.
    In Ubicomp 2001: Ubiquitous Computing: Third International Conference Atlanta, Georgia, USA, September 30 - October 2, 2001, Proceedings. 2001, 116.
  2. Merrill, D., Kalanithi, J., and Maes, P.
    Siftables: towards sensor network user interfaces.
    Proceedings of the 1st international conference on Tangible and embedded interaction, ACM (2007), 75-78.

Optional

  1. See Marquardt's Section 2
  2. Schilit, B., Adams, N., and Want, R.
    Context-Aware Computing Applications.
    Mobile Computing Systems and Applications, IEEE Workshop on, IEEE Computer Society (1994), 85-90.

XX. Proximity: People to Large Digital Surfaces

  1. Brignull, H. and Rogers, Y.
    Enticing People to Interact with Large Public Displays in Public Spaces.
    Human-Computer Interaction, (2003).
  2. Hawkey, K., Kellar, M., Reilly, D., Whalen, T., and Inkpen, K.M.
    The proximity factor: impact of distance on co-located collaboration.
    Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, ACM New York, NY, USA (2005), 31-40.
  3. Shoemaker, G., Tang, A., and Booth, K.S.
    Shadow reaching: a new perspective on interaction for large displays.
    Proceedings of the 20th annual ACM symposium on User interface software and technology, ACM New York, NY, USA (2007), 53-56.
  4. Streitz, N., Prante, T., Röcker, C., et al.
    Ambient Displays and Mobile Devices for the Creation of Social Architectural Spaces.
    Public and Situated Displays Social and Interactional Aspects of Shared Display Technologies, (2003), 387-409.
  5. Walther-Franks, B., Schwarten, L., Teichert, J., Krause, M., and Herrlich, M.
    User Detection for a Multi-touch Table via Proximity Sensors.
    Extended Abstracts of IEEE Tabletops and Interactive Surfaces 2008, (2008).

XX: Place vs Space

  1. Harrison, S. and Dourish, P. 1996. Re-Place-ing Space: The Roles of Place and Space in Collaborative Systems. Proceedings of the ACM Conference on Computer-Supported Cooperative Work CSCW'96 (Boston, MA), 67-76. New York: ACM.
  2. Dourish, P. 2006. Re-Space-ing Place: Place and Space Ten Years On. Proc. ACM Conf. Computer-Supported Cooperative Work CSCW 2006 (Banff, Alberta), 299-308

Topics We Will Chose From

Additional readings (including choices for your presentation) will likely center around one of these topics.

Theory: Embodied Interaction

Dourish introduces the notion of embodied interaction. While it doesn't deal specifically with proxemics, it is a more general notion that is very relevant to it and to Ubicomp.

  1. Dourish, P. (2001)
    Where the Action Is. The Foundations of Embodied Interaction. MIT Press, Cambridge MMA.

Optional

  1. Hall, E.T. The Hidden Dimension. Doubleday, Garden City, N.Y, 1966. (chapter 10 especially). We have copies of this book.
  2. Dourish, P. (2001)
    Where the Action Is. The Foundations of Embodied Interaction. MIT Press, Cambridge MMA.

Proxemic relationships between people and surfaces

How can we tune the relationship between a person and a digital surface based on proximity?

See Marquardt's Section 1

Proxemic relationships between people and mobile/desktop computers

How can we tune the relationship between a person and their mobile/desktop device based on proximity?

See Marquardt's Section 3

Proxemic relationships within multi-display environments (devices, surfaces, people)

How can we manage multiple relationships with different people and device types?

See Marquardt's Section 4

Privacy and Security

Privacy and security are very important concerns in Ubicomp systems.

Must Read

or

  • Mark Langheinrich
    Introductory paper
    Ubicomp 2001: Ubiquitous Computing, 2001 - Springer [pdf]

Highly Recommended: Design Issues

Optional: Special Topics and Others

  • Video: Tile - the world's largest lost and found

http://www.youtube.com/watch?v=pqDm3gZNZPM

Also see Marquardt's Section 12

Ambient Displays

More Theory

While Hall started the Proxemic theory, others have continued in it.

See Marquardt's Sections 5 and Marquardt's Sections 15

Frameworks and Toolkits for Sensing and Exploiting Proximity and other Information

What are the technical foundations that let us build proximity-aware and other ubicomp systems?

See Marquardt's Section 9

Application area: Proximity applied to media and video spaces

These papers apply proxemics to control / design various kinds of video conferencing systems

See Marquardt's Section 10

Application area: Proximity applied to Virtual Environments and Human Robot Interaction

These papers apply proxemics to control / design VE and HRI systems. This could be two topics

See Marquardt's Section 11

Application area: Proximity applied to Personal Control Devices

These papers apply proxemics, where one device can be used to control another device e.g., as a remote control

See Marquardt's Section 14