back to CPSC 581

Project: The Proxemic Display

Worth: 20%

Sketch, design, implement and document via a portfolio a display whose contents

• react to the proximity (distance) of a person from it
• can be manipulated by gestures appropriate to that distance

You will be using the Microsoft Kinect and its API to determine distance.

Digital displays used to be largely limited to monitors on a person's desk. This has changed. Displays are now often found scattered in the environments. They exist in the home (e.g., an information appliance dedicated to photo viewing or home calendars or gaming), in the workplace (e.g., a digital whiteboard displaying on-going work), and in the public (e.g., digital signage). For many such displays, interaction (if available) is done via touch or gestures vs. via mouse and keyboard.

Unlike the desktop computer, digital displays need to be designed so they work both as background (ambient) displays that people just past by, to foreground displays that people can interact with. Several researchers have investigated the use of proximity to regulate this. That is, the display will change its contents (as well as how people can interact with it) as a function of distance and orientation (see the papers and videos below for important examples).

Your task is to design, implement, document and demonstrate a proxemic display that fits a particular context of use and particular personas.

Possibilities are endless, but should be based on realistic settings, personas, and use cases. Be creative. While your product should be functional and relevant, it should also be aesthetic, engaging and intriguing within the scope of your audience.

As a starting point of inspiration for your own project ideas, here are a few possible use cases of such a proxemic-aware display:

• advertisement display
• home calendar
• photo display
• announcement board
• video player
• board to leave notes (e.g., use the windows phones)
• stock market overview
• ...

To prepare for this assignment, you will learn:

• how to program the Kinect
• how to calibrate the Kinect to your display coordinates.
• (maybe) how to recognize gestures
• how others have designed ambient displays.

Pedagogical Objectives

• Apply basic sketching techniques and lateral thinking to produce various designs in a somewhat constrained but still highly open-ended project
• Acquire experiences creating and developing sketches in your sketchbook using various sketching methodologies
• Acquire first-time experiences creating a display that reacts to proxemics and to gestures
• Acquire further experiences creating a persona as well as defining contexts of use.

Assignment Deliverables and Details .

1. 10 competing design sketches, 10 sketch details, and sketch refinements. In your sketchbook, generate a minimum of 10 competing (different) designs for the device pairs. Include details of how the a person might interact with the proxemic-aware display. As before, talk about your design with others. Choose one or more of these designs, and then explore that design in detail (again, a minimum of 10 sketches) where you consider both variants of that design and particular in-depth details of the how a sequence of actions would unfold over it. You will be asked to briefly present your best one(s). You will also be asked to summarize your design as a narrative storyboard. As your ideas change, sketch them out. You will be asked at any time to show your sketches to the instructor, TA, and/or the class.
2. The Proxemic Display
   • The system will react to people entering a scene and how they move relative to the display.
   • The system will allow some interaction via gestures, where gestures may mean different things at different distances.
   • It will optionally includes phidgets and/or the Windows phone as other devices that interact with this display (e.g., for input, output or both).
   • You must pay attention to its aesthetics, its level of engagement, its interactivity, how it fits its function and role, how it senses information from the environment (input), and how it conveys information (output).
3. Persona description. Include persona descriptions of the typical audience you are designing this display for.
4. Context of use. Include a description of the context of use. Write it up similar to how you would write up a persona. You must at the very least describe the physical setting (include photos), what that setting is for, and how people move through it.
5. Demonstrate your work. During class time, you will be asked to demonstrate all the above work to other class members and the instructor
6. Visual project summary. Within your web portfolio, create a web-based visual summary that illustrates your system. As before, provide a title, minimal text (perhaps as annotations), and visuals that reveal what this is about. Emphasize the visuals over text - let it tell its own story. Don't feel bound to a static page - you can have things unfold, have sets of pages, animations, overlays, etc.
7. Video demonstration in your project summary. You will be asked to produce a video of your system that follows certain criteria (e.g., length, quality, engagement, how well it demonstrates your system). Further details will be provided in class.

Materials

• Microsoft Kinect plus cables
• Phidgets / Windows Phone as needed (if any)

Evaluation

Your grade will be based on your sketches, your design creativity, your implementation, documentation and packaging, demonstration, and your portfolio summaries. However, great successes or failures, or incompleteness in one exercise criteria will likely affect your total grade. Note that a successful implementation is required: if you cannot demonstrate your system, you will automatically receive a zero. Similarly, missing, late or incomplete work will result in a large grade penalty.

Resources that may help you get inspired / think about issues

• Kinect Software for programming the Kinect in WPF/C#.
• Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. Includes Video. Vogel, D. and Balakrishnan, R. Proceedings of the 17th annual ACM symposium on User interface software and technology, ACM New York, NY, USA (2004), 137-146.
• Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. Ballendat, T., Marquardt, N. and Greenberg, S. (2010). In Proceedings of the ACM Conference on Interactive Tabletops and Surfaces - ACM ITS'2010. (Saarbruecken, Germany), ACM Press, pages 121-130, November 7-10.
• Gradual Engagement between Digital Devices as a Function of Proximity: From Awareness to Progressive Reveal to Information Transfer. Marquardt, N., Ballendat, T., Boring, S., Greenberg, S. and Hinckley, K. (2012). In Proceedings of Interactive Tabletops and Surfaces - ACM ITS. (Boston, USA), ACM Press, 10 pages, November 11-14.

Project-specific schedule (Fall 2012), including deliverables

Note: During this time, you will be instructed on coding the Kinect, on producing videos, and on other related matters.