back to CPSC 581

Project: Designing Smart Phone Interaction Workarounds

Worth: 20%

Note: This exercise and the description below make extremely loose use of particular patents, their meaning, how patent disputes are fought and resolved, whether patents are valid or not, and what designs would actually infringe on a patent. It does not reflect an opinion, nor does it necessarily reflect any reality beyond the fact that these patents exist. It should not be used to suggest the validity or invalidity of particular patents, nor should should it be used as a summary or characterization of what these patents are about. The sole purpose of this description is to create an artificial scenario for a student exercise that promotes design and programming skills.

The recent patent lawsuits between Apple and various other smart phone producers include various disputes over particular interaction techniques. One well-publicized example is based on Apple's slide to unlock patents. The technique described in the patent purports to prevent accidental activation, where an inadvertent touch by a user may unintentionally activate some function on the phone. The basic strategy is that the phone is kept in a locked state, where the user has to perform a particular type of input to unlock it and access the phone functionality.

While there are various versions of this patent and its claims (e.g., the '721 in the US and '022 in Europe), they usually describe how a user makes contact with a graphical 'unlock image' on a hand-held touch-sensitive display, and then moves the unlock image from a pre-defined location either along a pre-defined displayed path or to a predefined unlock region. The first figure illustrates how one patent provides an example of this, while the second figure shows how the Apple iPhone actually realizes slide to unlock.

You will design, implement, document and demonstrate an interface that provides an alternative way to unlock a mobile phone equipped with a multi-touch display and with multiple sensors. In particular, you will

• develop two different techniques, with
  • one based on gestural and/or multi-touch input,
  • the other based on sensor input (e.g., for one-handed use).
• use animation to provide feedback that the system is recognizing the action you are taking, and how the system then unlocks the device after the action is completed.
• show how the technique avoids both accidental activation and how it also can be used as a (perhaps weak) password

It is up to you to be as creative as possible.

To prepare for this assignment, you will learn techniques on: how to brainstorm and refine ideas via sketching, how to program phones in WPF Silverlight (e.g., how to access sensor and touch input, and how to create rudimentary animations), and how to create a portfolio entry displaying your results.

Pedagogical Objectives

• Apply basic sketching techniques and lateral thinking to produce various designs
• Acquire first-time experiences brainstorming ideas via sketches in a sketchbook
• Learn how to implement, package, install and document WPF programs
• Acquire first-time experiences creating a portfolio web site and summary

Assignment Deliverables and Details .

1. 10 competing design sketches, 10 sketch details, and sketch refinements. In your sketchbook, generate at least 10 competing (different) designs for each of the two techniques. Show and explain your design(s) to others. You will know which ones are the most exciting by how you talk about them and how others react. Choose the most promising one, and then produce 10 details and/or variations of a particular sketch in your sketchbook. You will be asked to briefly present your best ones. As your ideas change, sketch them out. You will be asked at any time to show your sketches to the instructor, TA, and/or the class.
2. An implemented WPF application. Implement a WPF application that demonstrates how your system works on the phone.
3. Web portfolio. Create a web site that will contain all your portfolio summaries. Include a brief introduction to yourself and (if you prefer) a photo. Make it engaging. For ideas, Google design portfolio and art portfolio. Also see Artist Portfolio Guidelines for some hints on portfolios and how they are used. Note that you will evolve this portfolio over the course.
4. Visual project summary. Within the above web portfolio, create a web-based visual summary that illustrates your system. Provide a title, minimal text (perhaps as annotations), and visuals that reveal what this is about. Emphasize the visuals over text - let it tell its own story. Don't feel bound to a static page - you can have things unfold, have sets of pages, animations, overlays, etc. Have impact.
   
   Specifically, create a portfolio summary that documents and details your design and implementation in a visually appealing manner. Note that this may become your standard style for other summaries. The portfolio must minimally include the following (or equivalents):

   1. a front page with an illustrative image, description and index
   2. an illustrated and annotated sequence describing your project. Avoid lengthy text. Use visuals as much as possible, e.g., screen snapshots, animation, video, etc. The viewer should get a good sense of how your project works from this
   3. a download page containing the complete source and executables, and instructions for running the sample.
5. Demonstrate your work. During class time, you will be asked to demonstrate all the above work to other class members and the instructor

Materials

• an LG touch-sensitive phone running Microsoft Windows (provided)
• lab computers with Visual Studio running Silverlight and the Windows Phone emulator
• access to the above software where you can download it to your own computer

Evaluation

Your grade will be based on your sketches, your design creativity, your implementation, documentation and packaging, and your portfolio summaries. However, great successes or failures, or incompleteness in one exercise criteria will likely affect your total grade. Note that a successful implementation is required: if you cannot demonstrate your system, you will automatically receive a zero. Similarly, missing, late or incomplete work will result in a large grade penalty.

Lateral thinking

Your first thoughts will likely be something that is fairly derivative of what you are already familiar with. To go beyond these, think about:

• the many ways you open, unlock, unlatch, and generally change the state of everyday things
• the many ways you signal some change of state via movements, gestures and touch

Look for inspiration from unusual sources. For example,

• As you move about your day to day world, consider what you see and the action you are performing, and ask yourself how it can relate and/or be applied to this exercise. For example, opening a book 'unlocks' it. Would this be a reasonable metaphor to use in a way that people can quickly understand it?
• Google images on gestures. For every one you see, sketch out a possible unlock method, no matter how unlikely. Don't worry about whether you can implement them or not, or if it is any good; you can make decisions on that later.
• Similarly, google images on anything that changes state: open, switch, latch, turn, etc. Let the images inspire you to think in different ways.
• List all the phone's sensor capabilities. Think about how you can use each one's input to unlock the phone, where you consider multiple variations for each one. Then combine sensor input to think about how the fusion of sensor data (perhaps combined with touch data) can unlock the phone.

Project-specific schedule (Fall 2012 schedule)

The claims of slide to unlock

Claim 1. A computer-implemented method of controlling a portable electronic device comprising a touch-sensitive display (408, 1014), comprising:

• detecting contact with the touch-sensitive display while the device is in a user-interface lock state;
• transitioning the device to a user-interface unlock state if the detected contact corresponds to a predefined gesture;
• and maintaining the device in the user-interface lock state if the detected contact does not correspond to the predefined gesture;

characterized by

• moving an unlock image along a predefined displayed path on the touch-sensitive display in accordance with the contact,
• wherein the unlock image is a graphical, interactive user-interface object with which a user interacts in order to unlock the device.

Claim 3. ... wherein the predefined gesture corresponds to moving the unlock image along the predefined displayed path on the touch-sensitive display to a predefined location on the touch-sensitive display.