Saul Greenberg

back to the course CPSC 681

CPSC 681 - Topic: Presenting a Methodology

Worth: 25% of your grade.

Introduction

You will research a particular methodological topic, present that topic to the class, and produce a web-based handout summarizing that topic (including a simple worked example that illustrates how that methodology is applied). I will normally assign topics to students, but if you have a topic that is of particular interest to you, you can discuss that choice with me.

Depending on the class size, you may be required to do the topic in groups of two rather than individually.

Deliverables

See schedule for tentative dates. All students will deliver written materials (presentations / handouts) on the same date. The date of the actual presentation will be scheduled by the instructor.

  1. A summary of your topic selection (early in the course) verifying your choice of topic
  2. A web-based handout, added to the CPSC 681 student site, in the form of a tutorial that concisely summarizes the methodology and the concrete exercise. The handout should include
    • Overview of the method: what it is, how it works, and (if known) a brief history of it
    • When and where it is typically used in the development/evaluation/research cycle
    • Benefits and problems of this method
    • A simple worked-through concrete exercise, suitable for teaching, of how to apply the method (should include all material needed to do the method (instructions, materials, etc.) The best examples are almost trivial ones that illustrate the method.
    • An annotated bibliography of several key papers / resources that others can use to help them get started
  3. A professional classroom presentation that introduces the methodology to the class and that has the class perform or walk through the above exercise of how the methodology is applied on a particular problem.

Resources

I will be happy to point you to a key resource as a starting point once you have chosen a topic. Ask me! However, you will be expected to find and research other relevant resources on your own.

  • Readings in HCI Evaluation Methodologies has a selected set of papers on particular evaluation topics.
  • Saul's Library contains a good number of relevant books that you can sign out. This includes textbooks with good introductory chapters on evaluation methods. Browse these for ideas.
  • Digital Libraries with HCI content abound, and include many articles illustrating, discussing and applying evaluation methods.

A note on plagiarism.

You must avoid plagiarism when using the above resources. Use your own words, quote selectively (with citations), and be careful to list your sources. You can selectively use a few images/examples provided by others, but you must cite them appropriately. Plagiarism - whether intentional or accidental - has severe consequences.


Example topics

I will select topics from the possible topics listed below. Depending on individual interests, I may focus on some topics over others.

For complex topics, I may assign several students to consider different aspects of it. If this is done, the students are expected to coordinate with each other.

Interviews

  • Interview techniques
  • Contextual interviews
  • Ethnographic interview
  • Focus groups

Heuristic Inspection

  • Heuristic evaluation (general)
  • Heuristic evaluation (specialized: groupware, ambient displays, etc.)

User/task models and walkthroughs

  • Personas
  • Task centered walkthrough
  • Cognitive walkthrough

Qualitative Analysis

  • Open coding
  • Affinity diagrams
  • Grounded theory

Questionnaires

  • How to design Questionnaires
  • How to design Surveys
  • Likert scales and statistical analysis

In the Wild

  • Field studies
  • Design ethnography
  • Cultural probes
  • Contextual inquiry

Creating and evaluating early designs

  • The role of prototyping
  • Wizard of Oz

Statistical methods

  • Statistical tests - non-parametric tests
  • Statistical tests - CHI-square distributions tests...

Modeling and evaluating low-level actions

  • Fitts Law
  • GOMS
  • Keystroke level model

Miscellaneous

  • Logging user performance
  • Instructional video - produce a video that illustrates a technique
  • Are 5 users enough?
  • Ethics