« back to Publications

Steves, M.P., Morse, E., Gutwin, C. and Greenberg, S. (2001)
A Comparison of Usage Evaluation and Inspection Methods for Assessing Groupware Usability. In Proceedings of ACM International Conference on Supporting Group Work – ACM Group’01. ACM Press, pages 125-134, September 30-October 3.
Type: Refereed conference (full paper).
DOI: 10.1145/500300.500306

Download as: Pdf

Abstract.
Many researchers believe that groupware can only be evaluated by studying real collaborators in their real contexts, a process that tends to be expensive and time-consuming. Others believe that it is more practical to evaluate groupware through usability inspection methods. Deciding between these two approaches is difficult, because it is unclear how they compare in a real evaluation situation. To address this problem, we carried out a dual evaluation of a groupware system, with one evaluation applying user-based techniques, and the other using inspection methods. We compared the results from the two evaluations and concluded that, while the two methods have their own strengths, weaknesses, and trade-offs, they are complementary. Because the two methods found overlapping problems, we expect that they can be used in tandem to good effect, e.g., applying the discount method prior to a field study, with the expectation that the system deployed in the more expensive field study has a better chance of doing well because some pertinent usability problems will have already been addressed.

Keywords: Evaluation, groupware usability, inspection evaluation techniques, usage evaluation techniques.