« back to Publications
Greenberg, S. and Buxton, B. (2008)
Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the ACM Conference on Human Factors in Computing Systems – ACM CHI’08. (Florence, Italy), ACM Press, pages 111-120, April 5-10.
Type: Refereed conference (full paper).
Awards: 🏆Best paper nomination.
Earlier version: Report 2007-879-31.
Additional materials: presentation and minute madness video.
Note: As this paper advocated reconsidering certain standard HCI practices, it served as the anchor for a (heavily attended) special panel session that brought in various points of views from experts in CHI.
DOI: 10.1145/1357054.1357074
Download as: Pdf
Download as: Video (minute madness presentation)
Additional download: 
Powerpoint
Abstract.
Current practice in Human Computer Interaction as encouraged by educational institutes, academic review processes, and institutions with usability groups advocate usability evaluation as a critical part of every design process. This is for good reason: usability evaluation has a significant role to play when conditions warrant it. Yet evaluation can be ineffective and even harmful if naively done ‘by rule’ rather than ‘by thought’. If done during design brainstorming, it can kill creative ideas that do not conform to current interface norms. If done prematurely during early system design, the many interface issues seen can kill what would could have been an inspired vision. If done to verify an academic prototype, it may incorrectly suggest a design’s worthiness rather than offer a meaningful critique of how it would be adopted and used in everyday practice. If done without regard to how cultures adopt technology over time, then today’s reluctant reactions by users will forestall tomorrow’s eager acceptance. Traditional usability evaluation should not be used to validate very early design stages or culturally-sensitive systems. Other reflective and critical methods should be considered in their stead.
Keywords: Usability testing, interface critiques, sketching, cultural artifacts, teaching usability.