Previous research suggests that the System Usability Scale (SUS) might not be generating usability scores that span the entire measurement range from 0 to 100, particularly at the lower end of the scale…
Archives: JUS
Usability Studies and the Hawthorne Effect
Abstract This paper provides a brief review of the Hawthorne effect, a discussion of how this effect relates to usability studies and help for practitioners in defending their studies against criticisms made on the basis of this effect. Practitioner’s Take Away The Hawthorne effect can be (mis)used as a basis on which to criticize the validity of human-centered studies, including usability studies. Therefore, it is important that practitioners are able to defend themselves against such criticism. A wide variety of defenses are possible; depending on which interpretation of the Hawthorne effect is adopted. To make an informed decision as to which interpretation to adopt, practitioners should be aware of the whole story regarding this effect. A precursor to any defense …
Group Usability Testing: Evolution in Usability Techniques
Abstract Usability testing has a long history. In its early form, it was conducted with many individual participants much like traditional research experiments. With the advent of discount usability engineering techniques, fewer participants were required (5-7 versus 30-50) and protocols were simplified. The evolution from “many to few” in usability testing has become the standard in formative testing. What is the next tool in our toolbox? This paper introduces a formative method called “group usability testing.” It involves several to many participants individually, but simultaneously, performing tasks, with one to several testers observing and interacting with participants. The idea for group usability testing arose as an answer to limited time resources and the availability of many users gathered together in …
Adapting Usability Investigations for Agile User-Centered Design
Abstract When our company chose to adopt an Agile development process for new products, our User Experience Team took the opportunity to adjust, and consequently improve, our user-centered design (UCD) practices. Our interface design work required data from contextual investigations to guide rapid iterations of prototypes, validated by formative usability testing. This meant that we needed to find a way to conduct usability tests, interviews, and contextual inquiry—both in the lab and the field—within an Agile framework. To achieve this, we adjusted the timing and granularity of these investigations, and the way that we reported our usability findings. This paper describes our main adaptations. We have found that the new Agile UCD methods produce better-designed products than the “waterfall” versions …
Expanding Usability Testing to Evaluate Complex Systems
Abstract This essay discusses ways that usability professionals can expand usability testing to evaluate complex systems, such as intelligence gathering and medical decision-making, that do not lend themselves to more traditional laboratory-based usability testing. In the essay, Redish explains what complex systems are, why they don’t lend themselves to traditional usability test methodologies, and what other techniques are available for gathering and analyzing the data. The essay also discusses the importance of involving domain experts in the design of the test to ensure that both the components and the system as a whole are being adequately tested.