Abstract
Many usability practitioners conduct most of their usability
evaluations to improve a product during its design and development. We call
these "formative" evaluations to distinguish them from "summative"
(validation) usability tests at the end of development.
A standard for reporting summative usability test results has been adopted
by international standards organizations. But that standard is not intended
for the broader range of techniques and business contexts in formative work.
This paper reports on a new industry project to identify best practices
in reports of formative usability evaluations.
The initial work focused on gathering examples of reports used in a variety
of business contexts. We define elements in these reports and present some
early guidelines on making design decisions for a formative report. These
guidelines are based on considerations of the business context, the relationship
between author and audience, the questions that the evaluation is trying
to answer, and the techniques used in the evaluation. Future work will continue
to investigate industry practice and conduct evaluations of proposed guidelines
or templates.
Practitioner’s Take Away
- There is little guidance in the literature for the good design of a
report on a formative evaluation. This is in contrast to summative evaluation
reports, for which there is an international standard. - There is wide variation in reporting on formative usability evaluations.
- The audience for a formative usability report should be carefully considered
in designing the report format. The content, presentation or writing style,
and level of detail can all be affected by differences in business context,
evaluation method, and the relationship of the author to the audience. - The IUSR project seeks to provide tools such as best practice guidelines
and sample templates to help practitioners communicate formative evaluation
results more effectively. To join the IUSR community, visit http://www.nist.gov/iusr/