DOI: Publication Download
Concept
The hypothesis for this research grew from heuristic evaluations that I authored for software evaluations. These reports were effective and efficient at communicating problem areas in the user experiences (UX) and interfaces (UI) of software products to team members and clients that were not UX, UI, or Human Computer Interaction (HCI) experts. This formulated the bases for our hypothesis that heuristics can be used as a common language to and communicate UX and HCI design decisions.
Pilot Study
After teaming up with a group of senior HCI researchers, in the visual analytics community, we decided to see if certain heuristics were better suited for certain visualization types. We developed a pilot study that asked a group of UX experts to evaluate a heuristic set against 10 different visualizations. The visualizations were all unique and each displayed unique data sets. The results from the pilot study suggested that our heuristic set, from Nielsen and Norman, was not well suited to evaluate visualizations. And the varied data sets for the visualizations made interpretation of the results difficult.
Methodology
Our research methodology used a published heuristic set from the visual analytics community from Forsell and Johansson. Along with a VAST challenge data set as to use as a consistent data source for each visualization to display. We recruited participants with domain expertise in UX design, UI development, and HCI to evaluate the heuristic set against the different visualization types. Our hypothesis is that the results would determine if certain heuristics were better suited to evaluate specific visualization types.
Abstract
Multiple sets of heuristic have been developed and studied in the Human Computer Interaction (HCI) domain as a method for fast, lightweight evaluations for usability problems. However, none of the heuristics have been adopted by the information visualization or the visual analytics communities. Our literature review looked at heuristic sets developed by Nielsen and Molich and Forsell and Johansson to understand how these heuristics were developed and their intended applications. We also reviewed heuristic studies conducted by Hearst and colleagues and Vaataja and colleagues to determine how individuals apply heuristics to evaluating visualization systems. While each study noted potential issues with the heuristic descriptions and the evaluator’s familiarity with the heuristics, no direct connections were made. Our research looks to understand how individuals with domain expertise in information visualization and visual analytics could use heuristics to discover usability problems and evaluate visualizations. By empirically evaluating visualization heuristics, we can identify the key ways that these heuristics can be used to inform the visual analytics design process. Further, they may help to identify usability problems that are and are not task specific. We hope to use this process to also identify missing heuristics that may apply to designs for different analytic purposes.
