Usability Inspection Methods

Usability is the basis of UX design and the research and analytics used to support design decisions for usability are pivital to the product or service’s success. I recently came across an article by MeasuringU that defined heuristic evaluation within the broader analytic context as a usability inspection method. This encouraged me to delve even deeper into this term with a specific question in mind.

Are usability inspection methods a viable substitute for empirical user research?

The Four Inspection Methods

There are four primary inspection methods used in the field.

Heuristic Evaluation

Introduced in 1990 by Nielsen and Molich, heuristic evaluation requires a small group of usability experts evaluating an interface against a set of defined guidelines and best practices. The quality of this evaluation varies directly with the experience of the experts used for evaluation. Nielsen divided experts into three categories: novice, expert, and double experts. I believe the first two groups are self explanatory, however the third group, double experts, are those who have a deep understanding of usability and the specific interface being tested.

When using a group of usability experts to evaluate an interface, 55% to 90% of usability issues are found. This is great as inspections can happen fairly early on in the product development cycle. The downside is that the issues found are not weighted and the impact of each issue is not always clear. It may be something that the user would either encounter rarely or when encountered not care. Meaning you may spend a lot of time optimizing for something that affects 1% of your users.

Cognitive Walkthrough

A cognitive walkthrough is an inspection method that evaluates the interface design for its ease of exploratory learning, based on a cognitive model of learning and use (Hollingsed and Novick 2). There are two parts to this type of evaluation. The first requires detailing the interface used, target users, task, and actions taken during the task. The second part of the analysis is to evaluate your initial task against a user’s four step interaction process:

1. Set a goal to complete
2. Determine affordances
3. Select action that will progress towards set goal
4. Perform action and evaluate feedback

One major drawback to this method is the absence of guidelines about what actions are available and known to the user. Labeling actions properly becomes paramount. The cognitive walkthrough method requires a accurate choice of task scenarios to be effective. It also requires at least a small group of testers to expand the range of problems you can uncover.

Pluralistic Usability Walkthrough

The pluralistic usability walkthrough is the “cognitive walkthrough plus”. It creates a group of evaluators whom include representative users of the interface, product developers, and usability experts. There are five primary characteristics of this method (Hollingsed and Novick 2):

1. Inclusion of representative users, product developers, and human factors professionals
2. The application’s screens are presented in the same order as they would appear to the user
3. All participants are asked to assume the role of the user
4. Participants write down what actions they, as users, would take for each screen before the group discusses the screens
5. When discussing each screen, the representative users speak first

Unlike the other two methods described earlier, this method takes input from actual users. The interface does not need to be fully formed at this time. You could run this evaluation with paper prototypes to allow quick iterations and changes on the fly.

The chosen group of evaluators should have varying perspectives due to their different backgrounds and roles as stakeholders in the project. This inclusiveness has been proven to increase effectiveness of the inspection method by at least 30%. It is by far the most used and widespread method in the industry, though it may be referred to by a different name.

Formal Usability Inspection

A formal usability inspection is a pluralistic usability inspection consisting of only experts and their peers. The inspection is a lot more technical due to the absence of representative users, however this is not proven to be better than the other methods. The use of this method is not as popular as some of the others mentioned previously, however there isn’t enough information out there to confirm its true level of use.

Are Usability Inspection Methods a Substitute to Empirical Research?

Usability inspection methods provide valuable feedback on the usability of an interface early in the development cycle. This means less cost and faster iterations. However, as mentioned throughout this article, the discovery rate of usability issues is not 100%. It cannot produce the same result as empirical user research, but it can give you a head start on where to improve early on. In a project with a small budget and a quick turn around, usability inspection methods are a a bare minimum in user research and testing.