Usability Testing

User selection and recruitment

Ideal participants are those that will eventually use the product. Representative users should ideally be selected on their match to actual users in terms of their general computer experience, experience with the product or similar products, and knowledge of the domain.

Usability goals

A successful usability test requires the development of clear, testable objectives. These objectives will then motivate the choice of tasks, data collection and interview questions.

Testing environment

Tests can be conducted in a variety of environments. Some considerations include the following:

• Minimize distractions
• Being able to observe and record the interaction
• Convenient location

Process

• Greeting and introduction
• Informed consent
• Questionnaire on user's background (optional)
• Task instructions (presented as scenarios)
• Task performance and data recording
• Debriefing, possibly with questionnaire and interview

Roles

• Greeter
• Facilitator (also called moderator)
• Computer (needed only for paper prototypes)
• Observer(s)

Often the greeter and the facilitator are the same person.

Materials

• Prototypes
• Script, with task instructions
• Checklists
• Questionnaire
• Consent form

Informed consent

The process of informed ensures that the potential participant fully understands the usability test and agrees to participate. See page 142 in McCracken and Wolfe for an example statement. It should inform the participant of the following:

• The purpose of the test
• Basic explanation of what will happen and for how long
• What and how data will be recorded (data will remain confidential)
• Test is voluntary
• Participant may withdraw at any time

Think-aloud variant

Sometimes it is helpful to ask the participant to think aloud as he or she performs the task. For this variant of the usability test, it is important that the user does not unnaturally reflect on the process. Helpful instructions include:

• As you are completing the task, please say everything that comes to mind.
• Say out aloud everything that you say to yourself silently. (paraphrased from Protocol Analysis: Verbal Reports as Data, by Ericsson and Simon)

Discussion Questions

• How should task instructions to test users be worded?
• What are strategies for making test users comfortable so that they act as they normally would when performing tasks?
• What are strategies for collecting observed behavior?