Preparation and continuity
The first post on discount usability testing looked at organising a day of user research. The output of such a day is typically 5 or so screen recordings + verbal commentaries and maybe some notes. Recording sessions is easier than taking notes when following a script and attending to the subject’s verbal and non-verbal behaviour. Giving observers post-its to note down their observation helps with highlighting significant results and engaging the team.
Preparing a subject involves describing the whys and hows of thinking aloud, and affirming that what they find difficult is as helpful as hearing about what works well. But commentating whilst trying to work something out isn’t a natural behaviour. Encouragement and prompting are often needed, at which point reflecting back what the subject’s just done and asking what they’re trying to do, helps to clarify the issue and get them back on task
Observer: “Is that an appealing deal ?”
Subject: reading silently
Observer: “You’re reading that carefully, what are you thinking, are you looking for something ?”,
Subject “I can’t see if “hotel offers” include passes for the rides.”
reflecting back to user adds information to the recording that is useful for writing up.
Results and analysis
Reviewing and recording results takes headphones and about as many hours as was spent testing.
Without smart tools like Dovetail and Aurelius issues and notes can be logged on a spreadsheet which collates feedback by task and subtask. It can also include time stamped links to exact places on the videos or reference the transcripts.
Also noting what was liked and worked well helps to balance the feedback and engage the team.
Self-organising teams do what works best for them.