Preparing for 5

Understanding usability testing 101, describes how “discount testing” with 5 people and no lab goes a long way toward identifying usability issues. The practical benefits are: –

  • 5, 1 hour sessions constitute a good day’s work
  • Its simplicity facilitates the good practise of testing early and often

Of course there’s more to usability testing; but understanding a simple methodology’s strengths and limitations, and being able to do it, is a good start.

This post describes 5 practical considerations to prepare before a day of testing.

 

1. Stakeholder knowledge and opinions 

For an existing product, marketing and customer support will have a great deal of relevant information and be in touch with users. They are often motivated to help, and work in the same building.
That said, stakeholders who are already familiar with a product or service usually have pre-conceived ideas about how it should be improved. These need to carefully unpicked and substantiated. (Jared Spool talking about engaging stakeholders). Though their insight and cooperation are invaluable, being familiar with something can lead to subjectivity and away from the users’ perspective.

 

1. Identifying the target audience

Audiences can be segmented by demography (e.g. age, gender), access (e.g. computer literate, online, device), circumstance (e.g. fostered, attending a clinic), ability (language skills, literacy),

Webstats are useful for challenging opinions and informing how a site’s actually being used, and by whom. They also describe the audience’s location, device, OS, age, and even gender. But of course they don’t say much about prospective users in new markets.

Exemplar data from Google analytics
Exemplar data from Google Analytics

 

2. Recruiting and organising

There are market research and usability testing agencies who will recruit, select, schedule and track attendance, they will also conduct research for you.

When recruiting participants in-house, it helps to think about why someone might want to participate. Whether they might be motivated to improve the product or service for themselves and others, incentivised by vouchers and discounts, or happy to gain kudos by having their contribution acknowledged or recognised in a community. Advertising might mention such things, alongside describing what’s involved and how to enrol.

Screen recording software enables testing to be done remotely moderated or unmoderated. When setting up a programme of research, it’s helpful to first step back and think about the requirements, approach, constraints and outputs. A research plan needn’t be long, and can be updated in stages.

If research is done in-house, then think about a light CRM programme to capture subjects’ contact information, preferences, availability and participation etc. Depending on the level of engagement, quite a large feeder group might be necessary to get 5 people from the target demographic to regularly participate.

Recruiting and organising can take a lot of effort and participants might just attend one usability test. To maximise the return on your investment, it’s worth thinking about other types of research they might also engage with. Such activities (e.g. a survey) can also be useful for keeping people interested who might have signed up some time ago.

 

3.  Reminding

A few days before testing, it’s worth reminding participants about the time, location, travel options, recording setup, and any due care considerations (e.g. bringing someone along for support). This will probably be their first time, so an overview of what it’s about and what to expect might also be helpful: –

  • What their input will help achieve
  • What they need and needn’t bring
  • What’s the research will involve
  • Where it is, directions and who to ask for
  • A contact number in case of difficulty

Having subjects arrive in good time, and orientated of course helps the day run smoothly.

 

4.  Safety and due care

Safety confidentiality, consent, recording, chaperoning, and premature termination, are difficult to specify as they’ll vary according to the product and the environment, that said –

  • Recordings can be named and referenced by an anonymous but useful
    convention
    e.g. “HMRC_20170203_01”
  • Data protection legislation is relevant
  • Alongside screen activity, a dictaphone running all the time safeguards everyone and acts as a backup for the session
  • Participants usually quickly forget about observers in the room
  • If for whatever reason a session isn’t going well, It’s polite and reassuring to ask someone if they’d like to stop
  • Just being in a strange and unfamiliar environment is stressful, so building rapport in the run-up and when they arrive helps to relax and reassure them.
OMG
I guess we all know that feeling.

 

5. Scripts

If there’s time, script writing can be a “team sport”, though the end result needs to be simple, coherent and readily comprehensible. One advantage to the team collaborating on the script is that it fosters a sense of
ownership and so increases the likelihood of results finding their way into the codebase.

  • Clarity comes from editing and refining a script until it’s a clear
    narrative, that tracks a straightforward user flow and peppered with simple, intuitive,
    tasks and questions
  • A “slim” script helps everyone stay on task, whilst affording time to investigate
    interesting things that might arise
  • A simple script is more accessible especially given the average reading age of UK adults
  • Plus it’s hard to read from a page and observe someone
  • Here are some “Interesting things” about writing clearly that are also relevant to script writing
  • Encourage participants at the start to be critical e.g.  “We’re here to test some early ideas not you, so please be frank and speak as you find.”

 

Leave a Reply

Your email address will not be published. Required fields are marked *