Rely on Actions, Not Words

By leelefever on November 24, 2004 - 3:03pm.

• Categories:
• Online Usability
• Comments: 0

I’ve been through a whirlwind of usability testing and focus groups for the past 3 weeks. We tested 3 concepts across 15 users and 2 focus groups. It’s a good thing we’re using paper prototypes or we couldn’t do it- we can barely keep up on paper.

Our very last test yesterday was fascinating to me. We tested 7 people on the same concept and all but the final person “got” the concept – they could describe the core functionality and how they would use it. What made the last user interesting was not that he didn’t “get it”; it was the gap between his words and actions.

If we were to show you a transcript of what he said about the product, you would think he got it and was really impressed. He had great things to say about the “potential” and said things were “great” many times. Never did he utter one word of confusion or dismay.

Yet, his actions told a different story. In observing him use the tool, you could see evidence of apprehension and confusion. Even as he spoke positive words, you could tell he wasn’t in tune with the core concepts that we were testing. It didn’t speak to him in the way it did to others.

As a tester, this was a great experience. Having gone through 6 others that got it, this one user made us look more closely at our assumptions about who else will get it. He showed us that some people will not get the current design and we have a more work to do. Also, it was a perfect example of why testers need to rely on actions and not words.

Here are my lessons from this session:

• Users sometimes want to please the testers and this desire can mask their actual feelings.

• Depend on what you observe, not what the participant says.
• Always be prepared for a user to come along open your eyes to new perspective that diverge from what you consider the “norm”.
• When someone doesn’t get it, don’t make it personal- focus on why the product is not communicating value or context to this type of user.

Happy Thanksgiving to my fellow Americans out there!