When a lawyer is in court, presenting his case, he or she is told to not ask leading questions, as to not elicit testimony the witness would not otherwise volunteer. The same is true for usability testing, whereas you do not want to lead your participant into the answer.
“You don’t want to give them the exact information they need to accomplish the task, otherwise you’re not really testing the product.”
— Paul Sherman, Creating the Testing Plan, KSU UXD
Usability testing questions, need to be structured in such a way that you can determine if the user can successfully accomplish a task, complete the task in a roundabout way, or if the user cannot complete the task at all. This will impact your future design iterations. If you provide the participant with leading questions, to elicit correct answers, the product will not be accurately tested.
Much in the same way, if a student taking a test is provided the answers, he or she does not learn the topic and when asked in the future about said topic, he or she is clueless.
Tasks should be centered around main areas such as, but not limited to: what’s important, critical areas, frequently used areas, error-prone areas and so forth.
When learning about the crafting of tasks, I recall a game I play with my youngest daughter, she is six. It is called the color game. She says to me, “Mommy, I am thinking of a color, guess what it is! It is purple.” I then say in return, “Sweetie, is it purple?” She responds, “YES!” excited that I got the answer correct.
While we want the answer to be correct, we want the user to find the answer on their own.
In usability testing, we need to make sure that we allow the user to complete the tasks and navigate the product. If they ask questions, we need to be sure to redirect them and say, “why don’t you tell me?” or other prompting questions, that do not lead them to the answer.
If not, our results will be purple and in contempt.