Measurement Overview

Measurement of the tasks were both qualitative and quantitative. The official report contains the expressed user feedback, along with the success rates of the tasks among users. 

USER PERSPECTIVE

Step into the users’ shoes and watch through the user’s eyes, as they navigate the three tasks set before them.

  • Listen as they verbalize their likes, dislikes and frustrations. 

  • Explore the top video clips for each of the three tasks, along with a few discovery points, revealed in each along the way in this short video (00:02:47), below.


TASK SUCCESS MEASUREMENT

As a quantitative measure, each task was analyzed based on the success and failures of how individual users were able to accomplish each task using the current interface.

Task Scoring Key

Task Success Definitions:

  • Located quickly, preferred method = 1

  • Located in a roundabout way = 2

  • Could not locate = 3

The scores were averaged to determine the overall success rate of the task, in order to convey if there is a need to improve a section of the interface. 

REVIEWING THE CHARTS

Overall, two out of three tasks were successful; and one out of three tasks was not successful.

  • In this usability study users were successful 11 out of 15 times in the three tasks.

  • While users were unsuccessful four times throughout the three tasks.

Reviewing the time on task data reveals the bumpy path for some to success. 

  • Task one, while mostly unsuccessful (3 out of 5 failed), it took the successful users (2) more time to complete the task than those who failed.

  • Task two was successful, although users completed this task in a roundabout way.

  • Task three was successful with those failing taking the most time on task. 

 
 

Task Findings detail

The following links detail the findings by task in the full report and conclusions that can be drawn from the successes and failures. 

 

OPPORTUNITIES FOR CHANGE

As displayed in both the quantitative and qualitative testing results in the full report, the ease, disappointment, disruptions and positive experiences of the users differed during the usability tests, providing actionable insights, which can become opportunities for change.

Future testing sessions will take place to identify if the implemented changes are making a measurable difference. 

Key Findings

  • Filtering bug removes filter, and displays unfiltered results, with some users not realizing it was removed. 

  • Result counts conflict when looking for books on Friendship. 

  • Additional filtering options desired by users in Best Book List section.

  • Users experienced issues with navigational terminology. 

  • Users expressed they liked the age filtering option and the site overall. 

Primary Recommendations

  • Additional filters should be added to “Book Lists” section. 

  • Fix the functionality of existing filters in “Book Lists” section.

  • Include a filter for reading level and/or Lexile score. 

  • Additional testing is needed to evaluate genre and topic similarities.

  • Content analysis needed to evaluate “Books” secondary navigation terminology.

  • Card-sorting exercise is needed to address slim navigation under “Books.”

  • Reduce pop-ups on the site. 

  • Further evaluate and conduct additional testing on search icons.