Usability Test Plan Template

Usability Test Plan Section

Test Plan for: Product/Application Component

Persona Type Tested: Primary

Context: at home, updating playlists
Duration: 5 minutes

Complete the following before starting: Greeting and Consent & User Background Information.

Overall Objectives
  • Assess the overall effectiveness of the U play3D application for users who are interested in discovering new bands, organizing their discoveries into new playlists, listening to music and sharing their findings with friends;
  • Identify obstacles to completing the creation of a new playlist, the addition of songs to it, playing a playlist and sharing it;
  • Have an evidence of the users that used the available Tutorial in order to do this task and ask them if it was helpful;
  • Evaluate the time it takes users to perform essential tasks such as playlist creation, songs addition, playlist sharing and listening;
  • Test if the terminology and the labels make the application easy and clear to use;
  • Test the systems response time
Scenario
  1. The user wants to search for a band that was recommended to him - "Imagine Dragons".
  2. To accomplish this, he accesses the application via his Google account. He enters his credentials in the login page.
  3. After authentication, the main menu of the application is visible.
  4. The user searches for the "Imagine Dragons" band.
  5. He looks through the results and selects three songs from each search category which he adds to a new playlist.
  6. After the playlist was completed, he starts listening to it.
  7. Realizing he really likes this new band, he decides to share this playlist with his friends on Twitter.
Tasks
  1. Authentication: 30 s;
  2. Search for the "Imagine Dragons" band: 30 s;
  3. Skim through the results: 30s;
  4. Pick three songs for every search result category and add them to a new playlist entitled "Imagine Dragons-playlist": 1 m;
  5. Play the newly created playlist: 15 s;
  6. Listen to a song for a couple of seconds then proceed to the next song: 1m 30s;
  7. Share the playlist on Twitter: 45 s;
Post-test Questions

These questions could be presented to the user as a questionnaire or verbally asked.

  1. Question 1: How easy was it for you to search for a band?
  2. Question 2: What did you think about the manner of presenting the search results? Anything you would change about that?
  3. Question 3: How easy was it for you to navigate through the search result categories?
  4. Question 4: How easy was it for you to add songs to a playlist?
  5. Question 5: Did you find the available tutorial helpful in performing this tasks?
  6. Question 6: Was the terminology understandable for you?
  7. Question 7: Was the systems response time satisfactory?
Notes for Test Coordinator
  • Remind the user that you would like them to 'think aloud' so that you can understand what they are thinking, about the task or the product/application;
  • Read the user the scenario for the test;
  • Read the user the tasks he has to complete. If the tasks are complex, you may want to put a sheet of paper in front of the user with each task written out;
  • Record the user's actions, comments, questions, and body language (write them in the Test Observations section accompanied by video/audio).
  • Don't offer help; let the user attempt to perform the task. If they ask for help reply with: "What do you think you/that would do?" "What do you think that means?".
References

Test Results for: Product/Application Component

Success Criteria

A successful design has been achieved when:

  • 50% of users complete the tasks before the allocated time.
  • 80% of users complete the tasks without the need of any external input.

Summary

  • The users became more familiar with the application layout, and most of them completed the tasks before the allocated time.
  • Search navigation is confusing for some users.
  • Searching depends to unwanted degrees on the tutorial.
Demographics
User Number Gender Age Range Persona Type Technical Level
User 1 male 20-25 primary medium
User 2 male 20-25 secondary medium
User 3 female 20-30 primary medium
User 4 male 20-30 primary medium
Interaction Notes
User Completion Time General Observation
User 1 2 minutes Compared with the previous tests, the user had no major difficulties completing the tasks.
User 2 4 minutes The user generally became more fluent with the application workflow and completed the tasks in a relative short time.
User 3 2 minutes There were some ambiguities regarding the term of search category.
User 4 3 minutes The most challenging process was that of searching. The fact that the searching area is not always visible and can be accessed either by mistake or by closely following the tutorial was an aspect which did not appeal to the user.
Post-test Questions Answers
User 1
  1. Relatively easy, the only issue was how to reach the artist sub page.
  2. The only confusing flow was how you search for items. You enter the search query on the right hand side of the page, but you view the result in the left panel. Switching between the two can be confusing.
  3. Relatively easy.
  4. The only issue was how to add a new playlist from the song menu. The input for the playlist name has no real indication that it creates a new playlist, which will be entitled according to the given input.
  5. It explained all the major tasks that had to be done.
  6. No terms were ambiguous.
  7. The system response time met the users aspectations.
User 2
  1. The input for the search query is hard to find. It was expected that the input would be centrally placed where the rest of the search results are, and not somewhere on the right hand side.
  2. Considering the amount of resulted information, the manner of presenting it is simple and not overwhelming.
  3. The artist subsection of the search results was hard to find.
  4. No issues met, especially with the arrow navigation feature.
  5. It was very helpful and had all the information needed to complete the tasks.
  6. The terminology used was clear and suggestive.
  7. Even when the search sessions number becomes considerable, the loading time of the search results is satisfactory.
User 3
  1. Searching for any content was easy because the tutorial explained how such a task was supposed to be handled.
  2. The aspects that brought confusion to the user are related to the secondary results which are hidden initially and how to increase the number of visible secondary results.
  3. Navigation was performed with ease.
  4. Adding songs to a playlist was extremely easy.
  5. It contained essential information about the tasks that needed to be performed, but these pieces of information were presented at a fast pace.
  6. The used terminology was very understandable.
  7. The systems response time raised to the users standards of acceptability.
User 4
  1. The user did not find the task of searching easy to perform. The problems surface from the way the search input area becomes visible and then disappears.
  2. The way of displaying the search results was satisfactory.
  3. The task of navigating through search categories was completed without encountering problems.
  4. The task of adding songs to a playlist was straightforward and easy to accomplish.
  5. If carefully followed, the tutorial provided the information needed to accomplish the tasks.
  6. There were no obstacles in understanding the utilized terminology and labeling.
  7. The user did not find the systems response time disturbing.
Test Observations

As for the previous tests, the users became more fluent with the general application workflow, being able to complete the tasks easily and with no external input. Most of the users completed the tasks before the allotted time, a testament to how accessible the learning curve of the application is.

Navigating between the search result pages was the only setback for the users. Most of them had a hard time finding how to choose the correct sub-category of the search items, and what each individual related item list meant.

Potential Design Improvements

The users had no major difficulties completing the tasks, as they became more in tune with the application workflow. We feel that the initial hurdles for the users were caused by the poorly executed tutorial, and not the flow itself. We believe that if we improve how the tutorial is presented to the user, and make the information more accessible, we will mitigate all the initial problems that the users had.

We should also investigate why the search categories were hard to find by the users. We should consider changing the layout of the four search categories to make them more apparent to the user.