Usability Test Plan Template

Usability Test Plan Section

Test Plan for: Product/Application Component

Persona Type Tested: Primary

Context: at home, updating playlists
Duration: 2 minutes 10 seconds

Complete the following before starting: Greeting and Consent & User Background Information.

Overall Objectives
  • Assess the overall effectiveness of the U play3D application for users who want to manage their playlists and the selection of currently playing songs;
  • Identify obstacles to completing playlists and player management;
  • Have an evidence of the users that used the available Tutorial in order to do this task and ask them if it was helpful;
  • Evaluate the time it takes users to delete a playlist, a song from a playlist, a search session, a song from the player.
  • Test if the terminology and the labels make the application easy and clear to use;
  • Use the References to guide you in writing your objectives.
Scenario
  1. The user wants to clean up his playlists collection.
  2. To accomplish this, he accesses the application via his Google account. He enters his credentials in the login page.
  3. After authentication, the main menu of the application is visible, including the "Playlist" section.
  4. The user navigates to his first created playlist and decides to delete it, since his music taste has evolved.
  5. He then picks another playlist and realizes that one of the songs does not go well with the rest of them. He deletes that song.
  6. Because he wants this playlist to have a specific number of songs and he has deleted one, he picks another song from the playlist and searches for similar songs.
  7. From the similar songs he adds one to the playlist, then deletes the created search session.
  8. He plays the playlist.
  9. He deletes from the player the first song.
Tasks

Write out the tasks for each persona type. Avoid complex tasks. Make sure the user is able to complete the test in the specified time interval. Assign an estimated completion time for each task.

  1. Authentication: 30 s;
  2. Navigate to the first playlist and delete it 30s;
  3. Delete a song from a playlist: 10 s;
  4. Search for a song similar to one from the previous playlist: 10 s;
  5. Add a similar song from the results to the playlist: 20 s ;
  6. Delete the created search session: 10 s;
  7. Play the previously modified playlist: 10 s ;
  8. Delete from the player the first song: 10 s;
Post-test Questions

These questions could be presented to the user as a questionnaire or verbally asked.

  • Question 1: How easy was it for you to navigate through existing playlists?
  • Question 2: Did you find the available tutorial helpful in performing this tasks?
  • Question 3: Was the terminology understandable for you?
Notes for Test Coordinator
  • Remind the user that you would like them to 'think aloud' so that you can understand what they are thinking, about the task or the product/application;
  • Read the user the scenario for the test;
  • Read the user the tasks he has to complete. If the tasks are complex, you may want to put a sheet of paper in front of the user with each task written out;
  • Record the user's actions, comments, questions, and body language (write them in the Test Observations section accompanied by video/audio).
  • Don't offer help; let the user attempt to perform the task. If they ask for help reply with: "What do you think you/that would do?" "What do you think that means?".
References

Test Results for: Product/Application Component

Success Criteria

A successful design has been achieved when:

  • 50% of users complete the tasks before the allocated time.
  • 80% of users complete the tasks without the need of any external input.

Summary

  • The users became more fluent in the application as they completed more tasks;
  • Some gestures explained in the tutorial were missed by the user;
Demographics
User Number Gender Age Range Persona Type Technical Level
User 1 male 20-25 primary medium
User 2 male 20-25 secondary medium
User 3 female 20-30 primary medium
User 4 male 20-30 primary medium
Interaction Notes
User Completion Time General Observation
User 1 4 minutes Compared with the previous tests, the user had no major difficulties completing the tasks.
User 2 3 minutes The user generally became more fluent with the application workflow and completed the tasks in a relative short time.
User 3 3 minutes Navigation between playlists was not performed with ease.
User 4 2 minutes 30 seconds The user was confused about the meaning of a "search session" and was not confident that he had completed what was required by some tasks.
Post-test Questions Answers
User 1
  1. It was relatively easy if you closely follow the tutorial.
  2. Yes, they were very helpful.
  3. The most confusing term was Search Session and what it meant to navigate between them.
User 2
  1. No issues had.
  2. Missed the fact that a playlist can be navigated using the arrow keys, which would have made navigating more easily, but generally it was very helpful.
  3. Search Session was the term which raised question marks.
User 3
  1. The task of navigating through existing playlists was a bit challenging because the user depended on the item bar to perform it.
  2. Missed the fact that a playlist can be navigated using the arrow keys, which would have made navigating more easily, but generally it was very helpful.
  3. The terminology used was clear.
User 4
  1. Navigating through playlists was easy.
  2. The tutorial was helpful and informative.
  3. The novelty element, that of having displayed the results from more than one search, was confusing. In particular, the user could not graps the meaning of the term "Search Session".
Test Observations

The users have become more fluent in the application workflow, which became apparent in how easily they completed the tasks in the allotted time. It became apparent to us that the learning curve of the application isn't as steep as previously thought, and with the proper improvement of the initial tutorial, to make features and gestures more apparent, the user can become productive in the application in a relative short time.

The only issue consistent with the users was the term "Search Session", and we might need a way to devise a better way for the user to understand how to search and navigate through previous search results.

Potential Design Improvements

As with previous tests, the users struggled with certain tasks which were explained during the initial tutorial. Most of the users complained that there was too much information to take in all at once, and we might have to split the tutorial into sub-parts to give the user less information to take in at any given time.