Usability Test Plan Template

Usability Test Plan Section

Test Plan for: Product/Application Component

Persona Type Tested: Primary

Context: at home, discovering new music
Duration: 3 minutes 10 seconds

Complete the following before starting: Greeting and Consent & User Background Information.

Overall Objectives
  • Assess the overall effectiveness of the U play3D application for users who want to view music charts and create playlists based on them;
  • Identify obstacles to completing playlists and player management;
  • Have an evidence of the users that used the available Tutorial in order to do this task and ask them if it was helpful;
  • Evaluate the time it takes users to add songs to a playlist.
  • Test if the terminology and the labels make the application easy and clear to use;
  • Test the systems response time
  • Use the References to guide you in writing your objectives.
Scenario

1. The user wants to create a playlist based on a chart.

2. In order to accomplish this, he accesses the application and logs in using his Google account.

3. After authentication, the user navigates to the "Chart" section and opens the chart for May 25th 2013.

4. After that, he adds the top three songs to a new playlist.

5. For each of the three songs added to the playlist, he searches for similar ones.

6. The user adds to the newly created playlist three similar songs from each search session.

7. He then plays the playlist.

8. The user rearranges the songs from the currently playing playlist so that the last three songs become the first three.

9. He then creates a new radio station using one of the songs added to the playlist.

Tasks
  1. Authentication: 30 s;
  2. Open the chart for May 25th 2013: 20 s;
  3. Add the top three songs to a new playlist: 20 s;
  4. For each song added to the playlist, search for similar ones: 20 s;
  5. Add to the newly created playlist three similar songs from each search session: 60 s;
  6. Play the playlist: 10 s;
  7. Rearrange the last three songs from the now playing playlist by making them the first three ones in the list: 10 s;
  8. Create a radio station based on one of the songs added to the playlist: 20 s;
Total Time: 3:10
Post-test Questions

These questions could be presented to the user as a questionnaire or verbally asked.

  • Question 1: How easy was it for you to open the desired chart?
  • Question 2: How easy was it for you to add songs to a playlist?
  • Question 3: Did you find the available tutorial helpful in performing this tasks?
  • Question 4: Was the terminology understandable for you?
Notes for Test Coordinator
  • Remind the user that you would like them to 'think aloud' so that you can understand what they are thinking, about the task or the product/application;
  • Read the user the scenario for the test;
  • Read the user the tasks he has to complete. If the tasks are complex, you may want to put a sheet of paper in front of the user with each task written out;
  • Record the user's actions, comments, questions, and body language (write them in the Test Observations section accompanied by video/audio).
  • Don't offer help; let the user attempt to perform the task. If they ask for help reply with: "What do you think you/that would do?" "What do you think that means?".
References

Test Results for: Product/Application Component

Success Criteria

A successful design has been achieved when:

  • 50% of users complete the tasks before the allocated time.
  • 80% of users complete the tasks without the need of any external input.

Summary

  • The majority of the users could not follow the tutorial until the end, and consequently struggled with some of the tasks which are explained in the tutorial.
  • The radio is confusing to some users.
Demographics
User Number Gender Age Range Persona Type Technical Level
User 1 male 20-25 primary medium
User 2 male 20-25 secondary medium
User 3 female 20-30 primary medium
User 4 male 20-30 primary medium
Interaction Notes
User Completion Time General Observation
User 1 6 minutes The most difficult tasks for the user were opening the charts at the desired date and creating a radio with personalized criteria.
User 2 5 minutes The only problematic task was creating a personalized radio, as the user had a hard time understanding that the radio is exactly and what can it be used for.
User 3 4 minutes Difficulties were encountered at opening the chart, the user finding the manner of time selection confusing and at creating a personalized radio, considerable time spend on discovering how to add criterias.
User 4 7 minutes 30 seconds The most time consuming task proved to be that of adding the top three songs from the chart to a playlist, since the user did not see the available options for every song for the start. Other issues surfaced when the task was to add criteria for radio generation.
Post-test Questions Answers
User 1
  1. No difficulties had, apart from identifying the correct section.
  2. No problems had.
  3. Some sections of the tutorial were confusing and didn't help much in completing the tasks.
  4. The charts section uses terminology that was not very clear.
User 2
  1. Finding the date picker for the charts was confusing. Also, there were too many options for choosing the date of the chart.
  2. No problems had.
  3. The tutorial contained too many sections and information which needed to be absorbed quickly, which might make new users abandon the tutorial half way or skip it entirely (which is a problem as the website is not intuitive without closely following the tutorial).
  4. There were some difficulties regarding terminology related to the radio sections and what a custom criteria is.
User 3
  1. Concerns regarding transparency and overlaying made it hard for the user to be aware of the current selection.
  2. No problems had.
  3. The tutorial was fast paced and required close attention. It did not contain detailed information about some sections that where covered in the test (chart, radio).
  4. Although the majority of terms and labels used were suggestive, some problems involved terminology used in the radio section.
User 4
  1. The task was completed with speed and confidence.
  2. Adding songs to a playlist was made difficult, because the user did not see the available options for the songs.
  3. The tutorial does not cover most of the tasks in this test.
  4. Terminology used in the radio section raised problems.
Test Observations

The general feedback from the users with their first contact with the application is that the general navigation is confusing without closely reading the tutorial.

Also, almost all users struggled with the radio section, by either not finding what a radio criteria is, how to add it, or what the actual radio is.

Potential Design Improvements

As most of the users struggled with the tutorial, we could clarify to the user that the general workflow of the application might be confusing for first time users if they do not follow the tutorial closely. We should also add an option to re-take the tutorial if the users find that they are struggling with a particular task.

A slight redesign of the radio page is also needed to make adding criteria more intuitively.

One of the users suggested that we should change the icon of the play button when paused, as the current approach (making the button transparent) is not obvious enough and the standard pause symbol (two vertical bars) is more appropriate.