How to Create an Effective Usability Testing Plan?

how to create a usability testing plan

The very first step in conducting usability testing is creating a plan. This plan will guide you from the beginning to the end of the test and help you not to get stuck and waste your time. Conducting a usability testing plan helps UX researchers to manage their time and resources and understand what’s the next step. 

In this article, we’re going to explain the usability testing plan step by step. 

Note: One of the best ways to conduct usability testing is using session reply software like WatchThemLive. Keep reading to find out more.

Steps to Create a Usability Testing Plan

For creating a usability testing plan, you should take several steps. We’re going to review them here.

1. Determine the Product You Want to Test

Determine the website, app, web app, product, or the part of any of them you want to test. Indicate if you’re testing a prototype version of the product, content, navigation, etc. 

2. Define the Purpose of the Test

Specify the questions you want to be answered. For example, “Can users easily find the support button?”, “Can users navigate to the x page without any problem?”, “Is anything distracting users from completing tasks?” 

Or maybe you have a hypothesis in mind that you want to confirm. 

By specifying these questions, you are determining the purpose of the test and what you’re expecting to gain from it. 

3. Determine the Location of the Study

You can conduct usability testing in three ways:

Moderated or Unmoderated 

  • Moderated usability testing: Moderated usability tests can be conducted in a lab or online in the presence of a moderator. This can be done using a conference tool or telephone. The moderator explains the purpose of the test, answers, and asks participants questions. Since there is direct communication with the sample group, you are able to collect in-depth data. However, this method is a bit expensive because you need time, resources, sometimes a lab, an expert moderator, and in some cases, compensation for participants. 
  • Unmoderated usability testing: There is no direct interaction with participants in this type. Researchers ask participants questions in a remote testing platform or record their voices and screen while completing tasks. You can conduct unmoderated usability testing without having to spend lots of time and money; however, the information might not be very accurate. 

In-Person or Remote

In-person usability tests are a great source of data compared to remote ones. But it’s usually hard for companies to run in-person tests because it’s hard to travel or users can’t travel to where you are. So they prefer remote usability testing most of the time. 

One of the best ways to conduct remote usability tests is using session replay software like WatchThemLive which records visitors’ screens as they interact with your website. It also gives you other data such as participants’ screen resolutions, browsers, operating systems, device types, etc. 

Sign up here and seize session replays now! 

In Lab or in Field

Most of the time the test is conducted in a lab, but sometimes it’s needed to see participants use the product in their own environment. 

Based on the complexity of the tasks and the data you need, you can choose one of these ways to conduct your test. 

4. Choose Your Equipment

Determine the kind of device you’re going to use. It could be desktop, mobile, laptop, or tablet. Also, include the information about participants’ screen resolutions, monitor size, browser, operating system, etc. 

5. Recruit Participants

Don’t just recruit any participant you find. Your participants should be from your target audience that represent your whole audience. They should match your demographics, psychographic segmentation, and personas. You can recruit the participants from your different channels, such as social media platforms. You also have to motivate them to participate by offering free stuff, cash, or gift cards. A way of easily recruiting participants is using remote usability testing tools.  

Related Article: How to Recruit Users for Usability Testing: Advice No One Will Give You

6. Choose the Metric You Want to Measure

Without measuring usability metrics, your work will be based on guessing. But by measuring them, you will make data-driven decisions. There are two kinds of metrics:

  • Quantitative metrics: The numeral data you gather is quantitative metrics. This includes the number of task completions, time on task, error rates, etc. 
  • Subjective metrics: These metrics are participants’ answers to your questions and show their satisfaction, ease of each task, if they’re going to recommend you to their friends or not, etc. 

A way to gather data for usability testing metrics is using a behavioral analytics tool like WatchThemLive. ًWith this tool’s session replay software, you will track subjective metrics. For example, you can understand what dissatisfies users before leaving your website or how easy it is for them to complete the task. Basically, you are able to see everything they do on your website. Watch this video to see for yourself:

Also, with WatchThemLive’s web analytics, collecting quantitative data is no longer a hassle. You can see each visitor’s information such as their country, screen resolutions, operating systems, device type, browser, the number of their sessions, the time they spent on tasks, etc.

web analytics

What are you waiting for? Sign up here and see what’s next!

7. Create Test Tasks for Participants

The next step of the usability testing plan is creating tasks and scenarios for participants. In this step, you ask users questions as they complete tasks and interact with your website. 

The script should include:

  • Contextual questions
  • Scenarios and tasks
  • Questions about their experience

Make the participants comfortable with the contextual questions. For example, if you’re running an eCommerce store, ask them, “how often do you shop online?”. This way, you’re collecting extra information about participants and creating a good atmosphere. 

Write a scenario that users can actually see themselves in. Then ask them to complete a task. Here’s an example:

Scenario: Your friend announced that they’re going to have a party in 2 days. You don’t have anything to wear. 

Task:  Open our website, go to the  “dress” category, and purchase our red dress with the 621 code. 

For more examples, read our article on usability testing questions

8. Assign Roles to Your Team Members

You can’t do everything yourself, and you need others to help you. Ask your team members to participate in the usability test and give them roles. The UX researcher needs a note-taker, observer, administrator, etc.

9. Conduct a Pilot Test

One of the important steps in a usability testing plan is conducting a pilot test. This way, you can detect any issues beforehand and test your equipment, script, and questions. You will also have a good idea of how much time the test might take. 

10. Run the Usability Test

You might be thinking, well, of course, I should be running the test now! That’s true, but there are some things you need to keep in mind. Be patient. Answer users’ questions without being judgemental. But don’t really answer their questions for them, lead them to find it themselves. Make sure the participants know that they won’t be hurting your feelings by being honest. 

Remember to use the think-aloud protocol and observe participants’ body language.

11. Analyze the Results

Be sure to put analyzing the results in your usability testing plan. You have to do this before implementing your findings to understand what issues and solutions come first. This prioritizing helps because usually, there is more than one problem found in the usability test. Take a look at this infographic to get quick information:

usability testing analysis steps

12. Report

As the last step of the usability testing plan, you need to report your findings and suggestions. This report shortly summarizes everything you did on the usability testing backed up by qualitative and quantitative data. You can also add your suggestions and discuss them with your team members. This information is needed because one of your team members might want to rerun the test. Take a look at this infographic for quick information:

usability testing report


They say “a goal without a plan is just a wish.” So before running any usability tests, you need a usability test plan. This helps you have a solid structure, know what to do as your next step, and conduct a fruitful test. 

In this article, we presented a usability testing plan with 12 steps that you have to consider.

If you want to begin planning your usability test and don’t know where to start, sign up to WatchThemLive for FREE and take your first step.

Cyrus Nambakhsh
Cyrus Nambakhsh
Cyrus is a serial entrepreneur, product-led-growth expert, a product visionary who launched 7 startups. He has built scalable platforms to help businesses and entrepreneurs. Visit my profile here: ==> Contact: [email protected]
Share This Article