Usability Basics: User Testing Questions

By running tests, designers can identify problems in the early stages of projects before they are coded, compare their existing site to competitors, and evaluate a site before redesign. As well as providing concrete evidence of what people are doing on websites, usability testing gives compelling insight into why. If UX professionals aren’t asking the right questions, they won’t get the answers they need.

According to Usability.gov, a leading resource for UX best practices, usability testing refers to “evaluating a product or service by testing it with representative users.” UX professionals use usability testing to identify problems, collect data, and determine overall user satisfaction.

In any successful user experience study, UX professionals need to write questions that will provide insight to improve their website or move a project forward. UX professionals should put a lot of thought into these questions so that they can trust the results and use them to guide future decision-making.

Qualitative Testing

UX professionals should incorporate qualitative testing. “Studies that are qualitative in nature generate data about behaviors or attitudes based on observing them directly, whereas in quantitative studies, the data about the behavior or attitudes in question are gathered indirectly, through a measurement or an instrument such as a survey or an analytics tool,” according to the Nielsen Norman Group.

Qualitative testing can enable designers to focus on the issues that have the biggest impact on users. It helps teams answer questions related to “why or how to fix a problem,” the Nielsen Norman Group explains.

Participants

Five to seven participants are typically needed to uncover the majority of problems. In fact, Nielsen Norman found five participants can uncover 85 percent of usability issues. Usability.gov notes that it is important to choose participants who align with typical users of your site. Depending on your site’s specifications, your audience might be broad or relatively narrow. If you have multiple potential user groups, include participants that represent each so that you get useful feedback. You shouldn’t use internal staff as participants because this can skew results.

Stakeholders should also be invited to observe the testing process. This boosts the credibility of your findings and reports and helps teams build empathy for users. It also gives context, increasing the likelihood that they’ll act on your recommendations based on the test results.

Ask the Right Questions

The following are some core best practices when writing usability test questions. They focus on eliminating bias and gaining real insight into how users interact with your design.

Understand the Areas of the Product That Need Answers

In order to ask the right questions of your participants, you first have to know what questions the business needs answered. The right questions aren’t just any questions, they are questions that will help you and your team move forward. So your first question should actually be one you ask yourself: “What do I want to learn?” In addition to helping you figure out outcomes, your answer can also help you decide how to use the results.

Don't Ask Leading Questions

Though it’s tempting to suggest answers to your users, do your best not to lead the conversation in a particular direction. Don’t influence the user, because you need open, truthful answers to improve your design. Leading questions defeat that purpose. For example, ask “How much would this product cost you? How easy or difficult was it to find that information?” rather than “How easy was it to find the pricing page?”

Ask Open-Ended Questions

Open-ended questions enable you to “give your test participant minimal explanation about how to perform the task or answer the question,” User Testing Blog says. That way, you can observe as participants navigate coming up with a solution on their own. Open-ended questions can’t be answered with a simple “yes” or “no.” They’re beneficial because they require users to provide an explanation, giving you even more insight into their thought processes. In general, they help you see how your user thinks, as well as noting typical user behavior.

Break up Complex Concepts into Multiple Questions

When you are covering complicated or overly technical aspects of your design, break up questions so the user doesn’t get overwhelmed. Instead, “break concepts up when you’re asking the questions and put them back together when you’re analyzing the results,” User Testing Blog says.

Don't Use Jargon

Technical terms don’t resonate with most users, so you’ll want to avoid them as much as possible. When you have to include them, it’s best to define terms within the question itself so that your user has enough context.

Ask "Show Me" Questions, Not Just "Tell Me" Questions

This can give you valuable insight into how a user actually navigates your site. Rather than solely asking your participants to explain their choices to you, watching them move around your site in real time can show you functionalities that don’t work well or aren’t as intuitive as they could be. One example is asking, “Show me where you might look if you wanted a new car that could seat six people.”

Don't Answer Participants' Questions

Instead, ask them questions back. This is because “introducing new information or giving unintentional clues can bias the research results and even invalidate that session’s data,” the Nielsen Norman Group says. Instead, say things like, “That’s a good question, what do you think?” or “I’m not sure, where might we look to find out?”

Don't Ask Participants for Solutions

The best user testing questions ask participants what they were looking for and what they expected to happen. Sometimes participants have trouble articulating what they found confusing. It’s the responsibility of the moderator to understand what was confusing them. He or she can typically find the source of participants’ confusion by asking them why they did something and what they thought would happen.

Lesley University’s online BS in Design for User Experience program prepares students to create meaningful, people-first design. The curriculum focuses on graphic design principles and collaborative learning, teaching students to design intuitive experiences across an ecosystem of products, services, and systems. You’ll learn to work through all aspects of the design process, from testing to product launch.

Related Articles & Stories

Read more about our students, faculty and alumni.