Usability Testing for Your Web Site

by Curtiss   |   last updated September 22, 2009   |   Usability Tutorials ·

For this session, Chisnell began by asking the audience for some of the main points we would like her to cover in the much shorter timespan she has today. To begin, someone asked the generic question about how we should be performing usability testing. I then asked, as advanced or “power” users, how do we, with limited resources, attempt to understand how the standard user will attempt to use our Web site. One other person asked if there were ways to increase or encourage participation in usability studies.

Chisnell then began her presentation. She started off by showing us slides with photos of various usability tests. The first slide showed the “old method,” or, as she calls it, “formal usability testing” (for this slide, she superimposed a photo of someone in a tuxedo standing in front of the test studio. She then began explaining a method of simply asking a single participant to surf your site and let you observe.

A Guinea Pig

She then asked for an audience participant, which ended up being me. She asked me to find the following things:

1. What is the application deadline for University of Pennsylvania?
2. When will I be notified of acceptance?
3. Are there any fees (and, if so, what are they?) to apply at University of Pennsylvania?

She then opened Google in her Web browser and asked me to proceed. I began by searching Google for “University of Pennsylvania application” and moved on from there. After quite a bit of frustration, I found the answers to all three questions.

She then informed me that I was the first of her eight test subjects, so far, to find the application fees (which, by the way, are $75 – I found this by locating an “application fee waiver” form and reading it).

Next, she asked me to summarize my feelings about the experience and then asked me what I liked about the site.

She then asked the audience for their observations of my testing. Someone pointed out that I said as much with my body language as I did with my words, and asked how you are supposed to monitor all of that together. She recommended that, especially when you are beginning your usability testing, to have an additional person there to take notes while you’re performing the test.

She also recommends that you perform a bit of gentle prodding to keep the user on task (reminding the user of the questions they are supposed to be answering) if they begin to get sidetracked (which I did, of course).

Simple is Hard

“Simple is hard,” says Chisnell, explaining that it can be very difficult to create a product that’s easy to use and understand. She used the test with me as the audience participant to show just how much information can be gathered from simply asking someone to perform a few simple tasks on the site. She recommends getting at least four or five different people from each target audience to participate in the testing so that you can start to gather common information.

Guidelines are Good

Chisnell explains that guidelines and best practices for accessibility and usability are a good place to start, but they don’t get you all the way there. You need to use usability testing to gauge the following areas of your Web site:

• Usefulness
• Efficiency
• Effectiveness
• Learnability
• Satisfaction
• Accessibility

You should use this usability testing to identify problems in your design that lead to misinformation, incomplete transactions and the need for support from your staff or administration.

You should perform it multiple times throughout the development and design cycle. The testing should attempt to answer the following questions:

• What should the design do?
• How should it work?
• Does it do what we want it to do?

A Test Plan

Sit down with your team to determine what problems or issues you are experiencing on your Web site. From that information, you should develop a plan or “blueprint” for your usability testing. The plan should include:

• Goals and objectives
• Research questions
• Participant characteristics
• Description of method
• List of tasks

You may not be able to answer all of these questions in a single usability test, but you should at least have them laid out so you know where to go. You need to identify who your participants should be and what characteristics they should have. Then, describe the method through which you will be performing the usability test (will you start them off at Google or on your Web site; will you put them in a studio or meet them in their environment, etc.).

She then shows us some examples of research questions that she used in a usability test previously. She explains that it is important to use open-ended questions rather than using yes or no questions.

When recruiting and evaluating participants, you need to lay out some characteristics of your target users. In most cases, these characteristics will not be standard demographics (such as age, educational background, etc.) but actual breakdowns of the information your users might want from your site.

Outlining Your Method

When outlining your method, it is acceptable to identify the actions or goals you would like the user to achieve. For instance, if you want your participant to visit the main page for admissions or prospective students, and to evaluate the information there, you can write that into your plan.

However, you do not want to phrase the tasks or questions that way when presenting them to your participants. You do not want to lead the participants; you want them to accomplish the task on their own to see if their result matches the results you expected. If they don’t, then you need to figure out why, and consider rearranging your site to improve that experience.

Evidence

• Observing the behavior of the participant
• Comments and quotes of participants
• Whether or not the user was successful

If you notice that the user is stuck on a specific step of the process, you might want to give a very specific prompt to see if you can get the participant to refocus. However, in doing so, you need to keep track of the suggestions you’ve made. For instance, if the user is having a great deal of trouble identifying where the information is on the page, you may notice that they are focusing on a specific geographic region and suggest that they look to other parts of the page. However, if you notice multiple participants getting stuck in the same area, you may need to reorganize your content so that that information appears in the geographic region on which the participants were focused.

She also recommends asking your participants for their thoughts and feelings throughout the process. For instance, at one point, she asked me how close I thought I was to accomplishing my goals.

Conclusions and Questions

Chisnell then shows an even “lite-r” version of the test plan. She explains that a template for the test plan is available on the Web site that accompanies her book. She now opens the floor for questions.

1. You should observe your participants carefully to identify specific behaviors, such as frustration or happiness. You can also provide ratings forms or surveys asking the participant to provide their own observations and rate their experiences with the site and the tasks they were asked to perform. These ratings forms can be especially helpful if you use the same questions throughout multiple phases of the testing (for instance, if you test with the current version of the site, then make changes based on your testing, then perform your test once changes have been made).
2. What kind of value do you place in mass usability testing? Chisnell answers that they are for very different purposes. For instance, analytics can show you that someone is spending a lot of time on particular pages of your site, but that won’t tell you why (are they staying there because it’s great and engaging, or is it because it sucks)? That is a good time to perform a usability test to see if you can find out why.
3. Is it desirable to have your team observe the testing sessions? Chisnell says that it is absolutely desirable to have everyone involved in the design of the site observe the sessions so that they understand how their contributions can help improve the usability. A designer can actually conduct the session, but they have to be able to stay neutral and quiet during the test, and separate themselves from their work on the project. A Web site is not a work of art. It is a constantly evolving tool that needs to be changed over time. As a designer, you should understand that and realize that it needs to change over time.
4. How important is it to use people at whom your site is targeted rather than using people that are easily accessible for your usability testing? For instance, if you have some folks in your office that are generally good at providing feedback, is it okay to use them for your testing? Chisnell explains that that is perfectly acceptable if your site is designed for that person. However, if it’s designed for a different type of person, you should gather participants from those groups rather than just using people within your team. It’s okay to use someone within your team as a rehearsal, or to get initial feedback, but you should not use them for your actual usability testing.

Chisnell did a fantastic job of compressing her six-hour session into a 75-minute summary. The information gathered from this session will be extremely helpful, and I hope it is also useful for you.

I am currently attending a conference targeted at Web professionals at colleges and universities. The conference is known as edUi 2009 and is being hosted at the University of Virginia in Charlottesville. I am attempting to live-blog from the sessions I am attending today. This is one of those posts.