Readers of these articles know that I view the high-fidelity prototype as the primary means of describing the product to be built. I have written elsewhere why a prototype is significantly more useful to the product team than the typical paper-based specification. However, that is really the secondary benefit. The primary reasons to create a high-fidelity prototype are to help you gain a much deeper understanding of your product, and ultimately so that you can actually test your ideas with real users before you have your engineering team take months to go build something that you have no real evidence will serve its purpose.
In this article I will like to talk about how to actually do this prototype testing. I’ll warn you up front that this article is relatively long, but I will also say that testing your ideas with real users is probably the SINGLE MOST IMPORTANT ACTIVITY in your job as product manager.
If your company is large enough to have its own usability testing team, by all means secure as much of their time for your project as you absolutely can. Even if you cannot get much of their time, these people are terrific resources and if you can make a friend in user research or usability engineering, it is l be a huge help to you.
If your organization has funds earmarked for outside services, you may be able to use one of many excellent firms to conduct testing for you. But at the typical $10,000 – $20,000 per round of testing (typically around 10 users) that most firms charge, chances are that you won’t be able to afford as much of this testing as your product will need.
But if you are like most companies, you have few resources available and even less money. But you can’t let that stop you. It is absolutely essential that you test your ideas out with real users. It is arguably the single most important part of your job.
So I will show you how to do this testing yourself. Don’t get me wrong, you won’t be as proficient as a trained usability engineer, and it will take you a few sessions to get the hang of it, but in most cases you will find that you can still identify the serious issues with your product, which is what is important.
One thing to note is that while most people think of usability testing? (seeing if people can figure out how to actually use your product) I consider that just one type of testing. You also need to test the desirability or usefulness of your product (do people actually want to use it?), and we will discuss both forms of testing here.
Finding Test Subjects
Before you do the prototype testing, you will need to round up some test subjects. If you are using a lab they will recruit and schedule the users for you, which is a big help, but if you are on your own, you are got several options:
– If you are established a charter customer program?as I will described earlier, you should have quite a few readily available. If you haven‘t you should.
– If you are doing a product for business, then trade shows are a great source of target customers.
– It is increasingly common to advertise for test subjects on Craigslist. If you do this, try to keep your participant description a notch more general than specific, and then screen on the general when you call them to talk about this.
– For consumer products, you can use your friends and family? Network ?just try to avoid people too close to you, and those in the tech industry, unless that is specifically your target, and be sure to use subjects from outside this network too.
– If you have a list of e-mail addresses of your users you can do a selection from there ?often your marketing team can help you narrow down the list.
– You can solicit volunteers on your web site ?lots of major sites do this now ?remember you will still call and screen the people to make sure you don‘t get a bunch of early adopter types.
– One technique I especially like is to set up regular prototype test sessions, say every other Friday, where you arrange for 10-20 or so users to come into your offices for a couple hours each, and then your product managers sign up for time slots, so a given user might test a couple prototypes each. I like this a lot because one person can do the logistics of invites and screening, and many PM will can count on a ready set of test users on an regular basis.
– 我特别喜欢的一种技术是建立定期原型测试会话，假定每隔一个周五，你安排10 – 20个左右的用户到你的办公室几个小时，然后你的产品经理报名参加，这样就可以让给定用户去测试几个原型。我非常喜欢这一点，因为一个人可以负责邀请和筛选的后勤工作，而许多PM可以定期依赖一组准备好的测试用户。
– You can always go to where your users congregate. If you are doing an e-commerce product, you may want to go to a mall. If you are doing a sports product, go to a sports bar. If your product is addressing a real need, you won’t have trouble getting people to give you an hour. Just bring gifts, and try not to look like you are trying to convert their religion.
– If you are asking users to come to your location ?especially for business use ?you will likely need to compensate the people for their time. If you are doing a consumer service sometimes a big sincere thanks along with a hat with your company logo on it will suffice, as people genuinely want to help in the creation of products, especially for companies they like. However, if you do compensate consider providing product like $50 of credit on your site.
– Realize that there is a very high no-show rate when when you schedule people to come in. It’s just a fact. Sometimes as much as 30%. But you can drop it to 5-10% if you give people a personal phone call the day before — even if you leave a voicemail. Note that email does not work equally well.
Preparing The Test
You will need to define the usability tasks you will want to test, and the interview questions concerning desirability:
– You will need to define in advance the set of tasks that you want to test. Usually these are fairly obvious. If you are building an e-mail client, your users will need to do things like compose a message, read new mail, and file away messages. There will also be more obscure tasks, but concentrate on the primary tasks, the ones that users will do most of the time. If you have time, you can get to less common tasks but it isessential the key tasks are tested well.
– There is a one-time only opportunity you have with each user you test. That is the opportunity to learn how they think about this problem today.
– If you are testing a new online restaurant rating service, rather than start them out at your prototype is home page, maybe you want to just start them out with an empty browser and see what they do. What review sites do they use today? Do they use Google or Yahoo will search to find the specific restaurant, or do they go to somewhere like OpenTable or Zagat? Do they search by neighborhood, by cuisine type, or price range?
– 如果您正在测试一个新的在线餐厅评级服务，那么，不要直接从你的原型主页开始，你或许要从一个空浏览器开始，看看他们是怎么做的。他们现在使用什么评论网站?他们是使用谷歌还是雅虎搜索来找到特定的餐厅，或者他们会去像OpenTable（老汤注：美国领先的网上订餐平台，2014年被Priceline以26亿美元收购）或Zagat（老汤注：一家有着近 40 年历史的老牌餐饮点评公司，11年被谷歌收购，18年又被谷歌出售给了The Infatuation）这样的地方?他们会根据社区、烹饪类型或价格范围来搜索吗？
– This type of incredibly valuable information is missed if you jump right into your prototype, which will necessarily have quite a few assumptions built in. Once they play with your prototype they can tell you what they like better, but they won’t be thinking about the problem anymore the way a first-time visitor would.
– Of course, you will then want to get them to your prototype, but there is one more thing before you jump into your tasks. See if they can tell from the home page or landing page of your prototype what it is that you actually do, and especially what might be valuable or appealing to them. Again, once they jump into tasks they will ose that first time visitor context, so don’t waste the opportunity. You will find that landing pages are incredibly important to bridging the gap between expectations and what the site actually does.
– After you are seen if your user can figure out how to do the tasks you are testing, it is now the right time to have a conversation with this user. Think of it as a one-person focus group. Does the user use a different product or site today to do the same thing? Or is this something they do manually or off-line today? How much better is this than what they use today? And don’t forget to ask my favorite question: How likely would you be to recommend this product to your friends? Now that the user has interacted with your prototype they understand the topic and you can have an extremely useful dialog with them about this problem. Most importantly, you are trying to gauge how much this person values this product.
– One technique I like for gauging value is to ask how much the user would be willing to pay for it, even if you have no intention of actually charging for use this way. It is a way to assess value and especially to track how the average value goes up or down over time as you change the prototype. Also, it is useful if you structure your questions so that the answer is on a scale, like 1-10. This is so that you can track the averages as they improve.
– Note that you don’t have to wait until you have a complete prototype in order to begin testing. You can start with the main tasks, and it is ok if you have dead ends in the rest of the prototype. If the user wanders over to one of those dead ends, just ask, and what would you expect to happen if you did that??This is a great question whether you have that path laid out or not. If you do have it laid out, you can see if they match. And if you don’t, you will get important info about that direction anyway.