Why Usability Testing Does Not Work For Me
In my recent post about Responsive Design, I received an interesting comment from one reader. He said that my point is misleading, has no technological merit, I have no idea what I am writing about, and I should delete my post.
Despite this not so favorable response (LOL), I also had another comment, which gave me the idea for this post. It also reminded me why I post, as I like the ideas that come from getting and giving feedback to these posts.
Regarding Usability Testing, I am sure it works well for many, but it does not work well for me, and here is why.
There is a big difference between a real user, and a usability expert viewing my website.
A real user might spend hundreds of hours or years on my website analyzing properties, and this makes sense. With the high price of homes, this is their biggest purchase. There is no money back guarantee, so they really need to do their homework.
A usability expert is only spending 20 minutes or so on my website, and they are not spending hundreds of thousands or millions on a new home. Their goal is to record about 20 minutes of video (or spend 20 minutes with me if we do it live), giving me suggestions on how they see my website. A lot of time is spent on the homepage, and the overall feel of the website, without ever doing any intensive searches or analyzing the results.
In my most recent try, the expert did not even know what my site was about when he landed on it. That really does not reflect a real user, because real users come to your website from a Google Search, an ad, or word of mouth. They know exactly why they are going there, and what they hope to find.
I also have had feedback from users that have a lot of experience with Real Estate sites, and they essentially want my site to look like other Real Estate sites that they like. While these other sites look good, it does not mean my objective should be to copy their look. I want my site to be unique, to stand out from the competition, rather than blend in with the competition.
I can see how usability testing works well for Walmart.com and Amazon.com and other similar sites. Their buyers don’t want to spend a lot of time on their website, they want to find what they need, buy it, and get on with their day. The purchases are normally under $500, and the risk is small, as they usually offer a money back guarantee.
The best suggestions I receive are from my actual users, and I encourage them to make suggestions, and let me know if they see a bug.
That being said you still might want to do Usability Testing, but keep in mind the points above when you get your test results.
Joe Segal
Posted at 14:31h, 07 JanuaryGreat post! Oahure.com is an awesome, completely unique website which obviously works well for you, your users, and your business goals.
Testing different user experiences and then lining those up with business goals as first priority trumps opinions all day long. Those who make the most sales wins, not the guy with the most beautiful “usable” design.
Bryn Kaufman
Posted at 20:10h, 07 JanuaryGood points Joe, glad you liked the post, thanks.
Kelley Howell
Posted at 09:47h, 08 JanuaryHey there Bryan, just a point of order. What you are desscribing is an “expert review” also sometimes called a “heuristic analysis” or “heuristic review”. An expert has insights about good design based on lots of existing research, has a set of principles to look for, and he checks off your site’s features and experience (download speed, etc.) against those standards.
It is a low-level form of analysis that is good for companies that have zero klew as to what they are doing. Which is not your problem, so you shouldn’t use those services. Wast of cash.
USability testing, on the other hand, is sitting down with a person who has a recorder trained on their screen and their face. you talk with them, asking them to perform tasks on your site. The goal is to find usability _problems_ . Is it hard to find the search field? Is it hard to find a way to contact an agent? Is it easy to use the filters on search results.
You also listen to what they say and watch what they do, which can be two different things. A user will say, for instance, that she hates sliders which slide in to alert her that she can contact the seller. But then the person uses it instead of the two other ways to contact the seller.
You also talk with them about what they do and don’t like about your site but bearing in mind that users aren’t always good at knowing what their want or need.
there are lots of other ways of working with users to find insights. Typically it’s doing a contextual inquiry: sitting with them, fly on the wall style, watching them search for homes. E.g., I sat with one of our loyal users, a fan, from about 3 pm to 10 pm and watched as she did a lot of things, occasionally using a desktop and tablet to search on real estate sites. That’s called contextual inquiry. I was so inconspicuous, she forgot I was there.
The number one rule about listening to user is don’t listen to them. Which is to say that you want to find out what their problems are and what their expectations are, but you don’t allow this to drive your product design process. You use it to _inform_ it because user experience is about integrating insights about usability (usefulness, ease of use, and pleasure in using) with insights about a business. Hence, your insight that what people say they want isn’t always what they actually act on. They may think they want a site that looks like Zillow, but that doesn’t mean that when you look like Zillow users will show up at your site more often.
Bryn Kaufman
Posted at 10:21h, 08 JanuaryGreat points Kelley.
Can you recommend some usability testing software?
I work from home so having a user come into my office for testing does not work, and it seems I might get more volunteers if someone can do it easily from their home, rather than driving to meet me some where.
I would love to watch my real users to see what they do, what they like and don’t like, how they use the website.
Is there some software that sends me a video where I can see their screen and hear their comments as if I was there watching them?
Steven
Posted at 14:30h, 08 JanuaryKinda related, I love Jared Spools hotel analogy regarding user wants and expectations.
If you ask a person what they want in a great hotel they may say things like a soft bed, great tv, or even a glass of wine waiting for them when they get back in the evening or a fresh cup of coffee when they leave in the morning.
What they won’t ask for is locks on the door, hot water in the shower, or air conditioning. Yet, if you don’t supply those you will anger your customers.
Bryn Kaufman
Posted at 18:57h, 08 JanuarySteven, I know what you mean, although I am trying to translate those hotel wants and needs to the wants and needs for a Real Estate site.
Sam DeBord
Posted at 15:47h, 08 JanuaryA couple of things:
If your best suggestions are from your current users, you’re taking advice from a self-selected group–only the people who like your site. All of the people who landed on it and left because of something they didn’t like aren’t included in your analysis of whether or not this is a good idea for overall traffic or just a squeaky wheel suggestion that will actually deter more users.
It sounds like you’re saying certain kinds of usability testing doesn’t work for you. Certainly you’d need testing specific to the real estate consumer and the analysis should be focused on that kind of browsing behavior, not a shopping cart e-commerce website’s user behavior. Specialized usability testing or expert review with your user base as a focus would probably work well for you.
Bryn Kaufman
Posted at 18:54h, 08 JanuaryGood points Sam.
We all would like to know why a user comes to our site and then leaves, or stays.
I would like to do some expert review with my user base, although not sure what is the best way to get that going.
Adani realty
Posted at 23:38h, 18 JanuaryNice post! It is describing the different between testing, user & real user. But that’s the real thing is that testing, user once time know how to use in it. The best options are real estate related software user and the trafficking the customer in the google listing website. Thanks a lot of information sharing with us.