The Importance of A/B Testing
The bottom line is that you don’t know the best way to accomplish a goal until you try several different approaches of reaching the same goal. In the web world, that process is generally referred to as “A/B Testing”. A few months ago, I created a “Recommended Technology Providers” page, and had ESM Exec Designs‘ graphic designer create several 300×250 ads to drive existing Geek Estate traffic to there. My goal was simple. Entice current agents and brokers already reading Geek Estate to look at the recommended technology providers page. I didn’t just create one ad. I created 3 (which I’ve inserted below) so as to measure which ads perform well and which ones don’t.
Wording: Real Estate Technology Products, Color: Green
Wording: Real estate software providers, Color: Green
Wording: Real estate software providers, Color: Blue
Which one do you think had the highest click through rate? If you guessed the blue one (my original guess), you are wrong.
Here are the raw numbers. The percentages in the far right column of the image below are the click through rate (columns left to right: ad name, weight, total impressions, impressions today, clicks, clicks today, overall click through rate).
The only difference between the first two ads is the wording “technology products” was swapped for “software providers”. Yet the technology products ad results in a 68% increase in click through rate. And it turns out that a blue ad doesn’t draw any more attention than a green ad, at least on this blog.
My takeaway from my A/B testing? Agents and brokers aren’t looking for software products, but rather for technology products — even though they are technically essentially the same thing.
I just de-activated the two ads that weren’t working and increased the weight of the technology products ad that was.
Are you A/B testing?
Ann Arbor Real Estate
Posted at 08:05h, 25 JuneWhat program are you using to A/B test your ads?
Drew Meyers
Posted at 12:15h, 25 JuneAd rotate
Ann Arbor Real Estate
Posted at 14:41h, 25 JuneThanks! Does it give you stats on the ads or do you have to integrate it with an analytics product to get stats?
Drew Meyers
Posted at 08:37h, 01 Julyit gives you high level stats. but not much more than what you see in the screen shot
Gahlord Dewald
Posted at 20:18h, 25 JuneI take away a slightly different thing: The audience is looking for a product not a provider. In other words, they want a transactional relationship not an engaged (and perhaps assumed more pricey) relationship.
Your text highlight is really testing four possibilities (technology vs software and product vs provider) but “bundling” the results.
For fun, it might be interesting to try the remaining variations: technology providers vs software products and see if more insights or lift in CTR can be teased out.
Great test Drew! Thanks for sharing!
Sam DeBord, SeattleHome.com
Posted at 07:32h, 26 JuneThis is a great reminder for us, Drew. So many on the RE side get too
caught up with the daily grind and just “try out new things”, without
ever analytically testing the results.
Agreed with Gahlord on the terminology. “Technology Products” immediately strikes me as a CNET review, something I’d like read. “Software providers” feels like a business relationship offer.
Drew Meyers
Posted at 08:38h, 01 JulyAgreed Gahlord. Further testing would be fun. Unfortunately, it’s not at the top of my list right now though. Got my designer working on other stuff 🙂