racolour (206)552-0606

Category Archives: Amusing

Flexible Enough to Change Your Mind

Remember back in high school? (For some of us it has been a while) Do you remember liking a song or a group and proclaiming they were THE best group of all time… only to find a new favorite in a month?

If we aren’t careful, we start to lose that ability. The one that lets you realize perhaps you haven’t found the perfect thing or solution. The trait that allows for experimentation and recanting.

To put it another way, it is also admitting that “maybe I was wrong”. In the business world (beyond high school) we are much more hesitant to admit that. Much more hesitant to follow a path that might prove that we were “wrong”. This is often the source of many a company’s hesitation to perform usability testing. It might prove that a genius idea is no longer so… genius.

Still though – wouldn’t you rather know? Isn’t the improvement and gain worth the experiment?

The UX world calls it usability testing or user experience research. The rest of the world calls it “keeping it real”. At least we did, back in high school.

Heroin, Sharks, and Why UX Testing is Important.

Once upon a time we thought heroin was a great cough syrup. And maybe it was. Too bad it caused other problems (like heart failure…) that we didn’t bother testing for. There was also a guy that said sharks don’t get cancer, so shark cartilage must help people with cancer… except we tested and found out sharks do get cancer.

There’s an element of gambling when you don’t test things first. You would never gamble with your health or your money, why would you gamble with your business?

Today’s new business health cure  is “Let’s redo our website! I know a great graphic artist.”  Now throw some money at it in the form of Google Adwords and you will probably get some customers. Does that mean you figured it all out? Maybe. Probably not. Actually I have no idea. Why not? Because nobody has researched the user experience with the old site yet. Unless you figure out what needed attention in the first place, you are just rolling the dice and hoping the usability is better this time.

Once upon a time we wanted things to better, but we didn’t know how. So we tested things to figure that out. Then we made things better, and tested them again to make sure of it.

Now things are better, and we aren’t just guessing why.

We eliminated the gamble, and everyone lived happily ever after.

The End.


New Users vs. Experienced Users: Good vs. Great Researchers

Let’s just get it out there right now: All qualified participants are NOT created equally (in usability testing I mean). Well maybe they were created equally, but lengthy experience with a product has a way of changing that. The more experience a group of users has with a product, the more experience a researcher is going to need to run the project.

Two spaces we tend to play in are consumer website usability and enterprise software user experience. The B2C website industry tends to focus mostly on newer or casual users, while enterprise software has far more experienced users that interact with the product frequently. New users need clear updates confirming that they are in the right place and doing the right thing (example: buying a gift for someone online). Experienced users already know that stuff because they’ve used the product repeatedly. They are more interested in efficiency and flexibility (example: running a financial report showing asset depreciation over 5 years).

Now which one of these is an easier experiment to run? Watching a newer user navigate a website that you quickly become more familiar with they they are after the third session, or watching a seasoned user run a complex software program that you may not even understand on a theoretical level? In my opinion this is what separates an “ok” researcher from a research “wizard”. The wizards can analyze abstract patterns that a group of seasoned participants will reveal. Not only that, but they can leverage the experience level of the participants to deepen the study.

A good moderator knows how to script an experiment to get unbiased results with minimal observer effect. A great moderator knows how to spot patterns with experienced participants that such a group may not even be able to articulate. And a “research wizard”? They have the delicate skills to handle both new and experienced users in their own unique way to reach the same ends – to enhance your user experience.

Usability: A Well Told Story

Have you ever had the pleasure to listen to a really good story-teller? The kind that you’ll hang around a little while longer even though you have other things to do… because the experience is so enjoyable. I was remembering a story-teller I met in Oregon a few years ago and it got me thinking about the parallels of story-telling and a good user experience. Really they are the same thing.

In both cases, you go into the experience hoping it will be a good one, and doubting the credibility of the endorser will weaken the experience – ruin it completely if it isn’t there. Be honest about what I’m here for, and then deliver on that.


Please stay on point – its hard to follow you if you are jumping all over the place.


You really want the entire experience to flow or weave together so that it makes sense and is consistent the entire way through.

Get to the Good Stuff – fast!

If a story-teller takes too long to present you with something interesting there’s a good chance you’ll move on before they finish. Sorry, my time is precious and this is painful.


There are probably a dozen more traits we could think of, but you can see the parallels. Story-telling is one of the oldest types of user experience, so there are a lot of really great examples out there. The next time you are creating your own user experience, think of the story you are weaving for someone, and make it an enjoyable one.


Moderated vs. Unmoderated Usability Testing: A Dating Analogy

Have you heard people taking sides about what is better – moderated or unmoderated usability testing? It is time for us to add our two cents to this debate. In our opinion, both methods have equal value – the question is which one do you do first, and why?

Let’s play a little game. Pretend a friend of yours is having trouble getting past a first date. You want to help. How would you help your friend? with some moderated observation or some unmoderated observation? Let’s play out both scenarios:

Moderated Approach: Your friends date has allowed you to tag along. Even better, you get to ask them questions about the interaction during the date (can you imagine?!?). If you friend does a poor job on the “kiss” at the end of the date, you can see their reaction and ask them what happened. If they get up and leave in the middle of dinner because the date is too difficult to stand any longer, you get to ask why. You have the option of digging deeper at any time! An upclose and personal approach to collecting data. Jackpot!

Unmoderated Approach: Now pretend your friends date is not so into the third-wheel scenario. Or perhaps you want to stay out of the spotlight to keep from interfering with the results. You devise a system where you get your friend to carry a little device to record checkpoints and how long they took. The date shows up at the restaurant? Check. They make it to dessert? Check. They leave together? Check. You even get your friend to take a little camcorder with them so you can get more detail. 100 dates, same results – everything is fine until the kiss. Now you know when, but why?

What happened? Well… when you ask 3 dates in a row in the moderated study, they confess that your friend has bad breath. Not terrible, but a bit of a deterrent. On top of that, each person noticed that your friend had a bit of dandruff going on, and talked about their dog a little too much. None of these things alone would be enough to say “no” to a next date… but combined? Wait a minute! You weren’t even making a point to test for dandruff or bad breath, and you definitely weren’t keeping track of how many times you friend mentioned their dog because you didn’t realize that was even happening!

…get it? Unmoderated testing is really good at capturing quantitative data on preplanned events… as long as you know what you need to watch for. It is extremely efficient in that way. The difference is that with moderated testing, you catch lots of things that you weren’t even looking for, and also get more thorough qualitative information on the things you are intentionally observing. You wouldn’t need to go on 100 dates… I’m sure you would catch notice the important details in about… 3 dates. You could tweak the system at that point if the reason was obvious and resume with a modified scenario (breath mints and special shampoo).

Is moderated testing always better? Well, we wouldn’t go that far.  Moderated testing is generally good to start with so that you can catch things that may be issues before you even realize you need to look for them. It also helps you dig deeper into “why” something is happening. Moderated testing is also more expensive, so if you are on a limited budget in fairly well-known territory, it is a good compromise to get some quantitative data that you can work with further.

Usability testing (interactions with “things”) is very similar to relationships (interactions with “people”). We can collect a lot of data to serve us, but a half-decent researcher will always be ready for what they weren’t looking for – immersing themselves in the process rather than simply waiting for automated data to be handed to them.