This morning, my team discussed the benefits and drawbacks of large-scale A/B testing.  Websites like Google and Amazon often use A/B testing where they randomly show some of their users a new version of a webpage, and measure whether the outcomes are different: a better click-through rate, for example.  There’s a lot be learned in this kind of testing.  It’s a powerful method for websites to learn about how design changes, both major and minor, can impact how users complete their task.

However, it doesn’t give you a complete picture.  It tells you what happened, but it doesn’t tell you why it happened.  One of the differences between good user research and great user research is in what you learn and how you can apply that information in the future.

In good user research, you learn that something happened.  Maybe you’ve learned that a user is completely blocked from finishing a task.  Maybe you’ve learned that users can complete their task 20% faster.  Maybe you’ve learned that, while they’re not doing anything faster, their satisfaction ratings are higher than usual.  Each and every one of these findings is important.

Each and every one of those findings can be made better if you know the reason behind it.  Sometimes it’s reasonably obvious, but oftentimes it’s not¹.  When you know the reason why a change has improved or degraded the user’s experience, you have a better opportunity to innovate in the future.  You only have data.  You don’t have insight.

Good user research allows you to react.  It allows you to evolve your designs.  With good user research, you will make improvements.  Your application will be better.

Great user research allows you to learn more about your users.  It gives you insight into how they think and what they’re trying to accomplish.  It allows you to make intuitive leaps and to truly innovate.  With great user research, your greater understanding of your users will allow you to make improvements to your whole business, not just your application.  Your business will be better.