Monday, August 28, 2006

Qualitative Data vs Behavioral Data: Who do You Pay Attention To?

Yesterday during a call with a potential client, this topic came up. This is a well-known desktop software company, and they have a unique challenge: Their primary measure of customer loyalty is the "Reichheld Score" aka the Net Promoter Score, which is based on customer responses to a single question -- "Would you recommend us to a friend?". Now, the interesting thing is that while this company is doing rather well in the market, their overall net promoter score is not that great. In fact, their score is lagging behind other comparable software manufacturers.

So, the obvious question is -- why doesn't their net promoter score correlate with company growth? Which metric should they rather measure as a driver of company growth?
As we talked, I also found out that this company hasn't done much in terms of evaluating the behavioral data on their customers -- you know, stuff like actual purchase transactions, new purchases versus repeat purchases, customer complaints, returns, etc. And I couldn't help but think that perhaps this is where they should start look first. A great deal of research work has shown us that past behavior is the best predicator of future behavior, and this is true when it comes to measuring customer loyalty as well. A 2002 HBR article "mismanagement of customer loyalty" describes this as:

Simply put: Not all loyal customers are profitable, and not all profitable customers are loyal. Traditional tools for segmenting customers do a poor job of identifying that latter group, causing companies to chase expensively after initially profitable customers who hold little promise of future profits. The authors suggest an alternative approach, based on well-established "event-history modeling" techniques, that more accurately predicts future buying probabilities. Armed with such a tool, marketers can correctly identify which customers belong in which category and market accordingly.

So why isn't this company looking at behavioral data on its coustomers? I didn't get a clear answer, but could it be that it is easier to conduct surveys rather than dig deep into data, specially when the data volumes are huge and the data is scattered around different corporate silos? Could this company be viewing the analysis of behavioral data as a long, complex exercise that involves getting down and dirty with data warehouses and analytical modeles, when all they needed was a nice simple metric that would measure customer loyalty and nicely correlate with company growth?

Yes, I am being a bit facetious -- but while I don't dispute the value of qualitative research, I think in this case, they are more applicable AFTER an initial study of behavioral data. This company needs to understand the "WHAT" first (i.e. what is going on with my customers? which customer attributes/behavior are best indicators of company growth?), and then they can apply qualitative reserach to understnd the "WHY", i.e. why things are happening the way they are.

1 comment:

Sandeep Giri said...

Adelino de Almeida picked up on this post with an interesting commentary.

NPS is basically yet another measure of customer satisfaction. The problem with the prospect with whom I had the NPS conversation was that NPS was the *only* measure they were looking at. Somehow there seems to be a line of thinking that if you could only have one measure of your marketing effectiveness, NPS would be it -- like it's the ultimate question that measures customer loyalty. And of course, since we all want something quick and easy, NPS becomes the express way to get your success metrics fast, never mind the accuracy.

On the other hand, we do have some clients who use NPS as one of the many measures to measure marketing success. Of course, I have yet to see NPS itself drive any particular marketing action yet -- but it can be somewhat helpful to measure how customer satisfaction is impacted by new marketing initiatives (or not).

Bottom line -- as you have pointed out, marketing can be only effective if some of the basics are taken care of first -- relevant/useful product and services, and a sound knowledge of your market performance based on behavioral metrics. And NPS only measures a piece of this aspect, hence you can't accept it for the complete truth.