Filum series — Episode #3: Summary Notes For MIT’s Webinar “Understanding Decision-Driven Analytics”
Following up with a counter-intuitive sounding article in the spring issue of MIT Sloan Management Review which is Leading With Decision-Driven Data Analytics, the respected guru couple on marketing and behavioral science, Bart de Langhe and Stefano Puntoni recently joined a webinar to elaborate more about the approach with interesting and easy-to-understand examples.
For Filum AI, analytics towards making data-driven decision is one of our core approaches that reflects in the Product Roadmap, the Sales deck, the Proposals, the Onboarding process and we consistently and repeatedly communicate this with our customers. So upon watching the webinar, we couldn’t resist from taking notes and sharing with Data and Analytics community.
Two trends
They started the talk with compelling trends that have emerged in the business in previous 2 decades
And the message is simple. We should not expect computers to solve problems of psychology (consumer behavior, buying journey falling into this domain). Instead, we would need psychology to solve problems with computers. In other way, we put customer at the center and other things (psychology, analytics, tools and platform) around towards making customers happy, and of course, by the right decisions.
Two problems
- Take data (scientist) at face value
- Answer the wrong question
Then they respectively threw out convincing examples for each of problems: from Shark Attacks and Ice-cream Sales, to public numbers by big business guys (Facebook against privacy change from Apple, Twitter with measuring sales impact using their own metrics)
The above example leads to a very critical question when we keep look data at the face: what’s behind the data?
Another example for causal correlation mistake is when Facebook provided a number (60%) that they explain is a cut from brand’s sales for ads spend. They question the conclusion from Facebook because they do not know how Facebook came up with that number. And this is similar to the sweet meat hypothesis from shark attack and ice cream above.
They also showed more example on marketing reports which everyday we have been searching for it and looking at it as if it is always true:
See the blue bar at the bottom which are wrong. People who see the Search Ads are actively searching for the product and they purchased it anyway, not just because the ads made them walk to the store and buy. So watch out when you see any data-based conclusions!
The example we like the most is about the churn prediction. In this example, we have 3 available data points about our customer, such as Tenure (how long s/he with the brand), Logins last month (the number of times he/she logged onto the brand website last month) and Extra purchase (number of additional fee s/he made past year). Based on the historical data, they build the churn model that predict the likelihood of churning for these 3 customers (Xiao, Yann and Zoe).
Now, if you were the business owner, which one would you take the action on, in this case send a gift to him/her?
In order to get the real engagement from the webinar audiences, they opened up a poll which show the following result:
The common best practice is to send the gift to the ones who are most likely churning, in this case, Xiao. Wikipedia and Accenture agree on this way. However, another experiment conducted by Professor Eva Ascarza at Harvard Business School which show that the more treatment we do, the higher likelihood of those customers to churn.
This example implies that we are answering the wrong question because we based only on the data we have while in more often cases, we need new data sources.
In order to answer the right question, they suggest to run Randomized Controlled Trial (RCT) experiment on a subset of your customer. Send gift to customer group 1 and do not send gift to customer group 2. And then measure whether customers renew versus churn. And if a systematic difference between 2 groups is observed, you can conclude that this is because of the gift that you sent. Bottom line, you very often need somehow to enrich your available data to answer the right question which is at the end really impactful to your customer’s decisions.
Four-step Framework for Decision-Driven Analytics
Finally, they proposed the framework to get the approach into action.
The framework is very useful and matching with our real experiences when working with customers. The highlighted note at the Step 2 is very important which is “You will often need new data” to find the best action. In reality, customers are jumping right to Step 2 which leads them stuck. This also implies a very fundamental principle:
Don’t Start with (Available) Data.
Start with Actions. Then Data. Then rank up the actions and Finally find the best action to make.
The bottom line reveals a lot of mistakes from Big Data projects which is led by the wrong person.
The rest of the webinar is also interesting with Q&A between moderator and two gurus.
You can watch again the webinar (below link) to understand more about the approach:
Thanks MIT and Bart de Langhe and Stefano Puntoni for eye-opening presentation and discussion.
