Differentiating intent and success in web analytics
03 Jul 2016 » Analytics Tips
Most of you will agree with me that the main task of a web analytics is to analyse the success metrics under different conditions. In general, this means analysing these metrics against different dimensions, segments, dates… This is precisely where Adobe Analytics excels. I will get into more details in a later post, but, as an analyst, you need to know how the data has been collected. One of the typical examples I give to my customers is when I explain them the different between intent and success.
Imagine the following scenario:
- A user browses your website and finds something he is interested in.
- He clicks on the add to basket link.
- This click makes an AJAX call.
- The AJAX call fails (for example, times out).
The question is ask is: where should the success metric be captured, when the user clicks on the button or when there is a confirmation that the action has been successful? In other words, what should be captured, the intent or the success?
Simplicity vs accuracy
For those of you who are developers, you will immediately think that the easiest thing to do is to track the click. In DTM, you can just create an event-based rule to capture the interaction, grab the product ID from the data layer, add the scAdd event and… ready! But you also know that this metric will have some error margin, as the AJAX call may fail. On the other hand, adding code to the AJAX callback is more cumbersome as it involves modifying existing code. But you also know that this will be more accurate, as only successful AJAX calls will trigger the scAdd event. I know that there is still the possibility that the image request fails, but this is something that can happen in either case.
When I present this situation to my customers they first scratch their heads, as they had never thought of this situation. As you can see, there is a trade-off between these two concepts, simplicity vs accuracy, which equates to our initial comparison, intent vs success. Usually, web analytics will want accuracy, whereas developers will prefer simplicity. Someone needs to decide which one needs to be tracked.
Track both
Or, instead of choosing, why not track both? I have found this option to be very popular. With this solution, in Adobe Analytics, you can see in the same report both metrics. Another advantage is that it allows you to check if there are any problems with the website. In my previous example, if the difference between clicks and successful AJAX calls is too high, it might mean there is an issue somewhere.
There is one downside, though: cost. Tracking both intent and success will incur in extra server calls and additional development effort. However, the customers I have worked with have been happy with the marginal extra cost, compared to the benefits they get from this additional information.
That being said, there are a few cases where you can only track intent. The typical example is sharing in a social network or via email. You will never know whether the visitor finally succeeded; there is no way of knowing it.
Other metrics to track both intent and success
The example I have provided is the add to basket, a typical example, but probably not the only one. Other situations where you will probably want to track both the intention and the success of the visitor:
- Purchase button vs order confirmation page. Probably the most important case and the most probable situation where the intent will not be successful: not enough stock, credit card declined… You can even add additional metrics, like total basket value on click, which will show you how much revenue you are not generating. In this case, you should only track this currency event only once.
- Form submission. Your web server is probably performing some server-side checks before accepting a form submission, be it a newsletter submission, a log-in or a credit card application.
- Complex search. If your website has a complex search engine, like those for hotels or flights, you will want to know how many times the user did not get any result.
I am sure you can think of many other examples where you can apply this approach.