The W3C data layer – part I

This is the first post of a series of posts, in which I am going to describe the W3C data layer. A few months ago, I explained why it was a good idea to have a data layer. In this series, I am going to dive into the details of one particular data layer implementation: the W3C standard. For those of you who do not know what the W3C does, it is the international body that creates the standards that we use everyday on the web: HTML, CSS, Ajax… Although there are other options for data layers, like JSON-LD, I personally prefer the W3C standard; after all, this body has created some of the most important standards in the Internet.

The first thing I suggest is that you download the W3C data layer standard: http://www.w3.org/2013/12/ceddl-201312.pdf. It is completely free. Have a look at it. You will notice the amount of well known companies that contributed to this standard, including Adobe, my employer. In total, 56+ organisations and 102 individuals have collaborated in the creation of it. So, if you choose to follow this document, you can be confident that you are not on your own.

You might have also noticed the recency of this document: it is less than two years old (at the time of writing). This is probably why many Web analysts have never heard of the concept of data layer. That being said, the word is spreading quickly and it is starting to become the norm, rather than the exception. In fact, a few of my customers, that are undergoing a major redevelopment of their websites, are including a data layer, which they did not have before.

I hope that, by now, you are fully convinced of the need of a data layer and the benefits of going with the W3C standard. Your should also start spreading the word within your organisation. I have found that this step can be important, as any new addition to the website will face some resistance. It must also be remembered that this data layer is not exclusive for Web analytics; other Web marketing tools, like Web optimisers and DMPs will greatly benefit from a data layer.

Probably, the development team is going to be the most difficult to convince. They might have a different approach or think of the effort it will take, but my experience shows that, once they understand it, they will support this concept.

Start defining your data layer

Once you have everybody aligned, you should create a document with the contents of your particular implementation of data layer. Remember to include in the documenting process all on-line marketing teams: Web analytics, optimisers, advertisers… I was recently involved in the creation of a data layer for a customer and it took 5 weeks until it was finished. This is probably an edge case, but you should be aware that this stage might take longer than initially expected.

In a future post I will explain what is the content of the data layer. For now, I suggest you review section 6 of the W3C data layer document, to see what you can expect to include in the data layer. There are a couple of examples in section 7.

Location of the data layer

Before starting the development, the location of the data layer must be agreed with all parties involved. Ideally, it should be at the beginning of the <head>  section of the HTML document. The reason is that it can then be used by any other JavaScript code. If this top-most location cannot be achieved, it should be located before loading any tool that needs will read the data layer. For example, if you are using a tag manager or a DMP like Adobe Audience Manager, the data layer should be placed above all of these tools.

There is finally one additional technical problem with placing the data layer at the top. Page-level information is usually retrieved from the CMS and can easily be cached and set in the HTML. However, depending on the CMS, there is some information, like user-level information, which is not available on page load and it requires an AJAX call. As a consequence, it is possible that the code that needs this data executes before the data is available. For example, the Web analytics code might be capturing the log-in status and will need the user-level information when executing. This problem needs to be solved on a case-by-case basis.

 

In future posts I will describe in greater detail other aspects of the W3C data layer:

  • Each of the JavaScript object
  • Integration with DTM

One or multiple report suites in Adobe Analytics

Back in the old days, before SiteCatalyst 15 was released, the limitation in segmentation meant that, usually, you needed multiple report suites. You would usually have a combination of JavaScript and VISTA rules to do that segmentation (in case you are wondering, the S in VISTA stands for Segmentation), sending the data to different report suites. After that, you would also need a rollup to try to get an overall picture.

With the introduction of SiteCatalyst 15, segmentation became much more powerful. You could have one single report suite and use segments in SiteCatalyst to analyse the data. The segmentation interface received a massive improvement with the May 2014 release of Adobe Analytics.

However, there are still many valid reasons why you would want separate report suites. My friend Jan Exner gave his point of view some time ago: one or two report suites. I would go one step further and talk about more than just two and other reasons why you would want many.

  • Mobile apps. You will probably want to put all mobile apps in a separate report suite, with the Mobile UI enabled. Having a single report suite will probably create some headaches, as some features are mobile app specific and others, web specific. If you need totals, you can use Report Builder and create the totals in Microsoft Excel.
  • Multiple currencies. If you are selling in different currencies and you need an accurate reporting for each currency, then it might be better to have one report suite per country or region. However, you can stick to one single report suite, you can track both in the standard location of the s.products string and in a numeric event, and copy the currency code to an eVar. With this approach, you can report on the report suite default currency and in the local currency in which the transaction occurred.
  • Multiple time zones. As with currencies, if you sell in very different time zones and need accurate intra-day reporting, you might have to create multiple report suites, depending on the time zones. However, generally speaking, reports tend to span more than just one day and the differences in time zones is less noticeable.
  • Different teams. Some large organisations prefer to have the data separated in different report suites, so that it is possible to give permissions to access the data in a more granular way. It is then possible to give the least amount of privileges to the web analysts, so that they only have access to the data they need. For example, I was working with a customer that had completely different teams analysing Android and iOS data and these teams did not even talk to each other.
  • Legal requirements. This might not be very common, but if certain information should only be accessed by a limited number of people for legal reasons, then you need to have many report suites and grant access to the report suites depending on the needs, just like in the previous case. As an example, I was working with a supermarket and they were selling both their own white brand together with other brands; the analytics of their own brand, for obvious reasons, are not allowed to see the information of the other brands; this solution required a VISTA rule.
  • Multi-suite tagging. If your budget allows for it, the best solution is to go for both worlds: one global report suite and multiple local report suites. For lack of a better word, I use local not a geographical meaning.
  • Different SDRs. Well, this is a sin you should avoid at all costs, but if you have inherited implementations that use different SDRs, then you need different report suites unless you are willing to redesign all Adobe Analytics implementations.
  • IP address segmentation. If you need to segment by IP address, with a granularity finer than what the geolocation reports can provide, then you need a VISTA rule and multiple report suites. For example, if you have a call centre that actually uses the website, you do not want to “pollute” the main report suite with call centre data; instead, you want the call centre to be reported in a specific report suite.
  • Human vs non-human interactions. In a previous job, we had a Web services API that offered very similar information to the website. In fact, the information from the API was presented on third party websites, but we were not allowed to add any tagging to these websites. The solution was to track server-side the API usage, obviously, using a separate report suite.

I would like to hear your ideas on this topic or situations that you have found, which have led you to one or multiple report suites.

Out of stock – Advanced reports

In my last post, I described a simple solution to track out-of-stock products using Adobe Analytics. As its name implies, this is a rather simple approach: you just get a count of the number of times an out-of-stock product is shown. For many, that might be enough, but there are many different requirements for a one-size-fits-all solution.

Another of my customers wanted a more detailed view of the stock level for all products, not just the fact that a product is out of stock. For this solution, we are going to need three events:

  • event1: stock level
  • event2: stock check
  • event3: out of stock

The implementation, in theory, should be very simple. For example, let’s consider a page with three products:

  • SKU1: more than 10 products in stock
  • SKU2: 7 products in stock
  • SKU3: out of stock

The code would look like:

s.products = ";SKU1;;;event1=10|event2=1,;SKU2;;;event1=7|event2=1,;SKU3;;;event3=1";
s.events = "event1,event2,event3";

In this example, any number above 10 products in stock is not relevant.

Now, when it comes to reporting, you need to create a calculated metric: event1/event2. This calculated metric will show the average of items in stock for each product. Using event3 in the reports, you will get the number of times each product was shown and it was out of stock.

Out of stock – basic reports

The wealthiest man in Spain (my home country) is the owner of Zara. There are Zara shops everywhere in the world. Just as an example, I was in Bangkok two months ago and I found a Zara store in one of the most popular shopping centres. The success of this company has been widely studied. One of the key success factors of this company is stock management. If you are interested in a detailed explanation, here you have a video that I found very interesting:

In real stores, the only way to determine if a product is popular or not is by the number of units sold. I am not saying that this is not useful, but the mathematical models used could benefit from additional metrics. In the online world, we can go one step further and include other metrics in the algorithm, like product views, add to carts and number of times it is out of stock.

With Adobe Analytics, product views, add to cart, remove from cart and orders are standard metrics that will be included in any typical retail implementation. On the other hand, there is no standard out-of-stock report. I am sure different people will have slightly different views on what “out of stock” is. For me, it is the number of times per visit a product has been shown to a visitor and it was out of stock.

Let me summarise why I chose this way of measuring. While a product is in stock, you can measure the popularity of a product using metrics like add to basket or units sold. However, the moment it is out of stock, you do not have any way to measure how popular it is: you just know it cannot be sold. It could well be that the product is not popular any more and you can just remove it from the inventory. Or, it might be the most popular product, with thousands of page views and frustrated visitors that cannot purchase it. With my solution, you can tell how popular an out-of-stock product is.

After this long introduction, let’s go to the implementation with Adobe Analytics. This is probably the simplest part of it. My suggestion is to use a cookie and a list prop:

  • In the list prop, you set a comma separated list of product IDs that are shown and are out of stock. You need a list prop as it is possible that on one page there are many out of stock products.
  • In the cookie, you should store the list of product IDs that have already been reported during that visit.

I would like to show you some code, but since it entirely depends on each implementation, I will just show you the results. Surprisingly, the best example is a bra web page, as it has many different sizes:

out-of-stock-1

In this example, there are four sizes out of stock, so the list prop will get four values (I used the pipe as the separator):

out-of-stock-2

Since we do not want to continuously be reporting these values for this session, I keep the value in a cookie. Before setting the prop, a piece of JavaScript checks whether the value has already been reported.

out-of-stock-3

Finally, the report looks like:

out-of-stock-4

In this case, I am only interested in instances, but visits and visitors are other valid metrics that can be useful. An alternative would be to remove the cookie and always report the products. In the end, it will depend on how you want to use those values.

Why a data layer is a good thing

Back in the old days, when we used the traditional division between an s_code and on-page code, the concept of a data layer made little sense. The developers had to add some code server-side to generate the on-page code. Gathering the information to be captured was a server-side issue: the CMS would have to collect the information from one or various sources (CMS DB, CRM…) and present it on-page, so that, when calling s.t(), the s object would have all needed information.

However, now that tag managers are becoming the norm, the previous approach does not work well. There is no on-page code; all code is generated in the tag manager and injected to the page through it. This means that, in order to track some data, it must already be in the HTML code or in other resources available to JavaScript, like query string parameters or cookies.

One might think that, as long as the required information is visible on the page can be extracted using CSS selectors, we are safe. Consider the requirement to capture the city in the delivery address and this code:

Using the selector #delivery-address .city, we should be able to extract “London”. But, what if the developers decide to change the id or the class? What if there is a new requirement to completely remove this data from the web page? Our tracking will be broken and, if this happens a few months after the release, we will probably not know why.

The most reliable solution is to add a JavaScript object, completely independent of the rest of the HTML code, with all the relevant information of the web page. Then, the tag manager just needs to reference directly the elements in the JavaScript object. The developers can then change anything in the HTML code and, as long as this JavaScript object is kept intact, the tracking will continue working.

DataLayer_Blog_Example-2

There are many ways to create a data layer, but all fall into two categories: create a custom data layer or follow a standard. I will never recommend to create a custom data layer. There are a few standards that are worth mentioning:

  • AEM client context. This is the de facto standard that comes with AEM. I have spoken with AEM developers and all say that this can be used in many cases.
  • JSON-LD. I have never worked with this one, but one of my clients was already using it. More information here: json-ld.org.
  • W3C Customer Experience Digital Data Layer. I always recommend this standard, as it has been produced by the W3C (the same body standardising the Web) and Adobe took a role in this standardisation, together with other companies like Google, IBM, Red Hat… The previous image is an example of how the data layer would look like in this case. The standard itself is freely available: http://www.w3.org/2013/12/ceddl-201312.pdf.

In future posts, I will add some details about the last standard.

A brief introduction to AAM

The concept of a DMP (Data Management Platform) is not new in the digital marketing arena. However, there are still many marketers who do not know of this type of platform and what it can do for them. I will explain what is a DMP and what is Adobe Audience Manager.

In today’s digital life, visitors give a lot of information about their traits. However, that information tends to be siloed. The two main sources of information, CRMs and web analytics, do not “talk” to each other. Your web analytics tool just gives information about on-line visitors and your CRMs only knows about static data. Ideally, we would like to have all that information combined in real time so that we could provide the visitor with the best experience.

Think about these scenarios:

  • An airline wants to offer credit cards to frequent fliers between London and Paris.
  • A retailer would like to retarget visitors to their website, who have just moved to a different website without purchasing.
  • A media company is interested in selling ads on their own websites and wants to show mainly relevant ads.
  • A car manufacturer has created a series of ads, which should be shown in sequence.

All these scenarios and many more can be fulfilled with a DMP. Without getting into much detail, a DMP uniquely identifies a visitor and tries to get as much information from that visitor, using different sources and marrying the data from each of the sources. One other key property of a DMP is that it should be completely anonymous; in other words, no Personal Identifiable Information (PII) should be managed by the DMP.

Demdex was one of the companies that offered a DMP. Adobe acquired Demdex a few years ago and included the DMP in the Adobe Marketing Cloud. This product is now known as Adobe Audience Manager (AAM). What are some of the differentiating features of AAM?

  • It integrates nicely in the Adobe Marketing Cloud.
  • It can be natively be delivered through DTM.
  • It supports Adobe’s Visitor API. As a consequence, you can use the Master Marketing Profile (MMP) in AAM.
  • It captures natively all Adobe Analytics variables (eVars, props, events…)
  • It can be used in conjunction with Adobe Target to create personalised experiences.

I have been working with AAM now for a while and I can see a lot of potential in it.

DTM permissions and workflow

Some time ago, I received an urgent call from a customer that claimed that DTM broke their website. They wanted to see me immediately, as that was causing a huge impact in them. Fortunately, the problems only manifested in staging, but they could not move to production. Once I arrived at my client’s office, I immediately realised what had happened: the data analyst had created some data elements using JavaScript he found on the Internet and he just copied and pasted the code. The code worked on his computer, so he went to approve and publish it. The reality was that his code only worked in IE and crashed in other browsers.

As I have already explained, there are many benefits from using a tag management solution. However, with great power, comes great responsibility. Using DTM does not mean that marketers can implement anything they want and publish rules as they like. To the contrary, they must be very strict. The previous example is exactly the opposite of what must be done.

The solution I suggested to my client is to follow a very strict workflow and use wisely the permissions:

  • Anybody who needs to create new rules should be given the User role. This allows the user to create new rules and make sure they work fine using the right DTM plugin.
  • You need to have a group of testers that have the Approver role. They are responsible of making sure the new rules not only work fine in one browser, but they are cross-browser compatible, do not break the website, capture the expected data…
  • Once the rules have been approved, it is time to publish them. Very few people should have the Publisher role and publishing rules should be done very carefully, just like a typical code release.

Workflow

Some additional recommendations:

  • Publish rules at a certain time every week, so that, in case something goes wrong, it is easy to identify that the new rules are the culprit and roll back to the previous approved version.
  • Get some developer’s time in order to create JavaScript rules. He will probably need 5 minutes per rule and will do it right the first time.
  • The testers, ideally, should be the same as the website testers, so that the tests are full tests, with the right tools and environment.

Custom conditions in DTM

When creating both page load rules and event based rules in DTM, you have the option of configuring some conditions to determine when to fire those rules: parts of the URL, cookies, browser properties, JavaScript variables… If you add more than one condition, the AND boolean operator is applied to all conditions. However, in some cases, these out-of-the-box conditions are not enough. Thankfully, DTM offers the custom condition, where you can write pure JavaScript.

The first thing to note is that the snippet of code must return a boolean. It might work with expressions that can be evaluated as true/false (like empty/non-empty string, 0 or different than 0), but I have just not tried them. In other words, there must be a return statement within the code. In fact, the return statement must be at the very end of the code.

This code will NOT work:

condition_not

What I always do in this situation, is create a variable named ret and return it at the very end.

condition_yes

Execute JavaScript in event based rules

If you create an event based rule that is fired upon click of a link and you want to execute some JavaScript, chances are that, if you add it to the “JavaScript/Third Party Tags”, it will never be executed. The reason is that the browser has already been instructed to unload the page and move to a new URL. It will not have time to load an external JavaScript and executed; if fact, I think that it will not even attempt to do it.

What can you do in this case? Use the custom condition of the rule, to make sure the JavaScript is executed. For example, to write a cookie:

custom_js

There is no need to return true in this case. However, it is more convenient to leave it as shown, as the console will then show the rule as executed.

Capture information from the clicked element

Imagine you have an event based rule in which you want to capture the full URL of the clicked linked and store it in prop1. Your first idea would be to create a data element that returns this.href:

de_this

However, one limitation of data elements is that they have no idea of what DOM element has been clicked, if any. If you try to use the this keyword in data elements, you will get a surprise: it will not work as expected.

One possible solution is to go to the custom code of the Analytics section as set the value there:

custom_code_this

Remember to include s.linkTrackVars if you are making an s.tl() call.

I do not like the previous approach, as it is not elegant and you need to explicitly open the editor to see the contents of the rule. My preferred solution is to use the custom condition and set a data element on the fly; then, in the analytics section, just reference the data element as usual:

custom_setvar

prop_de

 

Do you have any other uses for the custom conditions in DTM? I would love to hear your opinions on that.

How to organize a web analytics team

A few months ago, I was working with a customer on premise and the manager asked me a tricky question: how to organise the analytics team. That company was undergoing significant changes in the analytics front, as a few key members of this team were leaving. As with most questions in life, there is not a clear and definitive answer to this particular one. However, I can share what I have been seeing in my customers.

When companies are small, they inevitably go for cheap or free web analytics solutions. In fact, this is a no-brainer, given the cost of the licenses of proprietary solutions. In these companies, there is only one person in charge of the task of analysing visitor’s behaviour and, usually, this is only one of the many tasks he has to do. As the importance of web analytics grows, this person ends up working full time on web analytics. Sometimes, a second person joins this mini-team, but within a broader team that manages all internal analytics and reporting.

The next breakpoint happens when the company as a whole realises the benefits of having a very good understanding of the customer behaviour. In this situation, more and more people want to have access to the statistics gathered by the analytics tool. Not just only the web analysts, but others in the marketing department (or even other departments) want to have first hand data. These other roles do not need full access nor are going to spend their full working day on the web analytics tool, but they want to be able to self serve. As a consequence, more people demand more features and capabilities.

Inevitably, this approach leads to larger analytics teams, with a manager and a team of two or three people that are working full time on all aspects of the analytics: reporting, ad-hoc analysis, implementation, supporting other users… But not only that, another consequence is that free solutions become a problem rather than a solution and more powerful tools are needed, like Adobe Analytics. As an example, I remember one company switching to Adobe Analytics just because their previous solution did not scale well.

The last big change comes when the websites and apps are in continuous development, constantly adding new features, with releases every few weeks. It becomes impossible to keep up with the changes in the development and, at the same time, satisfy the reporting needs. At this point, there is a need to split the web analytics team in two: one dedicated exclusively on the implementation and another on the reporting. These two teams have different line managers, although they are closely connected.

Even in this situation, there is still room for improvement, especially in big corporations. You can see further separations in both teams for apps/web, different websites, desktop/mobile web…

In parallel to all of this process, there is another tip I would like to share. The web analytics job market is fairly small, but very dynamic. There are very few good web analysts and recruiters are always luring web analysts to fill vacancies in other companies. It is therefore very important to have a healthy balance between junior and senior analysts. Both are needed and junior team members need to receive a good training to be able to fill in the gap when more senior members or even the manager, eventually leave the company.

As I said, this is only my experience, but after having worked with dozens of companies, where this pattern tends to repeat, I am inclined to believe that this is the best approach as of today.

Quick tip: track sprint reference

After publishing my previous post, where I recommended tracking the s_code version, I realised it could have been enhanced with a new type of version to track. Another version number that is even more useful is the sprint reference or whichever value you use to track each of the releases of the website. This idea came from one of my customers (thanks Glenn), who is under the process of redeveloping the whole website and analytics implementation. If you either have a development team that is continously adding new features to the web or are in profoundly redeveloping the website, you want to know the ROI of this invesment.

Copying the sprint reference into an eVar, you can create reports that show the revenue or any other important conversion metric for each of the releases. Of course, this will require an extended period of time in the report and equal periods for each release and not all periods are the same. However, with this data, you can show the increment in revenue derived from the new release.