This article talks about a term that’s progressively becoming more common in the circles of digital marketing : The Data Layer. A data layer means different things to different people, and my particular goal here is to familiarize Web developers and Architects with the What, Why and How behind this term and offer some foundational thinking that Web developers and architects might find useful when building or enhancing their systems of engagement.

With the advent of digital, marketers have been trying to enable and leverage customer intelligence with advanced analytics, better targeting and personalized experiences to win, serve and retain customers.

To that end your organization might be  deploying a Martech ecosystem to enable some foundational tools that help collect marketing data, turn that data into insights and the insights into optimization of your digital experience and digital marketing.

Some examples of what your foundational martech ecosystem this will enable are:

• The Analytics platform will help you measure and get insights about your customer’s path to purchase, traffic and conversion metrics, as well as the influence of various marketing channels on acquisition of prospects and customers
• The Data Management Platform (DMP) will help leverage first party behavioral and CRM data to have a robust view of your audience and utilize that data to segment and target the appropriate message to an audience segment through your media, email and web channels
• The A/B testing and targeting platform will help you test variations on your online experience deliver personalization based on your audience segments

Your website is the primary source of customer interaction and contextual data that powers all the martech tools. So now when you flip the above picture sideways, you essentially have this one below:

You have the web pages at the bottom, which push data up to the insights layer and the result of that goes up to the optimization layer

So, what’s wrong with this picture? The problem here is that, in the bottom box, the marketing data and the logic to send that data to marketing tools are entwined within every web page. And this results in many challenges because the Integration of marketing tools from the web layer is becoming highly complex:

• Increasing types and quantity of data: The types of data that need to be sent to marketing tools from the web layer are becoming large and varied. Measurements ranging from which tab was clicked to the number of searches executed to what error messages a particular has visitor received are just a few examples of the types of measurements that need to be need to be captured.
• Variations in User Interaction Events: The variations of user interaction events, such as device orientation changes, video player events, social interaction events such as reviews or shares, are growing.
• Complex rules and data integration criteria: The criteria “when” the data needs to be triggered to the marketing tools, such as you may want to trigger a rule when the user is browsing on a mobile device and is in a particular geographical location and comes to your site from a traffic source such as trip advisor and conducts a particular search and is in the 3rd step of the booking processes, you want to trigger a particular event to your analytics and your personalization tool. So, these could involve a combination of conditions that are becoming more complex.
• Complex page designs: Finally, page designs have become more complex with significant interactive logic being incorporated into the front end

Clearly, this is a much bigger challenge than in the early days of analytics when we sent a few pieces of data to an analytics tool.

So all of this complexity increases development time, effort and puts the data flow out of the the pages at a risk of breaking.

And the consequence of the data flow breaking is your marketing team might end up missing data and making the wrong business decisions such as improper allocation of marketing spend, or your customers might see irrelevant emails and Ads, and so on.

And not only the data flow, something even more important is that we are putting the pages themselves at risk of breaking whenever we make changes every time there is a new marketing data requirement.

Imagine breaking the checkout page in your live e-commerce site, just because a developer added some code to send an additional piece data for analytics purposes!

So what is this solution? This solution is a design paradigm is what we are calling the digital data layer, which is what I’m going to talk about.

So what is the digital data layer?

Characterestics

What would a digital data layer for your martech stack look like? Before we get there let us talk about what needs to be the characteristics of what makes a good digital data layer that will make this a long term solution and design pattern that you could rely on.

• Centralized: The data layer needs to be centralized in terms of bringing together the marketing data, as well as the logic and operations around sending that data to marketing tools
• Efficient: The data layer should enable efficiencies in implementation, operation and administration of the data exchange
• Flexible: It should allow for implementation of marketing data requirements that minimizes regression issues either with the website functionality or with tool integration
• Lightweight: It should have a small footprint in terms of the overhead on the page weight and performance
• Governed: It should provide capabilities to enforce checks and balances to comply with both Marketing and IT standards and requirements
• Simple: Finally, the data layer should be simple in design and provide capabilities that make it easy to use and manage by developers

So obviously, for us to build a technology that has all these capabilities needs a lot of work, so this needs to be a combination of design patterns that we need to adopt as well as leveraging tools such as tag managers in the right way to help us with a lot of the heavy lifting. If we do that, it will be enable us to achieve these capabilities.

So now lets take a little deeper look at the Digital Data Layer. In our web pages we separate the structure and content, which is HTML, from the behavior of the page. But without a proper data layer design our dynamic marketing data is forced to live within the HTML or JS on the page. It will always be at risk and will cause the web experience to be at risk when changes are made.

What we are really trying to do here is add a layer that can stand independent and not be impacted by changes to the HTML or JS. All the marketing data across pages will live in this data layer.

In our web pages we separate the structure and content, which is HTML, from the behavior of the page. But without a proper data layer design our dynamic marketing data is forced to live within the HTML or JS on the page. It will always be at risk and will cause the web experience to be at risk when changes are made.

What we are really trying to do here is add a layer that can stand independent and not be impacted by changes to the HTML or JS. All the marketing data across pages will live in this data layer.

Now, there are two abstractions within the digital data layer. Lets look at each of these.

Website Data Layer

The website data layer is the data object that holds all the marketing data. This could be any javascript object with a simple easy to understand structure such as JSON.

This object will be predefined and leveraged in the pages to set the appropriate values.

If we do this then web developers will only need to worry about setting values in this data layer without concerning themselves with how the data will be assembled into tags or be consumed by marketing tools.

This is what a Website data layer (WDL) for basic ecommerce website could look like:

Tag Management System Data Layer

A Tag Management system such as Adobe DTM or Ensighten Manage are both great Tag Management Systems to consider for your TMS data layer.

The website marketing data layer is sucked into the Tag Management System data layer and becomes a subset of it.

Any universal DOM elements you need (page metadata, querystring parameters, cookies, browser details, javascript variables etc.).

If we are doing any page scraping, those values are pushed into the Tag Management Data Layer also.

And finally, any data from other data internal or external data source that can be pulled from a javascript request can be pulled into this.

Separation of Data Layers

Let’s take a conceptual look of how we will use the data layer to send data to your Analytics platform.

But before that lets get a common understanding of the terminology used by Adobe Analytics.

Web Analytics reporting mainly includes three types of reports : traffic reports, conversion reports and pathing reports.

Traffic Reports

Traffic reports give you insight on what gets visitors through the door and how visitors navigate your website. With these reports, you can determine which campaigns generate what traffic, the percentage of audience visiting from paid channels vs owned channels., determine popular site content, pages viewed, etc. Traffic reports are usually driven through traffic variables.

Conversion Reports

 Conversion reports display what makes visitors “convert”. Conversion is measured based on success indicators that you define for your website. With these reports you can measure metrics such as product purchase, conversion rates by campaigns, your advertising effectiveness, customer loyalty, etc. Conversion reports are usually driven through conversion variables.

Pathing Reports

Pathing reports enable you to track and record entire browsing paths of visitors. They display information about the order in which pages of your website are accessed. You could gather insights on where a visitor goes before and after any page they visited on your site. You can uncover full paths, most popular paths, longest paths, explain customer fallouts, how customer interaction patterns change over time, etc.

In order for us to be able to generate these reports, our web development teams need to capture tracking information and send it to the Web Analytics platform. For example, if we want to see what type of internal campaigns on our homepage (such as special offers, promotions, etc) might result in purchases, our web team must first capture the tracking codes for the Analytics platform to track the pages. When a success event is completed (like purchase) by a site visitor, the credit for that success is given to any conversion variables that are persisted on the visitor, the campaign ID being one of them. A report of success variables against campaigns will show which campaign generated the most conversion.

Traffic Variables

Traffic variables are usually counters called ‘props’ that count the number of times each value is sent to analytics. If we didn’t use a data layer, they would be embedded in the analytics code on each page. For example, if we want to see the most popular product, we can accomplish this by allocating one of our traffic variables to the product name. We could then implement code to pass the the product name on pages like the product details page into analytics, which will count the number of times the product name was sent and use it to determine the “popularity” of the product. With the data layer we just need to set this value in the object as part of the page.

Success Events

Success events are basically completed visitor actions on your website and these are usually counters, but also currency and numeric values could be used for example total bookings.

Conversion Variables

Conversion variables are used to determine which dimensions of the site contribute the most to success events. These are called eVars in Analytics. For example a conversion variable may be used to identify how well a promotion on your homepage brings visitors to purchase. When a visitor clicks the internal promotion you can use a conversion variable eVar to store a unique identifier for that promotion. When the same visitor completes a purchase and a success event is fired, the original unique identifier receives credit for the purchase event. Another example is to how logged in visitors compare to non logged in visitors for purchase completion. When a visitor logs in, an eVar is set to logged-in and if the visitor reaches the checkout page, the checkout event is attributed to the logged-in value. The resulting analytics report shows the total number of checkouts for logged in and non loggedin visitors.

When a conversion variable is set to a value for a visitor, Analytics remembers that value until it expires. Any success events that a visitor encounters while the conversion variable is active is counted towards that conversion variable value. A conversion variable can be visit based or function similar to cookies.

So with that, lets take a look at the flow of data between your website and analytics with a data layer.

Example

Let us take an example where we want to track products that to see the most popular ones. To do that, we need to set a prop in Adobe Analytics, which maps to the product name. So the data (name-value pairs) sent to Analytics should look like
prop1 = “product-name”, where product-name is the name of the product that the visitor viewed on the website. Let’s look at the setup and the execution for this use case.

Setup

Step 1: We define the attribute productName in the dataLayer object and populate this value within server-side logic. This could be grabbing the product name within the CMS or from a database, and so on.

This is all we need to do on the WDL. Everything else is managed in the TMS data layer.

Step 2: We import the dataLayer object into the TMS. This is an optional step and may not be supported by all TMS. Adobe DTM supports this in order for it to be in sync with the values of the dataLayer object. If your TMS doesn’t support this, you may need to write some logic for values that are constantly changing asynchronously after the page has loaded.

Step 3: Here we decide when and how we send the value from dataLayer.productName into Adobe Analytics. We could decide to send the value based on a variety of critera such as for pages under a specific URL hierarchy, the device type, traffic source, new or returning visitor, and so on. We set the value of dataLayer.productName into a prop.

Step 4: The output of Step 3 is a javascript library that could be deployed on the cloud or on premise.

Execution

Steps A,B,C: Here the page is returned when the visitor requests the page, while the page is being loaded it triggers the load of the javascript library that was deployed in Step 4.

Steps D,E,F: This loads all the logic necessary to send the data to analytics, which executes based on the conditions defined in Step 3 and sends the data to Analytics. Analytics responds back with a 1,1 transparent image pixel as response., which indicates the completion of the data capture.

Benefits

So, now that we reviewed how the data layer works, let us see how we can achieve the benefits we are expecting from a good data layer architecture.

Centralized: The digital data layer is centralized because the Website data layer consolidates all marketing data values into one place and the Tag management data layer centralizes all the event handling logic and and the workflow to send data to the marketing tools.

Efficient: GUI driven configuration of TMS data layer greatly reduces errors and bugs because there is very little hand coding. Additionally, the developers do not need to worry about coding to send user context information because the TMS supports universal DOM data elements out of the box.

Flexible: As we saw earlier, the two-speed approach allows for absolute flexibility and agility for IT to respond to marketing data requests while not compromising on long term maintainability.

Lightweight: So, one of the by-products of centralization is that a single data element in the data layer can be used across multiple tools without needing duplication of data or logic. Also, TMSs that surgically load only the code needed at a point in time to satisfy a particular condition speed up the overall site performance because all that code doesn’t need to be downloaded upfront. Why download the logic for capturing a cart abandonment conversion variable when the user ends up completing a purchase?

Governed: Tag management tools such as DTM and Ensighten provide workflows to create, approve and publish additions and changes to data elements and rules so that the proper technical, organizational and process governance validations can be done. Some tools such as DTM also allow testing across environments, which means you could make changes to a data layer in ‘staging’ and test it against a production instance without actually deploying the code to production. This is great for that final validation of new JS not breaking the existing page.

Simple: With the data layer, form is separated from function. So if you draw a parallel of this architecture with MVC, the developers only need to worry about developing the M (model), everything else is UI driven in the TMS data layer. This makes it much simpler than development without the data layer.

Conclusion

The digital data layer is foundation for using digital channels to track and measure customer behavior. It frees you from relying on silos of visitor data and therefore is a major essential step in effectively moving toward personalization, optimization, and deeper analytics. Adopting this combination of a website data layer and a TMS data layer will help you set your marketing and IT organizations for success in getting the most out of your martech ecosystem.