Home Latest news Understanding Google Analytics for campaign evaluation: Why GA can’t be trusted on its own

Understanding Google Analytics for campaign evaluation: Why GA can’t be trusted on its own

Written by: Rose Mountague | 23rd February 2022

10 min read

Our Associate Director Tom Salmon writes about Google Analytics and why the numbers you see need to be treated with caution.

Google Analytics is one of the most popular analytics tools on the web. Its ubiquity means it has become a common measurement tool for public bodies and charities to measure the success of their campaigns.

However, it is becoming less and less of a reliable measure of website traffic. If you’re relying on it to understand how many people are visiting your website, analyse their behaviour and evaluate your campaigns you could be drawing the wrong conclusions.

More and more people are taking action to opt-out of being tracked on the web. Browsers that protect the identity of their users and operating systems that prevent the sharing of personal data are becoming commonplace. There are even European rulings that Google Analytics might not be GDPR compliant.

In this article, we outline how Google Analytics works using cookies, how the world of the cookie is crumbling and why Google Analytics shouldn’t be used on its own to evaluate the success of digital campaigns.

There are a number of reasons why your Google Analytics might be giving you the wrong data. We’re not going to focus on some of the ‘common mistakes’ that can lead to broken Google Analytics data (for example Tracking codes being implemented incorrectly across the site, improper goal set up and broken campaign tags). Other important considerations such as the differences between sessions and clicks and discrepancies in data relating to paid campaigns on Google and Facebook are important to note and will be the topic of another blog in the future. You can find out more about the discrepancies here on Google Ads and Facebook Ads.

Instead of covering those topics, we’re going to focus on a set of bigger trends that are going to affect the utility of platforms like Google Analytics in the future.

They relate to user privacy and cookies. We’ll get on to that later, but let’s start with a run-through of how Google Analytics works.

How does Google Analytics work?

Google Analytics collects information about visitors to your website using a programming language called Javascript. Javascript has been around for about 25 years and is a language that allows website developers to make websites interactive. Unlike HTML (which your computer can read but can’t interact with) Javascript allows your computer or smartphone to send and receive messages to other computers.

As a user navigates between web pages, Google Analytics uses JavaScript tags (libraries) to record information about the page a user has seen, for example, the URL of the page.

The Google Analytics JavaScript libraries use HTTP cookies to “remember” what a user has done on previous pages/interactions with the website.

Cookies recognise your computer as it travels between Web pages — so you need them for critical things like logging into a website, or buying something online. They are small text files that live on your computer, and the information they contain is set and accessed by the servers of the websites that you visit. Cookies allow servers to identify you and remember things about you.

There are different types of cookies. Some store information about your browsing session, what you looked at, what your viewing preferences are, what’s in your shopping basket, etc. Lots of websites use these to remember you when you go to make a return visit so that, for example, you don’t have to sign in again. Cookies that are set by and read by the website are classed as first-party cookies.

Third-party cookies are cookies set by a website other than the one you are currently on. For example, a website might have some JavaScript code on it to render a Facebook “Share” or “Like” button. The code, written by Facebook, will do the rendering but also, and some might say sneakily, store a cookie on the visitor’s computer and that cookie can later be accessed by Facebook to identify the visitor and, rightly or wrongly, see which websites they visited across the web.

The most habitual third party cookie setter is Google, specifically Google Analytics.

When someone visits a website that is using Google Analytics, the site asks their computer to download a set of Javascript instructions (the cookie) from Google Analytics. Those instructions then ask their browser to send information to Google’s data collection servers.

Google then presents this information back to you in your Google Analytics dashboards, and you could be forgiven for thinking that the data represents an accurate view of how people are using your website.

That isn’t necessarily the case though. Because Google Analytics relies on cookies, the system can’t collect data for users who have disabled them, who don’t give consent or for users that are using a browser that prevents cookies.

Ad blockers and Google Analytics

People are growing more aware of how cookies affect their use of the internet and are increasingly opting out or blocking anything that can track them for advertising or marketing purposes.

In fact, 36% of people in the UK use an ad blocker (AudienceProject 2021).

There are lots of people who, even if they don’t use an AdBlocker, refuse cookies when a pop-up cookie banner asks them to accept them.

e-Privacy, GDPR and cookie banners impact Analytics

Cookie banners were brought in following the e-Privacy Directive which changed the law on cookies in 2011. It made the introduction of the cookie banner necessary as a means of securing consent to serve cookies to visitors to your website. This gives visitors to your website the option to accept or reject cookies. If they ignore your cookie Banner or reject your cookies you won’t see them in Google Analytics. The other legislation that regulates the use of cookies is GDPR, which defines what consent means, and the Privacy and Electronic Communications Regulations (PECR) which cover the use of cookies and similar technologies for storing information, and accessing information stored, on a user’s equipment such as a computer or mobile device.

PECR states that organisations must tell people if they set cookies, and clearly explain what the cookies do and why. They must also get the user’s consent. Consent must be actively and clearly given. There is an exception for cookies that are essential to provide an online service at someone’s request (e.g. to remember what’s in their online basket, or to ensure security in online banking).

The Information Commissioner’s Office (ICO), responsible for policing the e-Privacy Directive and GDPR, is clear that Analytics cookies are not essential and so need consent.

They say, “While we recognise that analytics can provide you with useful information, they are not part of the functionality that the user requests when they use your online service – for example, if you didn’t have analytics running, the user could still be able to access your service. This is why analytics cookies aren’t strictly necessary and so require consent.”

The issue of cookies and informed consent is a hot topic for the whole digital economy. As Justin Schuh, Director of Chrome Engineering at Google said, “Users are demanding greater privacy – including transparency, choice and control over how their data is used – and it’s clear the web ecosystem needs to evolve to meet these increasing demands.”

A number of high profile rulings in Europe have brought into question whether Google Analytics is GDPR compliant. EU lawmakers, led by the Belgium Data Protection Authority, recently ruled that the popups asking people for consent whenever they land on a site are illegal. That means all data collected via those popups from more than 1,000 companies including Google and Amazon must be deleted as of a result. The IAB Europe, the body that oversees the consent framework on behalf of advertisers in Europe, now has until May 2022 to find those actions before they’re submitted to the ruling’s leading regulator — the Belgian Data Protection Authority.

The consequences of this ruling are likely to challenge how data is collected and are another example of how the rights of individual people to control their own data are challenging how the web has been built over the last 20 years.

Safari and iOS14

Some browsers like Firefox and Safari and operating systems like iOS14 already allow their users to block cookies. Apple’s newest operating system iOS14 in particular restricts the data that is shared between different apps, blocking third-party cookies and having a big impact on the figures that are seen in Google Analytics.

To give an idea of how big this problem is in terms of its impact on accurate Google Analytics data, iOS represents 53% of the UK market, and 9/10 iOS users are running iOS14 (chart below).

iOS14.5 automatically opts users out of tracking. Users must actively agree to permit a mobile app to use their Apple Identifier for Advertisers (IDFA) and any other identifiers. This forces apps to be more transparent with how they use and track users’ data. If the user does not grant permission, outside data cannot be used and data cannot be shared outside the app. According to data collected by Flurry Metrics, 96% of iPhone users opted out of app tracking in the latest iOS 14.5 update.

The third-party cookie is crumbling. What are the alternatives?

These trends have significant implications for communications teams and campaign evaluators who are still relying on cookie-based technologies such as Google Analytics to help them assess the impact of their work.

It is more important than ever to look at holistic evaluation – not just at indicative metrics – in order to understand campaign effectiveness and impact.

With European rulings putting ever more pressure on Google it is worth considering how you can build different forms of measurement into your campaign evaluations.

Google itself has announced that it will deprecate third party cookies from its Chrome browser in 2023. It will join the likes of Safari and Firefox that have already implemented some blocking against third party cookies.

Organisations that plan for the phasing out of third party cookies now will have the benefit of understanding the impact of their campaigns more accurately in the future. There is a lot of value in creating benchmark data now in order to better evaluate future campaigns.

So what are the alternatives?

The jury is still out and there is a lot to be resolved in the whole post-cookie space but it’s likely that server-side tracking and analytics represent a more sustainable option. It is possible to run Google Analytics from your own servers. This turns third party cookies into first-party cookies and prevents most of your data from being blocked by ad blockers or browsers. It isn’t free, and requires more setup than ‘client side’ Google Analytics, but will give you a more robust set of analytics.

Of course, when it comes to an evaluation tool for behaviour change campaigns, Google Analytics is only ever a measure of output rather than an outcome and should be used alongside other evaluation tools that measure the real impact that the campaign has had.