Make Decisions Based on Data Without Special Background

I have spent over a decade working as a business analyst at some of the top digital marketing agencies in the world. This has allowed me to not only learn data science but also to observe how people in other specialties interact with data.

Here’s what I realized:

Data is incredibly powerful.
Analyzing it can help you make better decisions.
But it may not always be obvious how to get actionable insights from it if you don’t have training in data science.

That is why it’s so important to learn to read marketing data. And this short guide will help you do exactly that.

Why You Need To Be Prepared To Play a Detective

People often imagine data analysis like this: you look at the metrics on a dashboard, everything is immediately obvious. Company makes a lot of money. Of course, if that was the case, business analysts would have to find new careers because no one would need them.

The reality is that when you are dealing with big businesses, you often have to go through enterprise marketing data with a fine-tooth comb to figure out what is going. For example, let’s say that the management of an amusement park didn’t do maintenance on their equipment for a long time, which led to several attractions breaking down.

After that, only half of the attractions remained in operation, which resulted in massive queues and annoyed customers. However, despite this disaster, that month turned out to be the most profitable month in the amusement park’s history. What on Earth happened?

‍

Now, when you encounter a situation like that, you may be completely baffled at first. Managerial negligence that leads to broken down attractions and unhappy customers shouldn’t result in an increase in profit. But once you go through the data, you may realize that due to the queues people spent more time in the amusement park restaurants that have big profit margins, which was what led to the record monthly profit.

As you can see, business analysis is a bit like detective work, you have to piece the puzzle together from the available evidence (data). And you need to be prepared to do this detective work if you are serious about getting actionable insights from marketing data.

‍

3 Data Analysis Traps You Need To Avoid

Okay, so before we get into reading data, we need to discuss the three biggest pitfalls that you need to avoid.
‍
#1 Presuppositions

‍
It’s only human to be biased. It’s impossible to approach data without any pre-existing beliefs. That’s why you need to be aware of those pre-existing beliefs and how they may influence your data analysis. For example, let’s say that you run an ad campaign.

Based on your previous experience, you know that this type of campaign usually produces exceptional results. After a month, the campaign delivers great results, so you declare it a success. Now, you pressuposed that this campaign would work well because similar campaigns worked well in the past, so it is tempting to say “I knew it!” and call it a day without looking at the data.

But that would be a mistake. The reality is that until you look at the data, you don’t know why this campaign worked. It may well have been a fluke caused by some unusual event. So forget that you “knew” that it would work. Analyze the data. Why was this campaign successful? Once you figure it out, you will be able to create even better ad campaigns in the future.

Always do your best to prevent your presuppositions from clouding your judgement:

Ask yourself “What do I believe to be true that may not necessarily be true and how that may influence my data analysis?”
Put this presupposition aside and analyze the data as if it was the only information that you had available to you
See what the data says. It may support your initial assumption, it may demolish it. Be prepared to let go of the presupposition you had if the data disagrees with it.

Keep in mind that what you need to be the most wary of is this sense that you “just know” something without looking at the data because it’s “obvious”. You never “just know” and it’s never “obvious” so interpret this feeling of certainty as a sign that you need to be even more diligent with data analysis.

After all, data analysis is about, well, data, not gut feelings.

#2 Simple Explanations

Another temptation that you need to resist is simple explanations. Often when you are analyzing data you will immediately come up with a simple explanation that (seemingly) accounts for what happened. For example, metrics are never static, they are always in flux, so there’s always something going on with the data.

Often this ebb and flow of metrics is just random, but sometimes it’s a result of seasonality. However, seasonality is the simplest explanation, so it may be tempting to just go with it.

But if you just take the easy approach of explaining everything with seasonality, you risk missing out on valuable business insights.

So make sure to carefully go through all the data and check all other potential explanations before you declare that the simplest explanation is the answer.

#3 P-Hacking

You may have heard the term “p-hacking”. It’s a way to interpret data from a study to establish a statistically significant relationship between variables that are actually not related at all.

Here’s how Wired explains it in their article “We Are All ‘P-Hacking’ Now”:

“Suppose you’re testing a pill for high blood pressure, and you find that blood pressures did indeed drop among people who took the medicine. The p-value is the probability that you’d find blood pressure reductions at least as big as the ones you measured, even if the drug was a dud and didn’t work. A p-value of 0.05 means there’s only a 5 percent chance of that scenario. By convention, a p-value of less than 0.05 gives the researcher license to say that the drug produced “statistically significant” reductions in blood pressure.”

Journals generally prefer to publish statistically significant results, so scientists have incentives to select ways of parsing and analyzing their data that produce a p-value under 0.05. That’s p-hacking.”

It’s important to note that scientists usually don’t p-hack on purpose, they simply unintentionally interpret the data in a way that favors the hypothesis they are emotionally invested in.

And scientists are not the only ones guilty of this.

Card from “Cards Against Humanity (https://cardsagainsthumanity.com/)” game — *Card from “Cards Against Humanity” game*

Wharton marketing professors Christopher Van den Bulte and Ron Berman together with an Opendoor data scientist Leonid Pekelis and Facebook research scientist Aisling Scott did a study on p-hacking in marketing. They analyzed over 2,100 A/B tests run with Optimizely software in 2014 from nearly 1,000 accounts. What did they find?

Apparently, marketers were routinely “p-hacking” their A/B tests without even realizing it. Here’s an example the researchers provided:

An experiment is supposed to run for 4-5 weeks. 57% of marketers look at initial results daily and stop the test when it reaches 90% statistical significance. Berman explained that many people wait until the first time the A/B test reaches the 90% threshold and then stop the test.

“The reason it is a mistake is because if you waited a bit longer, you might go below 90%, and below 70% and fluctuate again because it is a random process.”

And here’s a bit that you may find surprising if you are not well-versed in statistics:
“While 70% might still sound high, that confidence level actually means there is no meaningful difference whether one uses webpage version A or B.”

Basically, the takeaway here is to let the experiment to run its course, even if you think that you already got the results. The reason p-hacking is so prevalent in business is presumably the same reason why it’s so prevalent in science: you are incentivized to do experiments that produce results.

But the reality is that most A/B tests don’t lead to any significant discoveries. You must accept that and be okay with it. Keep in mind that in business, p-hacking leads to mistakes, and mistakes in enterprise marketing can be very, very expensive.

That is why I suggest you to take time to learn more about A/B testing statistics as well as the general concept of p-hacking.
How To Read Enterprise Marketing Data

Now let’s get into reading the data.

Always start with a clear goal in mind. Don’t just “look at the data”, analyze a particular metric
When you are analyzing a particular metric, break it down into its component metrics, then analyze them. For example, the “revenue” metric can be split into “number of clients”, “average order value”
Go through the data methodically. For example, if you’re analyzing revenue, first complete analysis of “number of clients” and only then go to the “average order value”. Jumping around increases the chances that you will miss something important.

There may be cases when you can jump from one metric to another to test a hypothesis. For example let’s say that you are looking at the number of clients, and you see that it dropped.

You have a hypothesis that it happened due to the company changing its priorities toward big clients. So you may want to go to the average customer value metric, quickly check it to see if your hypothesis holds water, then go back to the number of clients.

That being said, it’s best to avoid jumping around as much as possible because, as I have already mentioned, it increases the odds of missing things.

You need to be methodical if you want to get the most out of the data that you have. You need to patiently go metric by metric, column by column, number by number. Sure, it’s not exactly glamorous, but it’s how business analysis is done. And it works.

Conclusion

We are fortunate to have so much marketing data at our fingertips. However, having that data is not enough, you also need to know how to read it. And you don’t have to become a data scientist to be able to benefit from the information that is available to you.

All you need to do is invest in learning basic statistics as well as basic data science principles. This will help you avoid costly mistakes, make better decisions, and get ahead of the competition.

It will be well worth the effort.