Becoming a more data-driven decision-maker can bring
several benefits to your organization, enabling you to identify new
opportunities to pursue and threats to abate. Rather than allowing
subjective thinking to guide your business strategy, backing your
decisions with data can empower your company to become more
innovative and,
ultimately, profitable.
If you’re new to data-driven decision-making, you might be wondering
how data
translates into business strategy. The answer lies in generating a
hypothesis
and verifying or rejecting it based on what various forms of data
tell you.
What Is Hypothesis Testing?
To understand what hypothesis testing is, it’s important first to
understand what
a hypothesis is.
A hypothesis or hypothesis statement seeks to explain why something
has happened,
or what might happen, under certain conditions. It can also be used
to understand how
different variables relate to each other. Hypotheses are often
written as if-then statements;
for example, “If this happens, then this will happen.”
Hypothesis testing, then, is a statistical means of testing an
assumption stated in a hypothesis.
While the specific methodology leveraged depends on the nature of
the hypothesis and data available,
hypothesis testing typically uses sample data to extrapolate
insights about a larger population.
Hypothesis Testing In Business
When it comes to data-driven decision-making, there’s a certain
amount of risk that can mislead a professional.
This could be due to flawed thinking or observations, incomplete or
inaccurate data, or the presence of unknown
variables. The danger in this is that, if major strategic decisions
are made based on flawed insights, it can
lead to wasted resources, missed opportunities, and catastrophic
outcomes.
The real value of hypothesis testing in business is that it allows
professionals to test their theories and assumptions
before putting them into action. This essentially allows an
organization to verify its analysis is
correct before committing resources to implement a broader strategy.
As one example, consider a company that wishes to launch a new
marketing campaign to revitalize sales
during a slow period. Doing so could be an incredibly expensive
endeavor, depending on the campaign’s
size and complexity. The company, therefore, may wish to test the
campaign on a smaller scale to
understand how it will perform.
In this example, the hypothesis that’s being tested would fall along
the lines of: “If the company
launches a new marketing campaign, then it will translate into an
increase in sales.” It may
even be possible to quantify how much of a lift in sales the company
expects to see from the
effort. Pending the results of the pilot campaign, the business
would then know whether it
makes sense to roll it out more broadly.
Key Considerations For Hypothesis Testing
1. Alternative Hypothesis and Null Hypothesis
In hypothesis testing, the hypothesis that’s being tested is known
as the alternative hypothesis.
Often, it’s expressed as a correlation or statistical relationship
between variables.
The null hypothesis, on the other hand, is a statement that’s meant to
show there’s no
statistical relationship between variables being tested. It’s typically
the exact opposite
of whatever is stated in the alternative hypothesis.
For example, consider a company’s leadership team who historically
and reliably
sees $12 million in monthly revenue. They want to understand if
reducing the price
of their services will attract more customers and, in turn, increase
revenue.
In this case, the alternative hypothesis may take the form of a
statement such as:
“If we reduce the price of our flagship service by five percent,
then we’ll see an
increase in sales and realize revenues greater than $12 million
in the next month.”
The null hypothesis, on the other hand, would indicate that revenues
wouldn’t increase
from the base of $12 million, or might even decrease.
2. Significance Level and P-Value
Statistically speaking, if you were to run the same scenario 100
times, you’d likely receive somewhat different
results each time. If you were to plot these results in a
distribution plot, you’d
see the most likely outcome is at the tallest point in the graph,
with less likely
outcomes falling to the right and left of that point.
With this in mind, imagine you’ve completed your hypothesis test and
have your results,
which indicate there may be a correlation between the variables you
were testing.
To understand your results' significance, you’ll need to identify a
p-value for the test,
which helps note how confident you are in the test results.
In statistics, the p-value depicts the probability that, assuming the
null hypothesis is correct,
you might still observe results that are at least as extreme as the
results of your hypothesis test.
The smaller the p-value, the more likely the alternative hypothesis
is correct, and the greater the
significance of your results.
3. One-Sided vs. Two-Sided Testing
When it’s time to test your hypothesis, it’s important to leverage
the correct testing method.
The two most common hypothesis testing methods are one-sided and
two-sided tests, or one-tailed and
two-tailed tests, respectively.
Typically, you’d leverage a one-sided test when you have a strong
conviction about the direction
of change you expect to see due to your hypothesis test. You’d
leverage a two-sided test when you’re
less confident in the direction of change.
4. Sampling
To perform hypothesis testing in the first place, you need to collect
a sample of data to be analyzed.
Depending on the question you’re seeking to answer or investigate,
you might collect samples through
surveys, observational studies, or experiments.
A survey involves asking a series of questions to a random population
sample and
recording self-reported responses.
Observational studies involve a researcher observing a sample
population and collecting
data as it occurs naturally, without intervention.
Finally, an experiment involves dividing a sample into multiple
groups, one of which acts
as the control group. For each non-control group, the variable being
studied is manipulated
to determine how the data collected differs from that of the control
group.