Statistical Significance vs Clinical Importance

Let’s keep this simple. Researchers usually begin with a hypothesis and use statistical methods to evaluate whether the observed data are reasonably compatible with a particular statistical model. You may have seen: p < 0.05

This is commonly used as a threshold for calling a result “statistically significant.” But a p-value does not tell us:

• how large the treatment effect was
• how important the improvement was
• whether the patient noticed the difference
• whether the treatment caused the result with certainty
• whether the result will apply to everyone
• whether the treatment is worth using

A p-value answers a much narrower statistical question. That’s why: “Statistically significant” does not automatically mean “clinically important.”

Statistical significance asks about the evidence for a difference under a statistical model. Clinical importance asks: “Is the difference large enough to matter in the real world?” For a person receiving treatment, that might mean:
Can I sleep better?
Can I turn my neck comfortably enough to drive?
Can I walk farther?
Can I return to work?
Can I exercise again?
Can I put my shirt on without struggling?
Can I pick up my child?
Can I get through the day with less disruption from my symptoms?
Those outcomes have meaning beyond a statistical calculation. This is why we keep returning to one principle throughout OLO: Measure what matters to your life.

This is where things become interesting. As sample sizes increase, researchers may be able to detect smaller differences with greater statistical precision.

That’s useful. But it also means a very large study can sometimes identify a difference that is statistically convincing while being too small to matter clinically. So: Small p-value ≠ large effect That’s an important sentence. A tiny p-value doesn’t tell you that a treatment had a huge benefit. To understand the size of the benefit, we need to look at something else. Look at the Effect Size

Instead of asking only: “Was it statistically significant?” ask: “How large was the effect?” An effect size describes the magnitude of the difference or relationship being studied.

For example:
How much did pain change?
How much did range of motion improve?
How different were the treatment and comparison groups?
How much did the risk of an outcome change?
Effect size brings us closer to the question we actually care about:
How much difference did the treatment make?

The effect estimate isn’t perfectly certain. That’s why researchers often report a confidence interval around it. Suppose a treatment appears to reduce pain by an average of one point. But the confidence interval is very wide. That means there is considerable uncertainty about the true size of the effect.

Another study may estimate a similar one-point improvement but with a much narrower confidence interval. Same estimated effect. Different precision. So don’t look only at: “What was the result?” Also ask: “How certain are we about the size of that result?”

Researchers sometimes try to estimate how much change is likely to be meaningful to patients. You may encounter terms such as: Minimal Clinically Important Difference or Minimal Important Difference often abbreviated as MCID or MID. The general idea is useful: How much change would be considered important rather than merely measurable?

But don’t turn MCID into another magic number.
What counts as meaningful can depend on:

• the condition
• the outcome being measured
• the starting severity
• the population
• the context
• the measurement method
• the individual person’s goals

A two-point change might be very meaningful in one situation and less meaningful in another. Clinical importance still needs context.

Now let’s reverse the problem. Suppose a small study finds that the treatment group appears to improve more than the comparison group. But the result isn’t statistically significant. Can we conclude: “The treatment definitely doesn’t work”? No. Maybe there truly is little or no difference. But perhaps the study was too small to estimate the effect precisely. Perhaps the confidence interval includes both potentially meaningful benefit and little or no benefit.

The correct interpretation may therefore be: “The result is uncertain.” That’s different from: “We proved there is no effect.” Absence of convincing evidence for a difference is not automatically proof that the treatments are identical.

This is why confidence intervals and study design matter. “Significant” Is an Unfortunate Word Part of the confusion comes from everyday language. Normally, when we say something is significant, we mean: Important. Meaningful. Worth paying attention to. In statistics, “statistically significant” has a much narrower technical meaning.

So a statistically significant finding can be practically unimportant. And a potentially important clinical effect can sometimes remain statistically uncertain. Same word. Very different meaning. No wonder everyone gets confused.

Imagine massage research finds an average improvement in shoulder range of motion. Interesting.
But what does that mean for the person?
Can they reach the shelf that was difficult before?
Put on a jacket?
Return to swimming?
Sleep more comfortably?
Perform their work?

The measurement itself can be useful. But we should connect it back to function. That’s why a clinical outcome might include: Pain, Range of motion, Strength, Movement tolerance, Work capacity, Exercise tolerance, Sleep, Daily activities, Quality of life

A number becomes much more useful when we understand what it means for the person.

Suppose an RCT reports: “Massage significantly improved pain compared with control.” Before writing: “Massage is proven to fix pain,” we should ask: How large was the difference?
What was massage compared with?
What kind of pain?
How long did the effect last?
Did function improve?
What did the confidence interval show?
Was the improvement meaningful to patients?
How good was the study?

And remember: Even a meaningful improvement in pain doesn’t automatically prove the proposed biological mechanism behind the treatment. Outcome evidence and mechanism evidence are different questions.

Let’s say Treatment A improves neck rotation by an average of: 4 degrees more than Treatment B. And the difference is statistically significant. Now ask:

Did four degrees matter? Maybe. Maybe not. It depends on the person and the task. If that change allows someone to comfortably check traffic while driving, perhaps it matters. If they already had sufficient movement for everything they needed to do, perhaps the additional range has little practical value.

This is exactly why: Don’t treat the number treat the person. Research gives us numbers. Clinical reasoning gives those numbers context.

Whenever a research paper uses those words, ask:

1. How large was the effect?
Don’t confuse statistical significance with magnitude.

2. What does the confidence interval show?
How precise is the estimate?

3. Would the difference matter to a patient?
Think beyond the spreadsheet.

4. What outcome was measured?
Pain? ROM? Function? Quality of life?

5. How long did the effect last?
An immediate change and a meaningful long-term outcome are different things.

Those questions will tell you far more than the p-value alone.

Instead of asking: “Was it statistically significant?” ask: “How large was the effect, how certain are we, and would that difference actually matter?”

That’s a much better research question. Because healthcare isn’t performed on p-values. It’s performed with people.

“I’d like my p-value reduced today, please.”

They want to move. Work. Sleep. Exercise. Recover. Participate in life.

So yes statistical significance matters. But it is only part of the story. Statistical significance asks whether the data provide evidence of a difference. Clinical importance asks whether that difference is worth caring about.

Good research interpretation needs both. And good clinical practice adds one more question: “Did it matter to this person?”