Interpreting Confidence Intervals

What confidence level actually means

Interpreting Confidence Intervals

What Does "95% Confident" Mean?

Confidence level describes the method, not a specific interval

Correct interpretation: "If we repeated this sampling process many times and constructed a 95% CI each time, about 95% of those intervals would contain the true parameter."

NOT:

  • "95% chance the parameter is in this interval" (parameter is fixed!)
  • "95% of the data falls in this interval"
  • "We are 95% sure this interval contains the parameter"

Visualizing Confidence Level

Imagine 100 different samples:

  • Each produces different CI
  • About 95 capture true parameter (green)
  • About 5 miss true parameter (red)

Our interval is one of these – we don't know if it's green or red!

Example: Correct vs Incorrect

95% CI for mean: (45, 55)

✓ Correct: "We are 95% confident the true mean is between 45 and 55."

✓ Correct: "If we repeated sampling many times, 95% of intervals would capture the true mean."

✗ Incorrect: "There is a 95% probability the mean is between 45 and 55."

✗ Incorrect: "95% of data values are between 45 and 55."

✗ Incorrect: "The sample mean has a 95% chance of being in this interval."

Components of Interpretation

Good interpretation includes:

  1. Confidence level: "We are 95% confident..."
  2. Parameter (not statistic): "...the true mean (or proportion)..."
  3. Context: "...test score for all students..."
  4. Interval: "...is between 73 and 82."

Template: "We are [C]% confident that the true [parameter in context] is between [lower bound] and [upper bound]."

Context Matters

Generic: "We are 95% confident μ is between 45 and 55."

Better: "We are 95% confident the mean height of adult males is between 45 and 55 inches."

Even better: "We are 95% confident the mean height of adult males in California is between 45 and 55 inches."

Always state parameter in context of the problem!

Margin of Error Interpretation

CI = statistic ± ME

Interpretation of ME: "We estimate the parameter is within [ME] of [statistic] with [C]% confidence."

Example: ME = 3, xˉ\bar{x} = 50, 95% confidence

"We estimate the true mean is within 3 of our sample mean of 50 with 95% confidence."

Width of Interval

Narrower interval:

  • More precise estimate
  • But requires larger sample or lower confidence

Wider interval:

  • Less precise
  • But higher confidence or smaller sample

Trade-off: Precision vs confidence

Factors affecting width:

  1. Confidence level: Higher → wider
  2. Sample size: Larger → narrower
  3. Population variability: More variable → wider

Comparing Intervals

Two non-overlapping intervals suggests difference

Example:

  • Group 1: (52, 58)
  • Group 2: (65, 71)

No overlap → strong evidence of difference

Two overlapping intervals:

  • May or may not be significant difference
  • Need formal hypothesis test to determine

Using CI for Decisions

Testing H₀: μ = μ₀ at α significance level

Equivalent to: Check if μ₀ is in (1-α) CI

Example: H₀: μ = 50, α = 0.05, 95% CI: (52, 58)

50 not in interval → Reject H₀

But: CI gives MORE information than test (plausible range of values)

Two-Sided vs One-Sided

Two-sided CI: Interval (L, U)

  • Most common
  • Symmetric around estimate

One-sided CI:

  • Upper bound: (-∞, U)
  • Lower bound: (L, ∞)
  • Less common
  • For directional questions

Practical vs Statistical Significance

Statistically significant: Interval doesn't contain null value

Practically significant: Interval contains values that matter in practice

Example: CI for improvement: (0.5, 2.5) points on 100-point test

  • Statistically significant (doesn't contain 0)
  • But practically? Is 0.5-2.5 point improvement meaningful?

Always consider both statistical AND practical significance!

Common Misinterpretations

❌ "95% of the data is in the interval"

  • No! Interval is for parameter (mean/proportion), not individual values
  • Prediction interval for individuals (different calculation)

❌ "There's a 95% probability μ is in the interval"

  • No! μ is fixed (not random). Interval is random.
  • Either μ is in it (probability 1) or not (probability 0)

❌ "We are 95% confident the sample mean is in the interval"

  • No! We KNOW sample mean (it's the center of the interval!)
  • Confident about population mean, not sample mean

❌ "95% of all samples will give this interval"

  • No! Different samples give different intervals
  • 95% of intervals (not samples) capture μ

Confidence vs Probability

Probability: Long-run frequency (objective)

  • Coin has 50% probability of heads

Confidence: Measure of method reliability

  • Method produces correct intervals 95% of the time
  • But specific interval either right or wrong

Subtle but important distinction!

Reporting Confidence Intervals

In writing:

  • State interval with confidence level
  • Interpret in context
  • Include units

Example report: "Based on a random sample of 100 students, the 95% confidence interval for mean study time is (8.2, 10.8) hours per week. We are 95% confident that the true mean study time for all students is between 8.2 and 10.8 hours per week."

Limitations of Confidence Intervals

CI only valid if:

  • Conditions met (random, normal, independent)
  • No bias in data collection
  • No measurement errors
  • Proper statistical procedure used

CI doesn't account for:

  • Sampling bias
  • Response bias
  • Measurement error
  • Non-random sampling

Garbage in, garbage out! CI from biased sample is meaningless.

Choosing Confidence Level

Common choices:

  • 90% (less stringent, narrower)
  • 95% (standard in many fields)
  • 99% (very stringent, wider)

Higher confidence:

  • Safer (more likely to capture parameter)
  • But less precise (wider interval)

Choice depends on:

  • Consequences of being wrong
  • Field conventions
  • Desired precision

Quick Reference

Correct interpretation template: "We are [C]% confident that the true [parameter in context] is between [L] and [U]."

Common mistakes to avoid:

  • Probability statements about parameter
  • Statements about data/sample
  • Forgetting context
  • Confusing confidence with probability

Remember: Confidence describes the method's reliability, not probability that this specific interval is correct. Always interpret in context with proper terminology!

📚 Practice Problems

1Problem 1easy

Question:

A 95% confidence interval for mean SAT score is (1180, 1220). Which of the following interpretations is correct?

A) There is a 95% probability that the true mean is between 1180 and 1220. B) 95% of students scored between 1180 and 1220. C) We are 95% confident that the true mean SAT score is between 1180 and 1220. D) If we took many samples, 95% would have means between 1180 and 1220.

💡 Show Solution

Step 1: Evaluate option A "There is a 95% probability that the true mean is between 1180 and 1220."

INCORRECT! ✗

Why wrong:

  • μ is a fixed parameter (not random)
  • Either μ is in interval or it isn't
  • Can't assign probability to fixed value
  • The INTERVAL is random, not μ

Step 2: Evaluate option B "95% of students scored between 1180 and 1220."

INCORRECT! ✗

Why wrong:

  • This describes INDIVIDUAL scores
  • CI is about the MEAN, not individuals
  • Individual scores have much more variability
  • Confuses parameter with population values

Step 3: Evaluate option C "We are 95% confident that the true mean SAT score is between 1180 and 1220."

CORRECT! ✓

Why correct:

  • Properly describes confidence in the interval
  • "Confident" (not "probability")
  • About the parameter μ
  • Standard correct interpretation

Step 4: Evaluate option D "If we took many samples, 95% would have means between 1180 and 1220."

INCORRECT! ✗

Why wrong:

  • This describes sampling distribution of x̄
  • NOT what CI says
  • Different samples give different intervals
  • Confuses interval for μ with distribution of x̄

Step 5: Proper understanding of "95% confident" Means:

  • Our METHOD captures true μ 95% of the time
  • If we repeated sampling many times
  • About 95% of resulting CIs would contain μ
  • This PARTICULAR interval either does or doesn't

NOT:

  • 95% probability μ is in this interval
  • μ moves around randomly
  • We're describing μ's distribution

Step 6: Visual explanation Imagine 100 different samples:

  • Each produces different CI
  • About 95 intervals contain true μ
  • About 5 intervals miss μ

We have one interval from one sample We're 95% confident it's one of the "good" intervals

Step 7: Common misconceptions WRONG: "95% probability μ is in (1180, 1220)"

  • μ is fixed, not random

WRONG: "95% of data is in (1180, 1220)"

  • CI is for μ, not for individuals

WRONG: "95% of sample means are in (1180, 1220)"

  • CI is for μ, not for x̄

RIGHT: "95% confident μ is in (1180, 1220)"

  • Confidence in our method

Answer: C is correct

"We are 95% confident that the true mean SAT score is between 1180 and 1220."

This correctly describes confidence in the interval containing the parameter, not a probability statement about where the parameter is.

2Problem 2easy

Question:

Explain what "95% confidence" means in the context of confidence intervals.

💡 Show Solution

Step 1: The correct interpretation "95% confidence" means: If we repeated our sampling procedure many times and constructed a CI each time, approximately 95% of those intervals would contain the true parameter value.

Step 2: What confidence is about Confidence describes:

  • The RELIABILITY of the method
  • The LONG-RUN success rate
  • The PROCEDURE, not a single interval

Confidence does NOT describe:

  • Probability the parameter is in THIS interval
  • Where the parameter "probably" is
  • How likely different values are

Step 3: The random element What's random:

  • The SAMPLE we get
  • The INTERVAL we construct
  • Which intervals capture μ

What's NOT random:

  • The true parameter μ
  • Whether μ is in our interval
  • The population

Step 4: Simulation example Imagine we:

  1. Take 100 different random samples
  2. Construct 95% CI from each
  3. See which intervals contain true μ

Result:

  • About 95 intervals contain μ ✓
  • About 5 intervals miss μ ✗
  • Some intervals higher than μ
  • Some intervals lower than μ
  • But ~95% capture it

Step 5: Our single interval We have ONE interval from ONE sample We don't know if it's "good" or "bad" But we're "95% confident" because:

  • Our method is right 95% of the time
  • We used a reliable procedure
  • Probably one of the 95%, not the 5%

Step 6: Common mistakes WRONG: "95% probability μ is in the interval"

  • μ doesn't move around
  • Can't assign probability to fixed value

WRONG: "μ is definitely in the interval"

  • Could be in the unlucky 5%
  • Not absolute certainty

RIGHT: "95% confident μ is in the interval"

  • Describes reliability of method
  • Long-run interpretation

Step 7: Analogy Like quality control: "95% of products meet specifications"

Doesn't mean:

  • THIS product has 95% chance of being good
  • Each part of product is 95% good

Means:

  • 95% of products pass inspection
  • Good manufacturing process
  • Confident in the process

Step 8: Why this matters Understanding confidence:

  • Prevents overconfidence
  • Acknowledges uncertainty
  • Recognizes sampling variability
  • Proper statistical reasoning

Answer: "95% confidence" means that if we repeated the sampling process many times, about 95% of the resulting confidence intervals would contain the true parameter value. It describes the reliability of our method, not the probability that this specific interval contains the parameter (which either does or doesn't, with no probability about it).

3Problem 3medium

Question:

A news report states: "A poll shows 52% support the policy, with a margin of error of ±3%." What does this mean? Can we conclude the policy has majority support?

💡 Show Solution

Step 1: Interpret the statement Point estimate: p̂ = 0.52 (52%) Margin of error: ME = 0.03 (3%) Implied CI: 52% ± 3% = (49%, 55%)

Usually implies 95% confidence (though should be stated!)

Step 2: What the interval means We are 95% confident (assuming 95% CI) that: The true proportion of support is between 49% and 55%

Step 3: Does this prove majority support? Majority means p > 0.50 (more than 50%)

The interval is (0.49, 0.55)

  • Some values are below 50% (like 49%)
  • Some values are above 50% (like 55%)

We CANNOT conclusively say majority supports!

Step 4: Why not conclusive? True p could be:

  • 49% (minority support) ✗
  • 50% (exactly half)
  • 51% (slight majority)
  • 55% (clear majority) ✓

All are plausible values within our CI!

Step 5: What can we say? We CAN say:

  • Support is CLOSE to 50%
  • Could be slightly below or above majority
  • Not enough evidence to conclusively claim majority
  • "Statistical tie" or "too close to call"

We CANNOT say:

  • Definitely has majority support
  • Definitely lacks majority support
  • 52% is the exact true value

Step 6: The 50% threshold Since 50% is IN the interval (0.49, 0.55):

  • 50% is a plausible value for true p
  • Can't rule out "exactly half"
  • Not statistically significant above 50%

If interval were (0.51, 0.57):

  • All values above 50%
  • Could claim majority support
  • Statistically significant

Step 7: Reporting considerations Responsible reporting should say: "Support appears close to 50%, but we cannot conclusively determine if a majority supports the policy. The true level of support is likely between 49% and 55%."

Misleading to claim: "52% support, so majority supports" (Ignores margin of error!)

Step 8: Connection to hypothesis testing This relates to testing: H₀: p = 0.50 Hₐ: p > 0.50

Since 0.50 is in the CI:

  • Don't reject H₀
  • Insufficient evidence for majority
  • Results "not statistically significant"

Answer: The poll estimates 52% support with 95% confidence interval (49%, 55%). We CANNOT conclusively claim majority support because the interval includes values both below and above 50%. True support could be as low as 49% (minority) or as high as 55% (clear majority). This is a "statistical tie" - too close to call with certainty.

4Problem 4medium

Question:

Two studies estimate mean height: Study A gives CI (66, 70) inches, Study B gives (65, 71) inches. Which study is more precise? Can we tell which is more accurate?

💡 Show Solution

Step 1: Define precision vs accuracy PRECISION: How narrow the interval is

  • Narrower interval = more precise
  • Less uncertainty
  • Smaller margin of error

ACCURACY: How close to true value

  • Does interval contain true μ?
  • Can't know from CI alone!

Step 2: Compare precision Study A: (66, 70) Width = 70 - 66 = 4 inches ME = 2 inches

Study B: (65, 71) Width = 71 - 65 = 6 inches ME = 3 inches

Study A is MORE PRECISE (narrower interval)

Step 3: Why different precision? Possible reasons:

  1. Different sample sizes

    • Study A might have larger n
    • ME ∝ 1/√n
  2. Different variability

    • Study A might have smaller s
    • Less variable population/sample
  3. Different confidence levels

    • Study A might use 90% confidence
    • Study B might use 99% confidence

Most likely: Study A has larger sample size

Step 4: Compare accuracy We CANNOT determine which is more accurate!

Why?

  • Don't know true μ
  • Both intervals might contain μ
  • One might contain μ, other might not
  • Neither might contain μ (both in unlucky 5%)

Accuracy requires knowing truth

Step 5: Possible scenarios Scenario 1: True μ = 68 inches

  • Both intervals contain 68 ✓
  • Both accurate!
  • Study A more precise

Scenario 2: True μ = 72 inches

  • Neither interval contains 72 ✗
  • Neither accurate!
  • Study A more precise but still wrong

Scenario 3: True μ = 65.5 inches

  • Only Study B contains 65.5
  • Study B accurate, Study A not
  • Study A more precise but less accurate!

Step 6: The tradeoff Precision vs Coverage:

  • Can have precise but wrong interval
  • Can have wide but correct interval
  • Want both: precise AND accurate

Confidence level affects this:

  • Higher confidence → wider interval → more likely to be accurate
  • Lower confidence → narrower interval → more precise but riskier

Step 7: What we can say About precision: ✓ Study A is more precise (narrower interval) ✓ Study A has smaller margin of error ✓ Study A probably had larger sample

About accuracy: ✗ Cannot determine which is more accurate ✗ Don't know if either contains true μ ✗ Would need to know true population mean

Step 8: Practical implications If both studies are well-conducted:

  • Prefer Study A (more precise)
  • Assuming same confidence level
  • Gives more specific estimate

But if Study A used 80% confidence and Study B used 99%:

  • Study B more reliable (higher confidence)
  • Tradeoff between precision and confidence

Answer: PRECISION: Study A is more precise. Its interval (66, 70) is narrower with margin of error of 2 inches compared to Study B's margin of error of 3 inches.

ACCURACY: Cannot determine which is more accurate without knowing the true population mean. Both intervals could contain μ, one could, or neither could. Precision (narrowness) doesn't guarantee accuracy (containing the truth).

5Problem 5hard

Question:

A researcher constructs a 95% CI for difference in means: (2, 8). Can we conclude there is a significant difference between the groups? What if the CI were (-1, 7)?

💡 Show Solution

Step 1: Understand CI for difference CI = (2, 8) for μ₁ - μ₂

This means:

  • We're 95% confident true difference is between 2 and 8
  • All values in interval are plausible

Step 2: Test for significant difference "Significant difference" means:

  • μ₁ ≠ μ₂
  • Equivalently: μ₁ - μ₂ ≠ 0
  • Zero is NOT a plausible difference

Key question: Is 0 in the confidence interval?

Step 3: Analyze CI = (2, 8) Is 0 in the interval? 2 < 0? No 0 < 8? Yes So 0 is NOT in (2, 8)

Conclusion: YES, significant difference!

Why?

  • All plausible values are positive
  • Difference is at least 2
  • Could be as much as 8
  • Cannot be 0 (no difference)

Step 4: Interpret (2, 8) μ₁ - μ₂ is between 2 and 8 This means μ₁ > μ₂

We're confident:

  • Group 1 mean is higher
  • Difference is real, not due to chance
  • Statistically significant at α = 0.05

Step 5: Analyze CI = (-1, 7) Is 0 in this interval? -1 < 0 < 7? YES

Conclusion: NO significant difference

Why?

  • Zero is plausible
  • Difference could be negative (-1)
  • Difference could be zero (0)
  • Difference could be positive (7)
  • Cannot rule out "no difference"

Step 6: Interpret (-1, 7) This means:

  • μ₁ might be slightly less than μ₂ (diff = -1)
  • μ₁ might equal μ₂ (diff = 0)
  • μ₁ might be greater than μ₂ (diff = 7)

We're NOT confident in direction!

  • Difference not statistically significant
  • Could be due to random chance

Step 7: Connection to hypothesis testing Testing: H₀: μ₁ = μ₂ (difference = 0)

CI = (2, 8): 0 not in interval

  • Reject H₀
  • Significant at α = 0.05
  • p-value < 0.05

CI = (-1, 7): 0 in interval

  • Fail to reject H₀
  • Not significant at α = 0.05
  • p-value > 0.05

Step 8: General rule For 95% confidence interval:

If 0 NOT in interval: ✓ Significant difference (α = 0.05) ✓ Reject H₀: μ₁ = μ₂ ✓ p < 0.05

If 0 IS in interval: ✗ Not significant (α = 0.05) ✗ Fail to reject H₀ ✗ p > 0.05

Step 9: Other examples CI = (3, 5): 0 not in interval → significant CI = (-2, -0.5): 0 not in interval → significant (group 1 lower) CI = (-3, 3): 0 in interval → not significant CI = (0.1, 4): 0 not in interval → significant (barely!)

Answer: CI = (2, 8): YES, significant difference at α = 0.05 level. Zero is not in the interval, so we can confidently say the groups differ. Group 1 has a higher mean, with difference between 2 and 8.

CI = (-1, 7): NO, not significant. Zero is in the interval, meaning "no difference" is plausible. We cannot conclude the groups differ - the observed difference could be due to random chance.

General rule: If a 95% CI for a difference includes 0, the difference is not statistically significant at α = 0.05.