TOPIC 3 / 3

Estimation, Tests & Regression

Useful preparation: Expectation, Variance & the CLT

For a long equation, swipe sideways inside its formula box to see every term.

Meaning & notation

Inference connects observed data to a population or process through an explicit sampling model. An estimate identifies a plausible parameter value; an interval describes uncertainty from repeated sampling; a test measures incompatibility with a specified null model. A fitted regression line summarizes an association, while causal interpretation requires evidence about how the data were generated.

Confidence interval: A frequentist $\displaystyle 95\%$ interval procedure covers the fixed population parameter in $\displaystyle 95\%$ of repeated samples under its assumptions. The realized interval either contains that parameter or does not.
P-value: A p-value is the probability, assuming the null model and its assumptions, of a test statistic at least as incompatible with that model as the observed statistic, using the prespecified test direction.
Least-squares regression: A fitted line predicts the response from an explanatory variable by minimizing the sum of squared vertical residuals. A residual is observed response minus fitted response.

Conditions before calculation

The one-sample t interval is exact for independent observations from a normal population with unknown positive variance. Large-sample approximations require suitable conditions and can be poor with heavy tails or severe skewness.
Random sampling supports population generalization; random treatment assignment supports causal comparisons. A larger sample does not automatically remove selection bias or confounding.
Classical regression t inference assumes a correct linear conditional mean, independent errors, constant error variance, and normal errors for exact small-sample results. Fitting a line alone does not establish these assumptions.

Why the method works

1. Name the parameter and design

State the target population, parameter, and sampling process before selecting a formula. Distinguish the standard deviation of observations from the standard error of an estimator.

2. Match the reference distribution

When estimating a normal-population mean using sample standard deviation s, uncertainty in s leads to a t reference with $\displaystyle n - 1$ degrees of freedom.

\bar x\pm t_{0.975,n-1}\frac{s}{\sqrt n},\qquad T=\frac{\bar x-\mu_0}{s/\sqrt n}

3. Interpret limited evidence

A small p-value challenges the specified null model; it is not the probability that the null is true. Failure to reject does not prove equality. Report effect size and uncertainty alongside the decision.

Worked examples

EXAMPLE 1

An independent normal sample has $\displaystyle n = 16$ , mean 12, and sample standard deviation 4. Construct $\displaystyle a\, 95\%$ mean interval and test mean 10 against a two-sided alternative.

The estimated standard error is 1. Use t with 15 degrees of freedom and critical value approximately 2.131.
$\widehat{\operatorname{SE}}=\frac4{\sqrt{16}}=1$
Form the interval around the sample mean.
$12\pm2.131(1)=[9.869,14.131]$
The statistic for mean 10 is 2, below the two-sided cutoff in magnitude; 10 also lies inside the matching interval.
$T=\frac{12-10}{1}=2<2.131$

Conclusion: Do not reject mean 10 at the $\displaystyle 5\%$ level. This does not establish that the mean equals 10 or assign it $\displaystyle a\, 95\%$ probability of lying in this realized interval.

EXAMPLE 2

Fit a least-squares line to points (0,1), (1,2), and (2,2).

Compute the coordinate means, then centered sums.
$\bar x=1,\quad\bar y=\frac53,\quad S_{xx}=2,\quad S_{xy}=1$
The slope divides the cross-product sum by the predictor squared-deviation sum.
$\widehat\beta_1=\frac12,\qquad\widehat\beta_0=\frac53-\frac12=\frac76$
Check the residuals; they sum to zero because the model includes an intercept.
$e=\left(-\frac16,\frac13,-\frac16\right),\quad\sum e_i=0$

Conclusion: The fitted response is $\displaystyle \frac{7}{6} + \frac{x}{2}$ . These three observations support a descriptive fit, not a causal conclusion or verified error assumptions.

Pitfalls & counterexamples

Tempting claim: A p-value of 0.03 means there is $\displaystyle a\, 3\%$ probability the null hypothesis is true.

Correction: The probability is calculated conditional on the null model; it is not a posterior probability for that model.

Test the claim: Different prior beliefs about a rare effect can yield different posterior probabilities even when a test gives the same p-value.

Check your understanding

Does $\displaystyle a\, 95\%$ confidence interval contain $\displaystyle 95\%$ of individual observations?

Write a reason before opening the explanation. This reflection is not graded or saved.

Compare your reasoning

No.

A mean confidence interval targets the population mean. Prediction and tolerance intervals answer different questions about observations.

Connect this idea

Inference builds on sampling distributions and standard errors. Regression also connects to orthogonal projection; diagnostics and study design determine whether the fitted mathematics answers the scientific question.

Practise this topic →Review concept flashcards

Reference reading

Use these texts to study the underlying definitions and theorems in greater depth. The explanations and examples above are written for Numerive.