Curriculum
Every masterclass starts with Unit 1 and runs in order. The Foundation masterclass ends after Unit 8, the Advanced masterclass after Unit 12, and Advanced + Statistical Learning takes all 16.
The Core of Statistics
Data, Variables, and the Statistician's Toolkit
Data, variables, sums, percentages, logarithms, and the first graphs that every later unit uses
About this unitThe toolkit of the program. What data are and the kinds of variables, population and sample, parameter and statistic, the summation sign and the mean, proportions, percentages, and rates, relative against absolute change, powers of ten and the logarithmic axis, the rules of an honest graph, and how data are collected, with a short history of counting people from the Bills of Mortality to Hollerith's machine.
- Kinds of variables
- Population and sample
- The summation sign
- Percentages and rates
- Logarithmic axes
- Honest graphs
Describing Distributions
Histograms, box plots, centre, spread, and standardized values
About this unitHow to turn a long column of numbers into a picture and a few honest summaries. The unit builds the histogram and shows what the bin width does to it, defines the mean, the median, the standard deviation, and the interquartile range and says when each is the right choice, packs five numbers into a box plot with Tukey's fences for outliers, standardizes measurements with z-scores so that a height and a weight can be compared, summarizes categorical data in frequency and two-way tables, and asks of every summary how badly one far-out value can fool it.
- Histograms
- Shape of a distribution
- Mean and median
- Standard deviation
- Box plots
- z-scores
Probability and Conditional Probability
The rules of chance, counting, conditional probability, and Bayes' theorem
About this unitProbability is the language in which every later unit states how sure it is. This unit teaches what a probability means and the three axioms it obeys, how to count outcomes with factorials and binomial coefficients, how information changes a probability, and how Bayes' theorem turns the accuracy of a medical test into the chance that a positive result is right. The birthday problem, the Monty Hall problem, the lady tasting tea, and the case of Sally Clark show the rules at work and the cost of breaking them.
- Axioms of probability
- Counting
- Conditional probability
- Bayes' theorem
- The birthday problem
- Monty Hall
Random Variables and the Normal Distribution
Expected value and variance, the binomial and Poisson distributions, and the normal curve
About this unitA random variable attaches a number to the outcome of a random experiment, and its expected value and variance say where that number sits on average and how far it strays. This unit builds the rules for sums and multiples, then the three distributions that carry most of applied statistics: the binomial for counts of successes, the Poisson for counts of rare events, checked against the Prussian horse-kick deaths, and the normal curve, read term by term, standardized, and tabulated, with the 68-95-99.7 rule, the normal approximation to the binomial, and a first warning that heavy-tailed quantities such as incomes and city sizes are not normal.
- Random variables
- Expected value and variance
- Binomial distribution
- Poisson distribution
- The normal curve
- The 68-95-99.7 rule
Sampling and the Central Limit Theorem
Random samples, sampling distributions, the standard error, and why means pile up in a bell curve
Free sample unitRead the full unit online or as PDFA poll of 1,000 people speaks for a country, and 2.4 million ballots once named the wrong president. This unit explains how a random sample is drawn, why bias does not shrink with size, and how far the mean of a sample can stray from the mean of its population. It draws thousands of samples from real data, derives the standard error, states the law of large numbers and the central limit theorem with their conditions, and ends with the Galton board and the margin of error of a poll.
- Random samples
- Bias and variability
- Sampling distributions
- The standard error
- Law of large numbers
- Central limit theorem
Estimation and Confidence Intervals
Point estimates, standard errors, Student's t, and confidence intervals for means and proportions
About this unitA sample statistic is never the truth about the population, so the honest report is a number with a range around it. This unit builds that range: what makes an estimate good, why the sample variance divides by n - 1, the confidence interval for a mean and what 95 per cent confidence promises, Student's t distribution for small samples, the interval and the margin of error of a poll, planning a sample size, paired data, and the comparison of two groups.
- Point estimates
- Confidence intervals
- Student's t distribution
- Proportions and polls
- Margin of error
- Comparing two groups
Hypothesis Tests and p-Values
Null hypotheses, test statistics, the p-value, errors of two kinds, power, and the chi-squared test
About this unitA difference in a sample can be a real effect or the ordinary wobble of chance, and a hypothesis test is the controlled way to tell the two apart. This unit builds the test on Fisher's lady tasting tea, defines the p-value exactly, carries out the z, t, paired, and two-sample tests, sets out the two kinds of error and the power of a test, plans the size of a trial, tests two proportions on Snow's water companies and tables of counts on Mendel's peas and Doll and Hill's smokers, and ends with what a p-value is not.
- Hypothesis tests
- The p-value
- t tests
- Type I and type II errors
- Power
- Chi-squared tests
Correlation, Regression, and Causation
Scatter plots, the correlation coefficient, the least-squares line, and what it takes to show a cause
About this unitThe closing unit of the Foundation masterclass measures the relation between two variables with the correlation coefficient, summarizes it with the least-squares line, and asks what such a relation can and cannot show. Galton's family heights give the coefficient, the line, and regression to the mean; Anscombe's quartet shows what one number hides; the kidney stone data reverse a comparison; and the smoking debate, Snow's water companies, and the draft lottery of 1969 show why a randomized experiment settles causation and what an accident of history can borrow from one.
- Correlation
- Least squares
- R squared
- Regression to the mean
- Simpson's paradox
- Experiments and causation
Models and Inference
Likelihood and Maximum Likelihood
The likelihood function, the derivative as a tool, maximum likelihood estimates, and their standard errors
About this unitThe first unit of the Advanced masterclass shows where the estimates of the Foundation units come from. The probability formula of a model is read backwards, as a function of the unknown parameter with the data held fixed, and the value that makes the observed data most probable is the maximum likelihood estimate. The unit teaches the derivative before it uses it, derives the estimates of a proportion, a Poisson rate, and a normal mean and variance, reads the standard error from the curvature of the log-likelihood, and turns the drop in log-likelihood into a test and an interval, with coin flips, horse kicks, geysers, earthquakes, and the words of Moby-Dick as the data.
- The likelihood function
- Log-likelihood
- Derivatives
- Maximum likelihood estimates
- Observed information
- Likelihood ratio
Multiple Regression and Linear Models
Several predictors at once, indicator variables, t and F tests, analysis of variance, and residual checks
About this unitAlmost every response worth modelling depends on several predictors at once. This unit extends least squares to the multiple regression model, teaches what a coefficient means when the other predictors are held fixed and why it changes when one is left out, puts categorical variables into the model as indicators, and tests coefficients and whole models with t and F statistics. It ends with the residual checks that decide whether a fit deserves belief and with prediction intervals for a new case.
- Multiple regression
- Indicator variables
- Gauss-Markov theorem
- t and F tests
- Analysis of variance
- Residual diagnostics
Logistic Regression and Generalized Linear Models
Odds and log odds, the logistic curve, fitting by maximum likelihood, Poisson regression, and one framework for all three
About this unitMany responses are a yes or a no, or a count, and a straight line cannot model them. This unit models the log odds of an event as a straight line, reads the coefficients as odds ratios, and fits the model by maximum likelihood with the Challenger O-ring data as the running example. It turns fitted probabilities into decisions with sensitivity, specificity, and the ROC curve, models counts with a Poisson regression and an offset for exposure, and ends with the generalized linear model that holds linear, logistic, and Poisson regression together, compared by the deviance.
- Odds and odds ratios
- Logistic regression
- The Challenger data
- ROC curves
- Poisson regression
- Generalized linear models
Bayesian Inference
Prior, likelihood, and posterior, the beta-binomial model, credible intervals, and sampling from a posterior
About this unitBayes' theorem turns a prior distribution and a likelihood into a posterior distribution for a parameter, and the unit works this through in full for a proportion, where a Beta prior and binomial data give a Beta posterior, and for a mean, where precisions add. It explains what a credible interval promises and what a confidence interval promises, how to choose a prior and show that it did not decide the result, why a batting average after 30 at-bats is shrunk towards the league, and how the Metropolis algorithm draws samples from a posterior that nobody can write down, with the diagnostics that say whether the run can be trusted.
- Bayes' theorem for parameters
- Beta-binomial model
- Credible intervals
- Choosing a prior
- Shrinkage
- Markov chain Monte Carlo
Statistical Learning
The Bootstrap and Cross-Validation
Training and test error, bias and variance, the bootstrap, and k-fold cross-validation
About this unitThe first unit of the Advanced + Statistical Learning masterclass turns from estimating parameters to predicting new data, and lets the computer stand in for formulas. Polynomials of rising degree fitted to the Old Faithful eruptions show why the training error flatters a model and the test error does not, and the bias-variance decomposition explains the shape of the test error curve. The bootstrap gives a standard error and an interval for a median and a correlation by resampling the data, with the result that a bootstrap sample holds about 63 per cent of the observations. Ten-fold cross-validation chooses the degree of a polynomial, the one-standard-error rule leans towards the simpler model, and the sleep data of Unit 7 are tested again by flipping the signs of the ten differences.
- Training and test error
- Bias and variance
- The bootstrap
- Percentile intervals
- Cross-validation
- Permutation tests
Ridge, Lasso, and Regularization
Shrinkage, ridge regression, the lasso, soft thresholding, and choosing the penalty by cross-validation
About this unitLeast squares comes apart when predictors are correlated or more numerous than the observations: the coefficients swing to absurd values, and with more predictors than patients there is no unique fit at all. This unit adds a price to the size of the coefficients. Ridge regression charges for squared coefficients and shrinks all of them, and the lasso charges for absolute values and sets some exactly to zero, so that it selects predictors while it fits; the diabetes data of 442 patients and the fuel consumption of 32 cars show both at work. The penalty is chosen by cross-validation with the one-standard-error rule, the penalties turn out to be Bayesian priors in disguise, and a worked example shows how many of 5,000 unrelated genes will look correlated with an outcome by chance alone.
- Collinearity
- Ridge regression
- The lasso
- Soft thresholding
- Choosing the penalty
- The elastic net
Trees, Random Forests, and Boosting
Decision trees, bagging, random forests, gradient boosting, and when to use them
About this unitA decision tree predicts by asking one yes-or-no question at a time, and the unit grows one by hand: the first split of a regression tree for the fuel economy of 32 cars and the Gini index before and after one split of the Palmer penguins. One tree is easy to read and unstable, so the unit averages hundreds of them on bootstrap samples, derives how far the correlation between trees limits the gain, and shows how the random forest lowers that limit and how the out-of-bag error comes free. Boosting then fits small trees one after another to the residuals, two rounds by hand on five cars and a derivation of why the residuals are the right target. A ten-fold cross-validation on the Cleveland heart disease data compares a tree, a forest, and a boosted model, and the closing section says what none of them can do.
- Decision trees
- Gini index and entropy
- Bagging and out-of-bag error
- Random forests
- Gradient boosting
- Interpretability
Multiple Testing and the False Discovery Rate
Family-wise error, Bonferroni and Holm, the false discovery rate, Benjamini-Hochberg, and selective inference
About this unitThe closing unit asks what the 5 per cent level of Unit 7 turns into when a study runs twenty, ten thousand, or eight million tests. The family-wise error rate and the Bonferroni and Holm corrections guard against a single false positive; the false discovery rate of Benjamini and Hochberg replaces that goal by the fraction of false leads in a list, and their procedure is worked through on the fifteen p-values of a heart attack trial, on a simulated screen of 10,000 genes, and on the Manhattan plot of a genome-wide study of 39,620 children. A dead salmon, the garden of forking paths, and the hundred replications of 2015 show where uncounted tests lead, and the knockoff filter shows how the lasso's selections can be tested honestly.
- Family-wise error rate
- Bonferroni and Holm
- False discovery rate
- Benjamini-Hochberg
- Genome-wide studies
- Selective inference