The shape of the lesson
By the end of the lesson, learners can
- False alarms and missed changes: Every decision could be wrong.
- Probabilities with discrete distributions: For binomial and Poisson tests, find the critical region first. \(\text{P(Type I)}\) is the probability of the critical region under \(H_0\), which is usually LESS than the nominal significance level. \(\text{P(Type II)}\) uses the acceptance region and the true value you are given.
- Probabilities with the normal distribution: For a test of a mean using a continuous distribution, \(\text{P(Type I)}\) equals the significance level exactly.
- Type I: reject a true \(H_0\). Probability = critical region under \(H_0\).
- Type II: keep a false \(H_0\). Probability = acceptance region under the stated true value.
- Discrete: \(\text{P(Type I)}\) is the actual significance level, at most the nominal one.
- Normal: \(\text{P(Type I)}\) equals the significance level; find the critical \(\bar{x}\) using \(\mu_0\).
- Context: describe each error in the words of the question.
- Reducing both: a larger sample lowers \(\text{P(Type II)}\) at a fixed significance level.
One picture every idea returns to
Type I error: rejecting \(H_0\) when it is true; \(\text{P(Type I)} = \text{P}(\text{result in critical region} \mid H_0 \text{ true})\), the actual significance level. Type II error: not rejecting \(H_0\) when it is false; \(\text{P(Type II)} = \text{P}(\text{result in acceptance region} \mid \text{a given true value})\).
Forty-five minutes, phase by phase
| Time | Phase | What happens in the room | Grouping |
|---|---|---|---|
| 0 to 5 min | Hook: predict | Open with the question in the title, Two ways to be wrong.. Paper 6. Put the figure on the board: it is the one picture every idea in the lesson returns to. | Whole class, then pairs |
| 5 to 13 min | Teach and check: False alarms and missed changes | Every decision could be wrong. Model the worked example Errors in context on the board, one line at a time. | Whole class, then pairs |
| 13 to 21 min | Teach and check: Probabilities with discrete distributions | For binomial and Poisson tests, find the critical region first. \(\text{P(Type I)}\) is the probability of the critical region under \(H_0\), which is usually LESS than the nominal significance level. \(\text{P(Type II)}\) uses the acceptance region and the true value you are given. Model the worked example Binomial on the board, one line at a time. Your turn: \(H_0: p = 0.5\), \(H_1: p > 0.5\), \(X \sim B(12, p)\), 5% level. Find the critical region, \(\text{P(Type I)}\), and \(\text{P(Type II)}\) when \(p = 0.8\). | Pairs |
| 21 to 30 min | Teach and check: Probabilities with the normal distribution | For a test of a mean using a continuous distribution, \(\text{P(Type I)}\) equals the significance level exactly. Model the worked example One-tailed on the board, one line at a time. | Round Table |
| 30 to 36 min | Check | The four Stage 1 questions from the student lesson, answered on mini whiteboards or by show of fingers. Re-teach the one most get wrong. | Individual, then whole class |
| 36 to 43 min | Plenary | The exam-style questions from Stage 2, one at a time: thirty seconds alone, then a random call. The feedback lines in the key give the examiner's wording. | Individual, random call |
| 43 to 45 min | Exit and homework | Set the worksheet below (6 questions, 26 marks). Learners self-assess against the objectives before they leave. | Individual |
Protect the parts that carry the learning
The sequence carries about fifty minutes of material, so the lab is driven by one student while the class predicts rather than explored freely. Times are a guide.
Protected: the check at 30 minutes. It is the only individual read on whether the key idea landed, and it is not cut.
Cut first if the build overruns: the last check in Probabilities with the normal distribution, never the check or the plenary.
In a 60 minute block: run worksheet questions 1 to 3 in class as a Rally Coach set and model both the hardest worked example and its check live.
Round Table
One sheet travels round the team. Each student adds one line of working or one idea and passes it on, so the solution is built four hands at a time and every contribution is visible.
The full structure, with its variations and the other twelve, is on the cooperative-learning page.
What to head off, and how
| Trap learners fall into | Teaching move that pre-empts it |
|---|---|
| Do not describe the errors with "accept" and "reject" mixed up. | Write the definition, then translate it into the context: Cambridge often gives one mark for each. |
| For a discrete test, \(\text{P(Type I)}\) is NOT the significance level (5%). | It is the probability of the actual critical region, here \(0.0355\) or \(0.0174\). |
| Find the critical value using \(H_0\)'s mean, but the Type II probability using the TRUE mean. | Standardising the critical value with 50 again just gives back 0.95. |
Type I and Type II errors: worksheet
6 questions, 26 marks. Written in the style of the papers; no past paper question is reproduced.
A drug trial tests \(H_0\): the drug has no effect, against \(H_1\): the drug lowers blood pressure. Describe a Type I error and a Type II error in context.
Calls arrive at a mean rate of 5 per hour. A test of \(H_0: \lambda = 5\) against \(H_1: \lambda < 5\) at the 10% level uses the number of calls in one hour.
(a) Find the critical region and \(\text{P(Type I)}\).
(b) Find \(\text{P(Type II)}\) if the mean rate is actually 2 per hour.
\(X \sim B(12, p)\). A test of \(H_0: p = 0.5\) against \(H_1: p > 0.5\) is carried out at the 5% level. Find the critical region, \(\text{P(Type I)}\), and \(\text{P(Type II)}\) when \(p = 0.8\).
\(X \sim N(\mu, 10^{2})\). A random sample of 25 is used to test \(H_0: \mu = 100\) against \(H_1: \mu \ne 100\) at the 5% level. Find \(\text{P(Type II)}\) when \(\mu = 105\).
A test at the 5% level of whether a coin is biased gives a result in the acceptance region.
(a) Which type of error could have been made? Explain.
(b) Explain why choosing a 1% significance level instead would make a Type I error less likely but a Type II error more likely.
The lifetimes of a type of battery are \(N(\mu, 3^{2})\) hours. The manufacturer claims \(\mu = 40\). A consumer group will test a random sample of 16 batteries at the 5% level to see whether the mean is less than 40 hours.
(a) Find the critical region for the sample mean.
(b) State \(\text{P(Type I)}\).
(c) Find \(\text{P(Type II)}\) if in fact \(\mu = 38\).
(d) State one way to reduce \(\text{P(Type II)}\) without changing \(\text{P(Type I)}\).
Answer key and marking notes
- Type I: concluding the drug lowers blood pressure when it has no effect.
- Type II: concluding there is no evidence of an effect when the drug really does lower blood pressure.
- (a) \(\text{P}(X \le 1) = 6e^{-5} = 0.0404 \le 0.10\); \(\text{P}(X \le 2) = 18.5e^{-5} = 0.1247 > 0.10\).
- Critical region \(X \le 1\); \(\text{P(Type I)} = 0.0404\).
- (b) Accept when \(X \ge 2\), with \(X \sim \text{Po}(2)\): \(1 - 3e^{-2} = 1 - 0.4060 = 0.594\) (3 s.f.).
- \(\text{P}(X \ge 10) = 0.0193 \le 0.05\); \(\text{P}(X \ge 9) = 0.0730 > 0.05\). Critical region \(X \ge 10\).
- \(\text{P(Type I)} = 0.0193\).
- Type II: \(\text{P}(X \le 9)\) for \(B(12, 0.8)\) \(= 0.442\) (3 s.f.).
- Standard error \(\dfrac{10}{5} = 2\). Acceptance region \(100 \pm 1.96 \times 2\): \(96.08 < \bar{x} < 103.92\).
- Under \(\mu = 105\): \(z_1 = -4.46\), \(z_2 = -0.54\).
- \(\text{P(Type II)} = \Phi(-0.54) - \Phi(-4.46) = 0.2946 - 0.0000 = 0.295\) (3 s.f.).
- (a) Type II only: \(H_0\) was not rejected, so the possible mistake is failing to reject a false \(H_0\) (the coin is biased but the test missed it).
- (b) A 1% level shrinks the critical region, so a true \(H_0\) is rejected less often (Type I less likely).
- The acceptance region grows, so a false \(H_0\) is more likely to be kept (Type II more likely).
- (a) Standard error \(\dfrac{3}{4} = 0.75\). Reject when \(\bar{x} < 40 - 1.645 \times 0.75 = 38.766\).
- Critical region \(\bar{x} < 38.8\) (3 s.f.).
- (b) \(\text{P(Type I)} = 0.05\).
- (c) Type II: \(\bar{x} \ge 38.766\) with \(\bar{X} \sim N(38, 0.75^{2})\): \(z = \dfrac{38.766 - 38}{0.75} = 1.022\), so \(1 - 0.8466 = 0.153\) (3 s.f.).
- (d) Increase the sample size: the standard error falls, the two distributions overlap less, and the 5% level is unchanged.
8 multiple-choice questions on this lesson
From the topic question bank. Assign them in the app for a marked score in your gradebook, or run the topic as a class game.
A school tests \(H_0: p = 0.6\) against \(H_1: p > 0.6\), where \(p\) is the proportion of students who pass an exam after a new revision programme (the old pass rate was 0.6). Which describes a Type I error?
- AConcluding that the programme raises the pass rate when in fact the pass rate is still 0.6
- BConcluding there is no evidence that the programme raises the pass rate when it really does
- CConcluding that the programme raises the pass rate when it really does
- DConcluding that the pass rate is still 0.6 when in fact it is 0.6
A hypothesis test at the 5% significance level gives a result that is NOT in the critical region. Which statement is correct?
- AOnly a Type II error could have been made
- BOnly a Type I error could have been made
- CEither a Type I or a Type II error could have been made
- DNo error could have been made
\(X \sim B(25, p)\). A test of \(H_0: p = 0.2\) against \(H_1: p > 0.2\) at the 5% significance level has critical region \(X \ge 9\). Find the probability of a Type I error.
- A\(0.0468\)
- B\(0.05\)
- C\(0.109\)
- D\(0.0294\)
\(X \sim N(\mu, 4^{2})\). A random sample of 20 values is used to test \(H_0: \mu = 30\) against \(H_1: \mu \ne 30\) at the 1% significance level. The critical region is \(\bar{x} < 27.70\) or \(\bar{x} > 32.30\). What is the probability of a Type I error?
- A\(0.01\)
- B\(0.005\)
- C\(0.99\)
- DIt cannot be found without the true value of \(\mu\)
Faults on a network occur at random at a mean rate of 3.5 per week. After an upgrade, an engineer tests whether the rate has increased by counting the faults in a 2-week period, at the 5% significance level. Find the probability of a Type II error if the rate has actually risen to 5 per week.
- A\(0.792\)
- B\(0.697\)
- C\(0.208\)
- D\(0.0270\)
\(X \sim N(\mu, 8^{2})\). A random sample of 16 values is used to test \(H_0: \mu = 60\) against \(H_1: \mu < 60\) at the 2.5% significance level. Find the probability of a Type II error when in fact \(\mu = 55\).
- A\(0.295\)
- B\(0.705\)
- C\(0.196\)
- D\(0.909\)
\(X \sim B(20, p)\). A test of \(H_0: p = 0.5\) against \(H_1: p \ne 0.5\) is carried out at the 5% significance level. Find the probability of a Type II error when in fact \(p = 0.7\).
- A\(0.584\)
- B\(0.416\)
- C\(0.762\)
- D\(0.0414\)
A test of \(H_0: \mu = \mu_0\) for a normal population with known variance is carried out at the 5% significance level. The same test is then carried out with a larger random sample, still at the 5% level. For a fixed true value of \(\mu\) different from \(\mu_0\), what is the effect?
- A\(\text{P(Type I)}\) is unchanged and \(\text{P(Type II)}\) decreases
- B\(\text{P(Type I)}\) decreases and \(\text{P(Type II)}\) decreases
- C\(\text{P(Type I)}\) is unchanged and \(\text{P(Type II)}\) increases
- D\(\text{P(Type I)}\) increases and \(\text{P(Type II)}\) decreases
Quiz key
Type I error: rejecting \(H_0\) when \(H_0\) is true. Here that means claiming the programme improves the pass rate when \(p\) is really still 0.6. The tempting second option is a Type II error (keeping a false \(H_0\), missing a real improvement). The last two options describe correct decisions, not errors.
\(H_0\) is not rejected. If \(H_0\) is true this is correct; if \(H_0\) is false it is a Type II error. A Type I error needs \(H_0\) to be rejected, so it is impossible here. The tempting "either" ignores the decision actually taken, and "no error" ignores that \(H_0\) might be false.
\(\text{P(Type I)} = \text{P}(X \ge 9 \mid p = 0.2) = 1 - \text{P}(X \le 8) = 0.0468\) (3 s.f.). The tempting \(0.05\) is the nominal significance level; for a discrete distribution the actual probability of the critical region is usually smaller. \(0.109\) is \(\text{P}(X \ge 8)\), and \(0.0294\) is only \(\text{P}(X = 9)\), not the whole tail.
Under \(H_0\), \(\bar{X} \sim N\left(30, \dfrac{16}{20}\right)\), and the two tails beyond \(30 \pm 2.576 \times \dfrac{4}{\sqrt{20}}\) have total probability exactly \(0.01\). So \(\text{P(Type I)} = 0.01\). The tempting \(0.005\) is only one tail of a two-tailed test. \(0.99\) is the probability of correctly not rejecting a true \(H_0\). The true \(\mu\) is needed for a Type II error, not a Type I error, which assumes \(H_0\) is true.
Under \(H_0\), \(X \sim \text{Po}(7)\). \(\text{P}(X \ge 12) = 0.0533 > 0.05\) and \(\text{P}(X \ge 13) = 0.0270 \le 0.05\), so the critical region is \(X \ge 13\). Type II: \(X \le 12\) when \(X \sim \text{Po}(10)\): \(\text{P}(X \le 12) = 0.792\) (3 s.f.). The tempting \(0.697\) is \(\text{P}(X \le 11)\), from wrongly taking \(X \ge 12\) as the critical region. \(0.208\) is the probability of correctly rejecting \(H_0\), and \(0.0270\) is \(\text{P(Type I)}\).
Standard error \(\dfrac{8}{4} = 2\). One-tailed 2.5% uses \(z = 1.96\): reject when \(\bar{x} < 60 - 1.96 \times 2 = 56.08\). Type II: \(\bar{x} \ge 56.08\) when \(\bar{X} \sim N(55, 2^{2})\): \(z = \dfrac{56.08 - 55}{2} = 0.54\), so \(1 - \Phi(0.54) = 1 - 0.7054 = 0.295\) (3 s.f.). The tempting \(0.705\) is the probability of correctly rejecting. \(0.196\) uses \(z = 1.645\) (the 5% value), and \(0.909\) uses \(\sigma = 8\) instead of the standard error in both steps.
With \(p = 0.5\): \(\text{P}(X \le 5) = 0.0207\) but \(\text{P}(X \le 6) = 0.0577\), so the critical region is \(X \le 5\) or \(X \ge 15\). Type II: \(\text{P}(6 \le X \le 14)\) for \(B(20, 0.7)\) \(= 0.584\) (3 s.f.). The tempting \(0.416\) is the probability of rejecting \(H_0\) (correctly) when \(p = 0.7\). \(0.762\) is \(\text{P}(5 \le X \le 15)\), with the boundary values wrongly included in the acceptance region, and \(0.0414\) is \(\text{P(Type I)}\).
For a continuous test, \(\text{P(Type I)}\) equals the significance level, so it stays at 0.05. A larger \(n\) shrinks \(\dfrac{\sigma}{\sqrt{n}}\): the critical value moves closer to \(\mu_0\) and the distribution of \(\bar{X}\) under the true \(\mu\) is narrower, so less of it lies in the acceptance region and \(\text{P(Type II)}\) falls. The tempting "both decrease" forgets that the 5% level fixes \(\text{P(Type I)}\). Lowering the significance level, not raising \(n\), is what trades a smaller \(\text{P(Type I)}\) for a larger \(\text{P(Type II)}\).
Slides for this lesson
A projectable deck for every pack is on the way. Until then, present the student lesson full screen: its sections, figure and lab tabs follow the same order as the plan above.