Evaluate errors with the right denominators
See this course’s project requirements
Evaluate the fictional box classifier and document its evidence and use boundaries in a model card.
Keep your working files in your own notebook or software. Save a private description, checklist, and work link in My Proof.
What to include
- Define the prediction task, baseline and human decision boundary.
- Document feature/label meaning and independent training/evaluation records.
- Compare two thresholds and freeze the chosen model.
- Calculate confusion counts, accuracy, precision and recall against a baseline.
- Report the separate condition audit and a targeted next evaluation step.
Review and revise
Use these criteria to check your work and choose what to improve.
- Concepts: features, labels, training and decisions remain distinct.
- Evaluation: arithmetic and denominators match the original records.
- Judgment: small samples and uneven performance are not overstated.
- Accountability: use boundaries and further evidence needs are concrete.
Want more background?
The idea in everyday life
An overall score can hide the type of mistake a model makes. A false positive flags a box whose actual label is no; a false negative misses one whose label is yes. You will apply the frozen threshold to eight new fictional boxes and count these outcomes explicitly. The small fixture demonstrates evaluation arithmetic, not statistically reliable deployment evidence.
A metric is a number used to measure performance. Accuracy, precision, and recall answer different questions about the same predictions.
One step at a time.
Use the sample or information you have permission to use. Keep your work as you go. You can pause between steps.
- 1
Use evaluation rows T1=1/no, T2=2/yes, T3=3/yes, T4=4/yes, T5=5/no, T6=6/yes, T7=2/no and T8=5/no. Apply the frozen rule: predict yes for scuffs >= 3.
- 2
Classify each prediction using the actual label. Count true positives where both are yes, true negatives where both are no, false positives where prediction alone is yes, and false negatives where actual alone is yes.
- 3
Your counts should be TP=3, TN=2, FP=2 and FN=1. Verify that the four categories sum to eight before calculating any percentage.
- 4
Compute accuracy=(TP+TN)/all=5/8; precision=TP/(TP+FP)=3/5; recall=TP/(TP+FN)=3/4. Explain each denominator (the number you divide by): all boxes, boxes predicted yes, and boxes actually labeled yes, respectively.
- 5
Compare with always predicting no, which matches four of eight actual labels. Record the model’s improvement on this tiny test but also its remaining misses and unnecessary flags; no single percentage establishes fitness for every use.
See how it works.
Fictional evaluation: threshold three predicts no for T1,T2,T7 and yes for T3,T4,T5,T6,T8. T3,T4,T6 are true positives; T1,T7 are true negatives; T5,T8 are false positives; T2 is a false negative. Accuracy is 5/8=62.5%. Precision asks how many flagged boxes truly have yes labels: 3/5=60%. Recall asks how many actual yes boxes were flagged: 3/4=75%. The always-no baseline gets four no-labeled boxes correct, or 50%. These counts expose errors that a single accuracy figure would hide, while remaining too small for a strong generalization claim.
Did you cover these?
- Confusion counts sum to 8.
- Metrics are 62.5%, 60% and 75%.
- Always-no baseline accuracy is 50%.
What to finish
An eight-row prediction table, confusion counts and three checked metrics.
Watch for this
Using all records as the denominator for every metric or treating accuracy as sufficient.
How can I tell I understand it?
Explain precision and recall using their different denominators.
Quick knowledge check
Answer all 3 questions. Get at least 2 right and the lesson completes automatically.
You’ll get an explanation for each answer. Signed in? Your results save to your account.
Pass the check. We’ll save the progress.
Answer all 3 questions above and get at least 2 right. You can review the explanations and try again.
Go to the knowledge checkTools, templates, and lesson sources
Tools and templates
Tools open in a new tab so you can keep your place here.
Sources and further reading
Explore the references behind this lesson.
Source & access notes · checked 2026-09-10
Read metric definitions, denominators and threshold trade-offs. Course arithmetic is independently checked on its original eight-row holdout fixture.
Access: Free public reading; no account or certificate is required for this Life Starter course.
Reuse: Linked for reference. Examples, activities and assessments here are original; free access alone does not grant permission to reuse source text or branding.
Report a problem with this lesson
Tell us what went wrong. Please leave out personal information and private project contents.