Build a small dataset without leaking the answer
See this course’s project requirements
Evaluate the fictional box classifier and document its evidence and use boundaries in a model card.
Keep your working files in your own notebook or software. Save a private description, checklist, and work link in My Proof.
What to include
- Define the prediction task, baseline and human decision boundary.
- Document feature/label meaning and independent training/evaluation records.
- Compare two thresholds and freeze the chosen model.
- Calculate confusion counts, accuracy, precision and recall against a baseline.
- Report the separate condition audit and a targeted next evaluation step.
Review and revise
Use these criteria to check your work and choose what to improve.
- Concepts: features, labels, training and decisions remain distinct.
- Evaluation: arithmetic and denominators match the original records.
- Judgment: small samples and uneven performance are not overstated.
- Accountability: use boundaries and further evidence needs are concrete.
Want more background?
The idea in everyday life
A dataset is a set of choices about what the model gets to see. An answer accidentally included as an input can create impressive-looking scores that fail on new cases. You will prepare a fictional training table and reserve a separate evaluation set. Different boxes may share the same scuff count, but the same box record must not be copied across sets.
Features are information available at prediction time; labels are the outcomes used for learning or checking.
One step at a time.
Use the sample or information you have permission to use. Keep your work as you go. You can pause between steps.
- 1
Define columns box_id, scuffs and damaged. box_id identifies an invented physical box, scuffs is a nonnegative whole-number feature, and damaged is a yes/no inspection label.
- 2
Create training rows R1,0,no; R2,2,no; R3,3,yes; R4,5,yes. Explain that four examples are enough to demonstrate arithmetic, not enough to establish reliable real-world performance.
- 3
Reject a proposed input field inspector_final_verdict, because it already contains the answer and would not be available when predicting before inspection. Treat such answer leakage as a design defect.
- 4
Reserve eight different box IDs T1 through T8 for later evaluation. Keep their labels out of threshold selection. In larger work, a validation set supports tuning while a final test set remains separate.
- 5
Check that repeated IDs do not cross sets and that feature values have documented units and meanings. Save the data dictionary, four training rows and split plan before comparing candidate thresholds.
See how it works.
Fictional training table: R1 has zero scuffs and label no; R2 has two and no; R3 has three and yes; R4 has five and yes. The feature is scuffs, while damaged is the target. A proposed feature named final_inspection_result would reveal the target rather than help predict it in advance. T1–T8 are different boxes reserved for evaluation. Sharing a scuff count with R2 is allowed; copying R2 itself into the test set is not an independent example. The distinction is between similar legitimate cases and duplicated source records that make evaluation misleading.
Did you cover these?
- Features exist before the prediction decision.
- Training IDs and evaluation IDs are disjoint.
- Small-sample limits are explicit.
What to finish
A feature/label dictionary, four-row training table and independent-record split plan.
Watch for this
Using the answer, or information available only after the outcome, as a model input.
How can I tell I understand it?
Identify leakage and distinguish duplicate records from different records with similar features.
Quick knowledge check
Answer all 3 questions. Get at least 2 right and the lesson completes automatically.
You’ll get an explanation for each answer. Signed in? Your results save to your account.
Pass the check. We’ll save the progress.
Answer all 3 questions above and get at least 2 right. You can review the explanations and try again.
Go to the knowledge checkTools, templates, and lesson sources
Tools and templates
Tools open in a new tab so you can keep your place here.
Sources and further reading
Explore the references behind this lesson.
Source & access notes · checked 2026-09-10
Read train/validation/test separation, representative evaluation and duplicate-example risks. Original box fixtures are used rather than source exercises.
Access: Free public reading; no account or certificate is required for this Life Starter course.
Reuse: Linked for reference. Examples, activities and assessments here are original; free access alone does not grant permission to reuse source text or branding.
Report a problem with this lesson
Tell us what went wrong. Please leave out personal information and private project contents.