The Life Starter
Sign up
Grade 7 · lesson 4 of 4

Keep Training Results Separate from Test Evidence

Use an untouched evaluation set for the stated generalization check and disclose its scope.

About 18 minutes, plus any extended practice

Download the grade pack

Supplied materials

  • Paper and pencil; answers may also be spoken or typed.
  • All cases, labels, numbers and rules are supplied on this page. No real account, private information or purchase is needed.

Learn the idea

Good performance on training examples may reflect fitting those examples rather than useful generalization. A held-out test set can provide different evidence, but only if it is not repeatedly used to tune choices. Report both set roles and their limitations. Our cases concern evidence quality, not a certified model.

Use an untouched evaluation set for the stated generalization check and disclose its scope. Reusing test answers to tune a model makes that set part of model development.

Use this supplied case: Score on examples used to fit the model. Which evidence role is described? The task is to connect the given information to the computing idea. Write a short evidence trail so another learner can reproduce your choice. The simplified case describes the stated model, not every real system.

  1. Represent the supplied worked case with a labeled drawing, table or short trace.
  2. Use an untouched evaluation set for the stated generalization check and disclose its scope.
  3. Explain why the worked answer fits and why one alternative does not.
  4. Try the new check and practice cases; revisit the stated rule if your evidence does not match.
Worked example

See the method in action

Score on examples used to fit the model. Which evidence role is described? The supported answer is Training performance. Use an untouched evaluation set for the stated generalization check and disclose its scope. Reusing test answers to tune a model makes that set part of model development. The supplied facts, rather than an assumed real-world detail, determine this paper case. Compare the recorded action or value with the stated rule before choosing an answer.

Your turn

Check your reasoning

Your answer is temporary and is not saved to an account. Download your current answer before leaving or reloading. Written work on paper or in another app stays in your own file.

Helpful tips
  • Use an untouched evaluation set for the stated generalization check and disclose its scope.
  • Reusing test answers to tune a model makes that set part of model development.
Score on held-out cases not consulted during tuning. Which evidence role is described?

Feedback appears here after you check. This is practice, not a graded school assessment.

See the answer and explanation

Untouched test evidence

The supplied case states: Score on held-out cases not consulted during tuning. Which evidence role is described? Under the stated distinction, this identifies Untouched test evidence. Good performance on training examples may reflect fitting those examples rather than useful generalization. A held-out test set can provide different evidence, but only if it is not repeatedly used to tune choices. Report both set roles and their limitations. Our cases concern evidence quality, not a certified model. Use an untouched evaluation set for the stated generalization check and disclose its scope.

Take it further

A withheld set is examined repeatedly to choose a threshold. Expected: it is used in development; a genuinely untouched evaluation set is needed for that separate role.

Nothing here is saved to an account or sent to a teacher. The answer download includes your current choice and any checked feedback. It does not include choices from the separate More practice section. Download that section separately.

More practice: try new cases

Choose an answer for each problem. Explain your method on paper, then use the feedback to decide what to revisit.

More practice: try new cases
1. Tune repeatedly after inspecting the test answers. Which evidence role is described?
Need a hint?
  • Use an untouched evaluation set for the stated generalization check and disclose its scope.
  • Reusing test answers to tune a model makes that set part of model development.
2. Model outputs guesses with no known labels. Which evidence role is described?
Need a hint?
  • Use an untouched evaluation set for the stated generalization check and disclose its scope.
  • Reusing test answers to tune a model makes that set part of model development.
3. Report accuracy on fitting examples only. Which evidence role is described?
Need a hint?
  • Use an untouched evaluation set for the stated generalization check and disclose its scope.
  • Reusing test answers to tune a model makes that set part of model development.

Your choices are temporary. Feedback appears after you check all the answers.

Nothing here is saved to an account or sent to a teacher. Keep your own notes or download them before leaving. The score describes these questions; it does not certify subject mastery.

Check the subject sources

These LifeStarter passages, scenarios and exercises are authored for this Academy with AI assistance and review. External resources keep their publishers’ terms. A grade label suggests a starting level; it does not establish alignment with your school’s complete requirements.