Study guide: Assess Data-Driven Decisions and Tradeoffs
A model has 3 false positives and 2 false negatives under 2/5 point weights. It detects 6 of 8 positives and receives a 0.6 score under the supplied action policy. Calculate and explain the bounded decision.
Ideas to explain in your own words
- Different model errors can have different consequences. A fictional decision policy may assign weights to false positives and false negatives. Multiplying counts by those weights gives a bounded cost model, not a universal value judgment or a true measurement of every harm.
- Recall for a positive category is true positives divided by all actual positive labeled cases. It differs from precision, which uses predicted positives. A group comparison should preserve denominators and sample sizes. The supplied cases calculate recall only and do not establish fairness for every possible criterion.
- A model output and the rule for acting on it are distinct. A decision policy can include thresholds, human review, abstention and appeal. A numerical score is not a guaranteed probability unless its meaning is established. Our fictional policy states the actions directly.
- A deployed model may encounter data unlike its evaluation set. Monitoring should define what change is measured, what triggers review and who is responsible. A trigger prompts investigation; it does not explain the cause automatically. The supplied policy uses fictional thresholds, not a universal deployment standard.
Useful vocabulary
- recall
- Detected positives divided by actual positives.
- decision policy
- Rules for actions based on available evidence.
- drift
- A relevant change in data or conditions.
Watch your reasoning
- Do not assume equal error counts mean equal modeled consequences.
- Do not use all predictions or predicted positives as the recall denominator.
- Model confidence is not permission to bypass a stated human review step.
- A drift alert indicates a review need, not proof of one specific cause.
A worked review example
Fictional policy: each false positive costs 2 points; each false negative 5 points. FP=2, FN=1. Total modeled error cost? The supported answer is 9 points. Applying the stated rule gives 9 points. Multiply each error count by its stated weight and add the two contributions. Do not assume equal error counts mean equal modeled consequences. Calculation: ((2*2)+(5*1)) = 9. Missing-input rate 12%. Policy: review if missing-input rate>10%; otherwise continue monitoring; absent measurements stay unknown. Choose status. The supported answer is Review trigger met. Compare the measured condition with the stated review rule and identify what further evidence is needed. A drift alert indicates a review need, not proof of one specific cause.
Make a practice plan
- Make a four-row summary, one row for each lesson, with its computing idea and one worked decision.
- Explain one tempting mistake using the actual supplied example.
- Complete the new unit review before consulting its explanations.
- Try the transfer task on paper and compare your evidence against the model response.