Study guide: Statistics: Study Design and Careful Inference
Evidence quality depends on how participants and conditions are selected. Random sampling supports population generalization, random assignment supports treatment comparisons, and uncertainty summaries do not repair biased recruitment or uncontrolled causal differences.
Ideas to explain in your own words
- Random assignment distributes participants among treatment groups to reduce systematic preexisting differences. Random sampling selects participants from a population; these are different procedures with different inferential purposes. Volunteer recruitment may limit generalization even after treatment assignment is random.
- Repeated samples from one model can give different means. Larger independent samples commonly produce less variable means under finite-variance conditions, improving precision without guaranteeing an exact population value. Sample size cannot eliminate a biased selection method or nonresponse problem.
- A reported estimate with an additive margin produces endpoints by subtracting and adding the margin. Percentage points measure additive differences between percentages rather than proportional percent changes. This exercise interprets a supplied report and does not compute a confidence level or validate the survey.
- An observational score difference can reflect preexisting differences between groups as well as an intervention. Without random treatment assignment or a justified alternative design, the whole difference cannot be assigned cleanly to one cause. Random sampling alone does not randomize treatment.
Useful vocabulary
- random assignment
- Random allocation of participants among conditions.
- sampling variation
- Differences among results from repeated samples.
- confounder
- A factor associated with both the proposed cause and outcome.
- percentage point
- One unit of additive change on a percent scale.
Watch your reasoning
- Volunteer recruitment may limit generalization even after treatment assignment is random. A study can randomize one stage without randomizing the other, so state exactly which step supports a proposed conclusion.
- Sample size cannot eliminate a biased selection method or nonresponse problem. The supplied simulation compares variation under one model; it does not claim every large sample is representative or every estimate correct.
- This exercise interprets a supplied report and does not compute a confidence level or validate the survey. Preserve the reported units; multiplying by a percentage factor would answer a different uncertainty question.
A worked review example
Complete supplied guide examples: Separate Random Assignment from Random Sampling — Fictional study 2 randomly assigns volunteers to practice A or B, then uses the same test. What does random assignment primarily help address? Candidate answers: (A) It eliminates every possible study limitation (B) It makes volunteers representative of every school (C) It proves all tests measure the same thing (D) Preexisting differences between treatment groups. Random assignment tends to balance preexisting differences across treatment groups. Random sampling addresses population representativeness; the two randomization steps answer different questions. Interpret Variation across Repeated Samples — In fictional simulation 2, repeated random samples of size 100 have means less spread out than samples of size 10. Which interpretation is warranted? Candidate answers: (A) Larger samples can make the sample mean more stable under this model (B) Larger samples guarantee the population mean exactly (C) Sample size removes nonresponse bias (D) The population changed whenever a sample mean changed. Repeated-sample variability is sampling variation. With independent sampling from the same finite-variance model, larger samples generally reduce the spread of sample means; they do not guarantee exactness or repair a biased design. Use Percentage Points in a Reported Interval — A supplied survey report gives estimate 50% with margin ±4 percentage points. What interval is reported? Candidate answers: (A) 4% to 50% (B) 46% to 54% (C) 46% to 58% (D) 48.00% to 52.00%. Subtract and add four percentage points: 50−4=46, 50+4=54. A margin in percentage points is not a 4% multiplicative change. Keep an Observational Association Qualified — In fictional observational study 2, students choosing a practice app have higher scores. What conclusion fits? Candidate answers: (A) Every app user must score above every nonuser (B) Random sampling alone proves the app effect (C) The app caused the whole score difference (D) The groups differ in scores; motivation or other differences may explain part of it. App choice was not randomly assigned. Existing motivation, prior preparation or other differences may affect both app use and scores. The observed association is evidence to investigate, not a clean estimate of a treatment effect.
Make a practice plan
- Study the four methods in the listed order; attempt each independent check before opening its explanation.
- Complete each lesson’s three changed-input questions and record the operation or reasoning step behind any correction.
- Take the four-question unit review, then solve the transfer task with written justifications.
- After correcting errors, attempt the separate unit exam and revisit the linked lesson for any unresolved method.