THE LIFE STARTER · PILOT EVALUATION PROTOCOL
Planning materials 2026-10-02
https://thelifestarter.com/organizations/partner-kit

Proposed method for a small feasibility pilot, not a completed study or effectiveness claim. Agree and version the protocol before recruitment. It does not automatically collect research data or export learner rows. A separate protected operational download describes record coverage in the open program; it is not an approved study dataset or an estimate of learning gains.

QUESTIONS AND SOURCES
Can learners understand and use the recommended next step?
Method: Ask the learner to explain the recommendation and choose an activity. Record whether they proceed independently, with help, or cannot continue, plus the reason.
Source: A consented observation and learner feedback; a page visit alone is insufficient.

Does feedback help learners improve a useful project?
Method: Review the first sample and revision using the same project criteria. Record criterion-level judgments, reviewer, rubric version and assistance. Have a second reviewer check a pre-agreed sample where capacity allows.
Source: Permitted work samples and review notes; distinguish self-review from staff review.

What changes between the starting and final checks?
Method: Match starting and final forms A/B using host-managed participant codes and the shared instrument version. Report paired scores, lesson-check attempts, group size and missing checks separately.
Source: Authorized check records; a small uncontrolled pilot describes change, not a proven treatment effect.

ANALYSIS AND SAFEGUARDS
1. Write the question, eligible group, collection dates, measure definitions and decision criteria before recruitment. Targets are proposals, not achieved results.

2. Keep invited, enrolled, started, attended, submitted, reviewed and completed counts separate. Name the denominator for every percentage; report missing records and withdrawal rather than removing them silently.

3. The program final check unlocks after all guided lessons and eight projects are completed/submitted. Available final-check results therefore represent completers who responded, not the original cohort. Do not enter a missing final score as zero.

4. For a proposed descriptive paired comparison, the host must first establish pre-instruction timing, measure compatibility and approved permissions. Form A/B labels alone do not establish equivalent difficulty. Include only learners with both compatible checks in the paired calculation; separately report all enrolled learners and missing baseline/final counts. Calculate each learner's final minus starting score before averaging; use percentage points only when both scores use the same 0–100 scale.

5. Offer a separate host-managed exit task to every enrolled learner, including those who stop or do not complete the program. Record offered, returned, missing and withdrawn counts. The app final remains completion-gated. Administer the starting task before instruction and version tasks, rubric, assistance and collection dates; the app does not establish this timing or research permission. Published practice cases with answers are not unseen exit forms.

6. Keep knowledge, confidence, project quality and external actions separate. A resource click or self-report does not verify employment, service receipt, enrollment or a credential.

7. Agree success criteria with the host. Review feasibility, learner feedback, support effort, cost and access barriers together; continue, revise or stop based on the combined evidence.

8. Use the host's approved system for analysis. Keep identifying keys separate from coded work; coded data is not automatically anonymous. Do not publish individual work or small-group results that could identify someone.

9. Any formal research, external data access or publication requires a separately agreed purpose, permissions, data protections and any applicable institutional or ethics review. Choosing not to join optional research must not remove ordinary learning access.

WORKED REPORTING EXAMPLE — FICTIONAL
10 enrolled; 8 submitted; 6 received a staff review. Report 8/10 (80%) submitted and 6/10 (60%) reviewed; among submissions, 6/8 (75%) were reviewed. These describe different denominators. They do not establish project quality or program completion.
Three learners have compatible paired scores: 40 to 60, 60 to 70, 80 to 80 on the same 0–100 scale. Changes: +20, +10, 0 percentage points; mean paired change: +10 percentage points, n=3. Separately show how many enrolled learners lack either check. This is observed change, not a causal effect.

END SUMMARY TEMPLATE
Host, cohort dates and protocol/version: ____________________
Eligible group and recruitment: ____________________
Invited / enrolled / started / baseline / final / paired counts: ____________________
Submitted / reviewed / completion counts and definitions: ____________________
Measure / numerator / denominator / missing / source / collection date: ____________________
Work-sample changes, reviewer agreement and assistance: ____________________
Access barriers, withdrawals, support effort and learner feedback: ____________________
Actual cost, cost basis and denominator: ____________________
Limits and alternative explanations: ____________________
Decision to continue / revise / stop, owner and next review date: ____________________

SHARING METHODS
This blank protocol and the blank measurement worksheet are publicly inspectable. No learner dataset is published or promised. Any future external research arrangement must be separately reviewed and authorized.
