How to measure a primary care pharmacy training pilot

A primary care pharmacy training pilot should answer more than "how many people completed?" Completion matters, but it cannot show whether learners improved, whether variation narrowed, whether workplace behaviour changed or whether the service benefited. A useful evaluation separates those questions and labels who owns each data source.

1. Reach and adoption

Report the full denominator. If 80 of 100 learners activated and 60 completed, the completion rate is 60% of the commissioned cohort—not 75% of activated learners without context. Show variation by agreed role, place or pathway, and record reasons where access, time or implementation affected uptake.

2. Assessed learning

Use a baseline where practical, then report change in assessment and fictional simulated performance. Include safety-critical errors or persistent themes rather than only an average percentage. A score should be interpreted against the activity and rubric; a raw experience-points total is an engagement mechanic, not a workforce outcome.

3. Workforce application

Employers and supervisors own real-world evidence. They might record observed competence against a local framework, common supervision themes, time to an agreed scope or whether a new starter completed a consistent induction. Keep private learner reflections and patient information out of system-level reports.

4. Service outcomes

Choose a small number of locally meaningful measures that match the intervention: for example completion of an agreed high-risk-medicine audit process, quality of documentation sampled under local governance, or turnaround in a selected workflow. The ICB must define the method, data protection route and attribution. A training platform cannot claim that any observed change was caused by learning alone unless the evaluation design supports that conclusion.

A commissioner report structure

• Executive summary and decision requested.
• Cohort, inclusion criteria, pathway and implementation activity.
• Reach funnel: invited, activated, active, completed.
• Baseline-to-end learning measures and safety themes.
• Variation by agreed cohort with small-number suppression where required.
• Accessibility, support and non-completion themes.
• Locally supplied workforce or service measures.
• Limitations, costs and recommended next action.

Set thresholds before launch

Agree what would trigger stop, adjust, extend or scale. Thresholds will vary, but they should cover minimum adoption, acceptable evidence quality, implementation feasibility, cost and any local outcome target. Setting them afterwards encourages selective interpretation.

Benchmarking: useful when designed carefully

Opt-in benchmarking can help practices understand improvement and variation, but it needs fair denominators, comparable cohorts, transparent rules and privacy safeguards. Prefer completion, improvement and agreed safety measures over a public league table based on raw points. Small groups may need suppression or aggregation to avoid identifying individuals.

What Medicine in Practice can report

Authorised organisation views can include learner activity, module progress, assessments, simulated performance, certificates, last activity and dated exports. Local managers see their own authorised scope. Training cases are fictional; patient data and private learner notes do not belong in manager reporting. Explore the commissioner reporting model or use the business-case structure.