1. Starting point and outputs
Who benefits
- Training and development directors and program owners
- Performance, evaluation, and HR analysts
- Department directors, funding bodies, and supervisors of workplace application
What you will produce
- An evaluation plan defining the decision, questions, design, and data sources
- A baseline and verifiable learning and application assessment tools
- Analysis distinguishing observed change from impact attributable to the intervention
- A report explaining results, limitations, alternatives, and the improvement decision
- A cost register and benefit estimate when the conditions for estimation are met
Inputs to prepare
- Program purpose, learning outcomes, and the organizational problem it is expected to address
- Participant lists, groups, selection conditions, and when application opportunities become available
- Prior performance data, indicator definitions, and concurrent operational changes
- Tests or practical tasks, rubrics, and supervisors able to observe performance
- An evaluation budget, data officer, retention rules, and suitable access permissions
2. Working methodology
Start with the decision, not the survey
Define the decision required: improving content, scaling up, addressing weak application, or comparing two options. CDC guidance emphasizes early planning and distinguishing learning from its transfer to work. The Union provides the following templates and steps as a proposed practical approach that can be adapted.
Distinguish observation from causal attribution
Higher performance after a program is an observed change; claiming that training caused the increase requires a design that tests what would have happened without it. Systems, incentives, seasonality, and case mix may change at the same time. A suitable comparison, phased implementation, or trend analysis can be used, depending on feasibility and ethics, with each design's assumptions stated. When evidence is weak, describe possible contribution rather than claiming proof of causation.
Proportionate measurement and protection of trust
Evaluation cost should be proportionate to the importance of the decision. Explain why data are collected, define access and retention, and collect only what is necessary. Separate names from analysis and present aggregated results where appropriate. The guide is a development tool and does not, on its own, determine employee sanctions or professional fitness.
This is an adaptable practical approach developed by the Union with reference to the sources below. Adjust steps, timeframes and measures to the organization and its data. It is not a certificate of compliance or accreditation by the referenced organizations.
3. Implementation steps
1 Build the intervention logic
Map a clear link between the learning activity, skill, workplace behavior, and organizational outcome. For each link, write an assumption: the employee learns a triage method, then receives cases and authority to use it, reducing errors. Identify external factors that could prevent application. Review this logic with the work manager; if the outcome is beyond the program's reach, narrow the question rather than making training responsible for change it cannot produce.
- Suggested owner
- Evaluation officer and program owner
- Required output
- An outcomes and assumptions map
- Progression check
- A plausible mechanism links learning to the selected outcome
2 Define questions and the evaluation design
Select a few questions linked to a specific decision: Was proficiency achieved, did it transfer to work, what were the barriers, and did the indicator improve? Define a method, evidence, timing, and owner for each question. Decide whether a before-and-after description or a stronger comparison is feasible. Do not use a comparison group with fundamentally different tasks and experience and then interpret the difference as impact. Document design limitations before results emerge.
- Suggested owner
- Evaluation officer and decision-maker
- Required output
- An evaluation matrix and analysis plan
- Progression check
- Each question has a clear use and an obtainable source
3 Collect the baseline and pilot the tools
Measure participants' performance before the program using a task or test aligned with the outcomes, and collect operational data from a suitable period. Pilot measurement tools with a small number to identify ambiguity and scoring difficulties. Fix the definitions of the unit, duration, and cases included in calculations. If the test later changes, document the effect on comparison. Do not replace objective pre-measurement with a later recall question without disclosing that it is a weaker self-reported estimate.
- Suggested owner
- Data analyst and subject matter expert
- Required output
- A baseline, piloted tools, and data dictionary
- Progression check
- Pre- and post-measurement definitions match
4 Measure learning during and after the program
Use problems, simulations, or work products, and gather feedback that helps learners correct their performance. Compare results for people with paired measurements, separating those absent or incomplete. Review findings by outcome to locate weaknesses, rather than using the overall score alone. Collect satisfaction and content relevance feedback to improve the experience while keeping them separate from evidence of skill acquisition.
- Suggested owner
- Trainer and evaluation officer
- Required output
- Learning results and outcome analysis
- Progression check
- Assessment measures the task rather than only recall of its terminology
5 Verify learning transfer
Agree with the supervisor on a work task and observation indicators, and allow enough time for actual practice. Collect a product sample or documented observation, then interview the learner about support and barriers. Record who lacked an application opportunity and why; do not confuse lack of opportunity with lack of ability. Use a consistent definition of acceptable application, and review samples of supervisors' assessments to reduce differences in judgment.
- Suggested owner
- Line supervisor and application assessor
- Required output
- An application and barrier register, and follow-up coverage
- Progression check
- Each performance judgment has reviewable evidence and a known application opportunity
6 Analyze results and alternative explanations
Calculate change alongside sample size, missing-data rates, and the distribution of results. Examine differences in workload, seasonality, resources, and systems. If using a before-and-after comparison for two groups, check the plausibility of prior trends, similarity between groups, and absence of intervention spillover. Present sensitivity results when a material assumption changes. If data are sparse or unrepresentative, state this and support interpretation with qualitative examples without excessive generalization.
- Suggested owner
- Evaluation analyst and data owner
- Required output
- Results analysis and limits of inference
- Progression check
- Report language does not exceed the strength of the design and data
7 Estimate costs and benefits where appropriate
Collect design, delivery, follow-up, and participant time costs within a stated accounting scope. Do not convert every improvement into money; separate verified financial benefit from service quality or risk reduction that has not been priced. Distinguish time freed up from actual cash savings. If a benefit attributable to the intervention can be estimated, show its assumptions, time horizon, and sensitivity, and prevent double counting of the same benefit.
- Suggested owner
- Evaluation, finance, and outcome owner
- Required output
- A cost and benefit statement and estimation limitations
- Progression check
- Every monetary value has a calculation basis and a documented attribution share
8 Turn results into a decision and follow it up
Write a report beginning with the question, finding, and proposed decision, followed by evidence and limitations. Distinguish content problems, delivery problems, and workplace environment problems, as each requires a different remedy. Specify an action, owner, and review date, then verify implementation in the next cohort. Retain versions of measurement tools, definitions, and calculations, and share an understandable summary with participants that protects privacy and explains how the organization used their contribution.
- Suggested owner
- Decision-maker and training director
- Required output
- A decision report and improvement plan
- Progression check
- Evaluation does not end with a report that has no owner responsible for using its findings
Guide and editable templates: https://uhrda.net/en/organizations/guides/impact-evaluation/
4. Working templates
Review the illustrative row, then complete a copy of the blank row using your organization’s information. Download each template as CSV or print it within the guide.
Evaluation questions matrix
Complete before delivery; the first row is a hypothetical example and the second is blank. Set timing according to data availability and the opportunity to apply learning.
| Evaluation question | Evidence | Source | Collection time | Related decision |
|---|---|---|---|---|
| Has the triage skill transferred to work? | A case sample assessed using a common rubric | Supervisor and case register | After sufficient opportunity to practice | Additional support or adjustment of training cases |
Change interpretation register
Helps review alternatives before attributing an outcome to training. Record supporting and opposing evidence and express uncertainty clearly.
| Observed change | Possible explanation | Available evidence | Missing evidence | Conclusion wording |
|---|---|---|---|---|
| Fewer triage errors | Training alongside an updated form | Change date and error sample | A suitable comparison group | Improvement followed two interventions; their contributions cannot currently be separated |
5. Hypothetical example: evaluating request-processing training
A hypothetical organization trained one group and followed another that had not yet started. Before the intervention, the error rate was 12% in the trained group and 10% in the comparison group, with task similarity and prior trends needing examination.
- The evaluator used the same error definition and equal periods, and recorded case volumes.
- After implementation, the rate was 7% in the trained group and 8% in the comparison group.
- Concurrent changes and group differences were reviewed, and samples demonstrating application were collected.
Interpreting the result
In the educational calculation, the trained group improved by five percentage points and the comparison group by two; the difference in changes is therefore three percentage points in favor of the trained group. This is a hypothetical estimate conditional on a valid comparison and assumptions, rather than automatic proof of causal impact or an actual Union result.
Learning from the case
Subtracting the comparison group's change helps interpret change, but does not by itself address every selection bias or difference in circumstances.
6. Measures and follow-up
Agree each definition, data source, baseline, target and accountable owner before measurement. Targets require organizational approval. A zero denominator makes a ratio not calculable rather than zero.
| Measure | Calculation | Source | Review frequency |
|---|---|---|---|
| Change in learning results | Average post-score minus average pre-score for the same people on an equivalent scale; report in points with the number of pairs | Pre- and post-tests or tasks identified by version | For each cohort |
| Verified workplace application | Number meeting the application standard ÷ number assessed after an opportunity to practice × 100; those without an application opportunity must not be excluded without disclosure | Work samples and assessor checklist | After an application period appropriate to the task |
| Follow-up coverage | Number of participants with valid follow-up data ÷ total participants targeted for follow-up × 100; explain reasons for missing data | Participant and follow-up register | With each application report |
| Difference in performance indicator changes | Change in the trained group minus change in the comparison group, where change = post-value minus pre-value; interpret the direction of improvement according to the indicator | Comparable performance records and a documented comparison design | At the end of the comparison period |
| Estimated financial return when its conditions are met | (Financial benefit attributable to the intervention minus its total cost) ÷ its total cost × 100; do not calculate where cost is zero or attribution cannot be defended | Financial data, documented benefit assumptions, and sensitivity analysis | At the end of the specified benefit horizon |
7. Indicative implementation plan
The periods below are adaptable planning stages, not guaranteed delivery times.
| Period | Action | Deliverable |
|---|---|---|
| Phase one: before delivery | Define the decision, intervention logic, and design, and collect the baseline | An evaluation plan, tools, sources, and responsibilities |
| Phase two: during the program and at its end | Measure learning and delivery quality, and document missing data and observations | A learning report and immediate improvements |
| Phase three: after sufficient opportunity to apply learning | Follow up on behavior, results, costs, and alternative explanations | An impact report with clear limitations and a decision; timing is adapted rather than imposed as a standard |
8. Common pitfalls
Treating satisfaction as evidence of impact
Collect separate evidence of learning, application, and outcomes.
Attributing every improvement to training
Review concurrent changes and comparison strength, and use conditional language.
Ignoring participants lost to follow-up
Show coverage, reasons for missing data, and their possible effects on results.
Assessing application before an opportunity exists
Link timing to the work cycle and record who has not practiced the task.
Overstating financial savings
Separate time freed up from cash savings, and document attribution, costs, and sensitivity.
9. Review checklist
- □ The decision the evaluation will support is defined.
- □ Outcomes, definitions, and the baseline are fixed.
- □ Tools have been piloted and their suitability for the task reviewed.
- □ An application opportunity is available and follow-up coverage is documented.
- □ Alternative explanations and attribution limits have been examined.
- □ The report includes actions, owners, and review dates.
Use this checklist at the final review. Record observations, the decision owner, approval date and document version.
10. Further reading
Original practical material prepared for the Arab Union for Human Resources Development. Sources support the concepts and further reading; the illustrative models and cases belong to this guide. Reviewed: 16 October 2026.
- CDC — Evaluate Training: Measuring Effectiveness
A reference for distinguishing learning from its transfer to work, the timing of their measurement, and the limitations of a post-test alone.
- CDC — Program Evaluation Framework: Step 3, Focus the Evaluation Questions and Design
A reference for identifying who will use findings and defining evaluation purpose, questions, and a design proportionate to context and resources.
- CDC — Evaluate Training: Building an Evaluation Plan
A general reference for early planning and linking evaluation questions to a program improvement decision.