Utilization Measurement: Focusing on the "U" in "D&U"
This special report argued that disability researchers planning dissemination and utilization should give as much attention to the second half, how research outcomes are actually used, as to the first. It introduced utilization-focused evaluation as a practical way to learn how audiences apply findings, and it laid out five planning steps.
Dissemination versus utilization
Dissemination mainly concerns knowledge transfer: who, what, when and how information moves from its sources to recipients. Utilization concerns purpose and impact: why people should receive the outcomes, what use is intended, and how they are in fact used. Focusing only on dissemination was compared to dialing nine digits of a ten-digit number. Evaluation is the tool that shows whether information reached people and whether they applied it.
Basic ideas about evaluation
- Evaluation is more than compliance. Its goal is improvement, so project staff should help design and carry it out instead of leaving it to a separate department.
- Evaluation is not the same as research. It aims to support action, though it still needs reliable data, which may come from monitoring statistics as well as research methods.
- Good evaluation meets four standards: utility, feasibility, propriety and accuracy.
- No single best method exists, since each situation is unique and utility should not come second to methodology.
- Use has two dimensions: how much people use outcomes, and how well, from effective use to misuse. Intended use and effective use can differ, because consumers sometimes adapt findings in ways researchers never foresaw.
Steps in planning
- Identify the intended users of the evaluation. Stakeholders are many, so the list is narrowed to primary users, and the presence of people who personally care about the findings strongly predicts whether results get used.
- Identify the purpose: judging merit, improving the work, or generating knowledge. Mixing purposes without priorities causes confusion.
- Specify the questions to answer, such as who is using the outcomes, which media worked best, how uses compare with intentions, what went unexpectedly, and where efficiencies lie.
- Specify criteria. Goals stated as activities are weak; clear statements of intended change among users work better, with indicators that are reasonable and meaningful. Question-driven or goal-free approaches suit improvement-oriented work.
- Choose methods with intended users involved, with the evaluator acting as technical adviser. Quality standards for methods depend on context.
Conclusions
Adopting this approach means more work and a change in thinking. The report listed common diversions, such as letting evaluators make all decisions, naming vague audiences instead of real people, waiting until findings arrive to identify users, and ignoring intended users after the start.
About this summary: this page condenses a longer original publication and does not reproduce it.
This page is a summary. Related material is available from the content areas page.