Independent archive. Not affiliated with the original program, its host organization or its funder. Material is kept for reference and describes the program as it was when each page was written.

Disability Research Library Archive

What Are the Standards for Quality Research?

This technical brief discussed what makes research high in quality, how such standards are set, and how reporting can be standardized. It separated quality of research from quality of evidence and noted that disagreement over standards is common across health, education, disability and social welfare.

Research quality versus evidence quality

Research quality concerns the scientific process: fit between methods and questions, choice of participants, measurement of outcomes, and protection against bias and faulty inference. Evidence quality is a judgment about how much confidence a whole body of findings deserves. Three criteria commonly used to rate a body of evidence are the aggregate quality of the studies, their quantity (number, sample size, power and effect size), and the consistency of results across studies and designs. Good research usually comes before good evidence, and a solid evidence base normally needs more than one study, though a rare study can be convincing on its own.

Commonly cited standards

Qualitative work is often judged by parallel concepts: credibility, transferability, dependability and confirmability. Frequently mentioned standards for research in general include:

  • asking an important question that can be studied empirically and that is linked to theory;
  • using methods that suit the question and reasoning supported by the literature;
  • giving enough detail to replicate the study, including samples, interventions and comparison groups;
  • measuring variables reliably and appropriately;
  • weighing alternative explanations and possible bias;
  • undergoing peer review and following reporting standards.

No single formula exists, and meeting just one or two standards is not enough. Publication in a leading journal and citation counts were described as imperfect signals, since citation measures mostly reflect quantity.

How standards are set

Standards rest on the principles of scientific inquiry, the philosophy of science, and above all consensus among scholars. Consensus is pursued through position statements, conferences, peer review and systematic review. Examples included a multidisciplinary conference convened by a European research organization, criteria developed by divisions of a professional psychology association, and a national committee advising on evaluation of government-funded research.

Standardized reporting

Published studies often omit essential details, which makes quality hard to assess. Checklists address this and go beyond the basic introduction, methods, results and discussion format. Named examples were CONSORT for randomized trials, QUOROM for systematic reviews, MOOSE for observational meta-analyses, TREND for nonrandomized designs and STARD for diagnostic accuracy studies, plus others for specific specialties. Checklists are not evaluation tools, but their use has been linked to better reporting. Some critics find checklists too prescriptive for qualitative research.

Why it matters for knowledge translation

Research quality and evidence must be judged sufficient before dissemination and utilization efforts begin. Debate continues in disability and rehabilitation research about which criteria and checklists to adopt, and existing public resources can inform that discussion.

About this summary: this page condenses a longer original publication and does not reproduce it.

This page is a summary. Related material is available from the content areas page.