Appendix D: Quality of Execution/Reporting Instrument

(1) Instrument






Overall Quality/Execution Rating

(1-3= Weak; 4-7 = Moderate; 8-10 = Strong)














(2) Guidelines for Reviewers:

A. Population and sampling

  • (1) Sampling method and/or sampling frame were described.  Sample was selected using appropriate methods; control and intervention group well matched, if applicable; sample is not voluntary)
  • (1) Characteristics of population were described in detail (age, sex, ethnicity/race, socioeconomic status)
  • (1) Sample was not compromised by very low participation rate or high refusal/drop-out rate (>25%)

B. Description of exposure and measurement

  • (1) Characteristics of intervention were described in detail (duration, direction and frequency of contact (uni-directional v. bi-directional, human resources, system features and functionality)
  • (1) Study participants’ exposure to variable is described and placed in context (i.e., authors note whether intervention is part of a larger program or delivered in isolation; authors note whether some participants used the system more than others and attempt to take this into account).
  • (1) Evaluators were blinded to intervention

C. Data collection and analysis

  • (1) Use of validated surveys or scales. Reliability of data collection instruments is described.
  • (1) Appropriate statistical methods for analyzing data.

D. Interpretation of results

  • (1) Authors describe the limitations of the study at a reasonable level of detail and identify confounding variables.
  • (1) Authors’ conclusions follow logically from their findings and alternate explanations are considered.

Scoring: /10

Leave a Reply