Repeat every 5 years or more infrequently:

  • Set criterion weights (this step requires a panel, as discussed below).

Repeat at least every 3 years:

  • Solicit nominations of candidate conditions and technologies.

  • Reduce the list of nominations through the "winnowing" procedure laid out in Chapter 4 and Appendix 4.1.

  • Obtain the data required for the objective criteria.

  • Review the objective data and decide what will be used to calculate the priority score (this step requires a panel). Establish the subjective criterion scores (this step requires a panel).

  • Calculate the priority score.

The next section presents key points about setting criterion weights. Later, the chapter discusses critical concerns regarding the remaining activities in the context of resources needed to implement the process and, more specifically, the responsibilities of the priority-setting panel.


The criterion weights mentioned above in the priority-setting cycle and examined in Chapter 4 are intended to represent the preferences of society. The committee envisions a broadly constituted panel that would set criterion weights not oftener than every 5 years. Once OHTA has established the weight-setting system, it should test and establish its reliability; then it could repeat the procedure only infrequently. Although the committee sees this weighting task as a group process, it might be accomplished by some other means (e.g., voting by mail), if those means were shown to be reliable. Although AHCPR's National Advisory Council might function as this weight-setting panel, the committee suggests that a separate group be constituted for this and subsequent panel tasks, in part because the task requires a particular array of expertise, but also because the workload could be considerable.

Apart from setting the criteria weights, the committee sees the priority-setting cycle as occurring every 2 to 3 years, but not less frequently than every 3 years, because of the current pace of technological change. The time that elapses before repeating the process would depend not on a fixed interval but on how many assessments have been completed.

The number of assessments—as opposed to the number of conditions and technologies that the quantitative model can rank—will depend principally

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement