finds a much smaller difference, policymakers are at a loss to understand what assumptions inside the "black box" create these conflicting results. But when prototypical resource models of adequacy have different costs, it is clear what the reasons for this difference are: one professional judgment model may propose a 3rd-grade class size of 15 students, for example, while another may propose a class size of 20. Policymakers, educators, and voters can then enter this debate, exercising their own best judgments about whether the research evidence on the benefits of smaller class sizes in the early grades is sufficiently persuasive to justify the additional cost. As this research evidence advances, the professional judgment method will be able to improve the precision of its results.

For now, the professional judgment method makes explicit what more statistical methodologies tend, unwittingly, to hide: As the Rhode Island Supreme Court stated in a recent adequacy decision, "what constitutes an 'equal, adequate, and meaningful' [education] is 'not likely to be divined for all time even by the scholars who now so earnestly debate the issues'" (quoted in Minorini and Sugarman, Chapter 6 in this volume).

Nevertheless, even after having expressed our preference for the professional judgment approach, principally because of limitations inherent in black box statistical approaches and the limitations of many standardized measures of student outcomes, we acknowledge that the professional judgment approach is itself imperfect. It suffers from the possibility of not being reliable. Will double panels arrive at similar conclusions when provided with similar information? In the Wyoming circumstances described below, two panels, operating 6 months apart, did arrive at similar judgments. However, this is a sample size far too small to arrive at a conclusion regarding the utility of the professional judgment process in every setting.

Also, when professional panels make judgments, there is always the risk of conflicts of interests. This would be so whether the panels include physicians, attorneys, engineers, or educators. "Adequate" may be defined by panel members with an eye toward their own circumstances and the personal consequences for them of arriving at particular "adequate" revenue levels. And, of course, one cannot be absolutely sure of the basis of their professional judgment in such matters. In effect, each professional panel participant is his or her own "black box," the internal machinations of which are not immediately transparent.

In the following pages, we describe the Wyoming professional judgment approach, to illustrate how our methods unfold in practice.

Establishing A Definition Of Adequate By Professional Judgment In Wyoming

Guthrie et al., the Wyoming consultants, operated with two waves of independent panels of education experts. The first wave was comprised of Wyoming education experts. Its principal objective was to identify the components of an



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement