program designs. Since the total effect of these organizations is amplified across its affiliates, experimental evaluation with random assignment helps illuminate how effective these programs are.

The final context for comprehensive experimental evaluation is when some bold new idea for a new kind of service surfaces and critical examination shows that the substantive theory behind the idea is reasonable and that it is indeed likely to be able to be implemented. This situation is often called a demonstration project. Such demonstrations provide the substantive new ideas out of which the next generation of superior services is likely to emerge. As such, they deserve to be taken very seriously and to be evaluated by rigorous experiments.

Programs that meet the following criteria should be studied on a regular, ongoing basis with a variety of either nonexperimental methods or more focused experimental, quasi-experimental and interrupted time-series designs, such as those advocated by Box (1958):

1. An organization, program, project, or program element has not sufficiently matured in terms of its philosophy and implementation;

2. The evaluation has to be conducted by the staff of the program under evaluation;

3. The major questions of interest pertain to the quality of the program theory, implementation of that theory, or the nature of its participants, staff, or surrounding context;

4. The program is quite broad, involving multiple agencies in the same community; and

5. The program or organization is interested in reflective practice and continuing improvement.

An explanation of the reasons why a program is effective is important because it identifies the processes that are thought to be present for effectiveness to occur (Cook, 2000). This knowledge is crucial for replication of program effects at new sites because of the uniqueness inherent in delivering the program to new populations and in new settings. Causal or explanatory knowledge not only identifies the critical components of program effectiveness, but also specifies whether these components are moderator variables (variables that change the relation between an intervention and an outcome; common moderator variables include all types