Theoretically sound ideas for a new demonstration program or project emerge, and pilot work indicates that these ideas can be implemented in other contexts.
Comprehensive experimental evaluation is useful when the focus of evaluation is on elements that can be found in many organization providing services to youth: how recruitment should be done and continued participation should be supported; how youth and parent involvement should be supported; how youth’s maturity and growing expertise should be recognized and incorporated into programming; how staff members should be recruited and then trained; how coordination with schools should be structured; how recreational and instructional activities should be balanced; how mentoring should be carried out; how service learning should be structured and supported; and how doing homework should be supported. Typically, these are components in any single organization, but they are extremely important because they are common issues in organizations across the country. In our view, such elements are best examined by way of experimental research in which a sample of organizations is assigned to different ways of solving the problem identified. Although such work has the flavor of basic research on organizational development, it is central to an effectiveness-based practice of community youth programming.
Equally critical to an effectiveness-based practice approach is knowledge of the individual- and community-level factors that influence the effectiveness of these practices for different groups of individuals or communities. Consequently, it is also important that experimental methods be used to assess the replicability of such practices in different communities and with different populations of youth. Particular attention here needs to be paid to such individual and community differences as culture, age and maturity, sex, disability status, social class, educational needs, and other available community resources.
Comprehensive experimental evaluations are also called for in two other contexts. The first is when the target of evaluation is a national organization that has affiliates in many locations across the United States. The best model of this is the evaluation completed on Big Brothers Big Sisters to assess the effects of membership and of participation across the affiliates included in the sampling design. Many of these national organizations have been providing services to youth for many years and carry a disproportionate burden of current service provision. As a result, even when programs are still developing at the margins, many have mature