Programs—Caveats and Concerns
Even when a program is created, its design may differ from implementation. Flaws in either case harbor implications for the future: Will the program thrive long enough to be institutionalized locally, adapted and transferred to similar populations in other sites, and perhaps scaled to serve different populations and contexts? “Sustainability” is often used as a test of impact. If so, then we tend to call the program a “success;” if it cannot survive beyond its original outside funding and leverage other resources to continue with succeeding cohorts of participants, the program fades and is considered a failure.
Of course, such ideal-types are rigid and beg many questions, including differing expectations, measures of impact and success, and the role of program leaders and practices. In short, all programs—even the best ones—evolve. Can we capture their “life cycle,” identify similarities and differences in program content and sequence, distinguish intentions (design) from behavior (implementation) to evaluation (measurement)?
What does the program do? What is its core character and purpose? Any program should cite evidence that supports whatever is planned as a means of advancing the policy or mission of the organization offering it, and should be tailored to accomplish that goal.
How the program is executed on behalf of the served population is a translation of grand plans into on-the-ground delivery of services.
Following BEST as a template, we can employ a small set of criteria as a “wish list” for judging empirically what a promising program should entail. Those criteria would include
• the form of intervention (typically more than one kind of activity) designed to produce a desired outcome;
• a specified target population;
• a track record of minimally five years of operation;
• evidence of positive outcomes (ideally documented through third-party evaluation or a research study, preferably with a comparison group); and
• findings that inform the operation of similar programs.
If all of these criteria are viewed as requirements for what constitutes a promising program, and not as a menu from which program organizers can pick-and-choose, then we conclude that few programs qualify. In the BEST population, less than 10 percent of the nominated programs passed muster as “promising” or “exemplary” according to the expert review panel. Clearly, funding and leadership are keys to longevity. However, the environment must be receptive to moving a “soft-money” project into an organization’s operating budget. This transition signals that a marginal effort by a few contributes to the mainstream of the organization’s mission, thereby warranting “hard money” support.
Bias and Barriers: Fulfilling the Potential of Women in Academic Science and Engineering. Washington, D.C.: The National Academies Press. On minorities in science, see National Research Council. 2011. Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. Washington, D.C.: The National Academy Press.